From jeanluc.menut at free.fr  Wed Jun  1 11:01:26 2011
From: jeanluc.menut at free.fr (Jean-Luc Menut)
Date: Wed, 01 Jun 2011 17:01:26 +0200
Subject: [AstroPy] modifying  FITS data
In-Reply-To: <4DE51499.9060800@stsci.edu>
References: <4DE50103.8010300@free.fr> <20110531160120.GA12108@free.fr>
	<4DE51499.9060800@stsci.edu>
Message-ID: <4DE65446.4070106@free.fr>

Le 31/05/2011 18:17, Erik Bray a ?crit :
> orig['OI_VIS'].data['VISAMP'][:] = the_value

Thanks a lot to you and Fr?d?ric.

I actually must type :
orig['OI_VIS'].data.field('VISAMP')[:] = the_value

But it works.

Thanks again

Jean-Luc


From taro at ap.smu.ca  Wed Jun  1 11:49:57 2011
From: taro at ap.smu.ca (Taro Sato)
Date: Wed, 1 Jun 2011 12:49:57 -0300
Subject: [AstroPy] modifying FITS data
In-Reply-To: <4DE65446.4070106@free.fr>
References: <4DE50103.8010300@free.fr> <20110531160120.GA12108@free.fr>
	<4DE51499.9060800@stsci.edu> <4DE65446.4070106@free.fr>
Message-ID: <BANLkTimP+dsvyz1RJurFqTYvZzYPsjKAOw@mail.gmail.com>

On Wed, Jun 1, 2011 at 12:01 PM, Jean-Luc Menut <jeanluc.menut at free.fr> wrote:
> Le 31/05/2011 18:17, Erik Bray a ?crit :
>> orig['OI_VIS'].data['VISAMP'][:] = the_value
>
> Thanks a lot to you and Fr?d?ric.
>
> I actually must type :
> orig['OI_VIS'].data.field('VISAMP')[:] = the_value
>
> But it works.


I used to have related issues and what Jean-Luc wrote here is the solution.

This actually makes me wonder why PyFITS doesn't allow a "standard"
scheme of updating values in a named array.

Sometimes, I want to write a simple function or something that takes
in a numpy named array and do some processing.  And reuse it for the
array read via PyFITS.   Since the "right" way of accessing the values
in PyFITS bintable is to use the field method, I often need to write a
wrapper to take care of those cases separately.  Or convert the array
to a standard numpy array before feeding to the function.

Is there any reason PyFITS require us to use the field method?  I
guess I should make a feature request.

Cheers,
Taro


From taro at ap.smu.ca  Wed Jun  1 12:22:41 2011
From: taro at ap.smu.ca (Taro Sato)
Date: Wed, 1 Jun 2011 13:22:41 -0300
Subject: [AstroPy] modifying FITS data
In-Reply-To: <4DE6652E.9010307@stsci.edu>
References: <4DE50103.8010300@free.fr> <20110531160120.GA12108@free.fr>
	<4DE51499.9060800@stsci.edu> <4DE65446.4070106@free.fr>
	<BANLkTimP+dsvyz1RJurFqTYvZzYPsjKAOw@mail.gmail.com>
	<4DE6652E.9010307@stsci.edu>
Message-ID: <BANLkTimx4cDXSDaYCYhDq_Q6DJLsP1ekhA@mail.gmail.com>

On Wed, Jun 1, 2011 at 1:13 PM, Erik Bray <embray at stsci.edu> wrote:
> On 06/01/2011 11:49 AM, Taro Sato wrote:
>>
>> On Wed, Jun 1, 2011 at 12:01 PM, Jean-Luc Menut<jeanluc.menut at free.fr>
>> ?wrote:
>>>
>>> Le 31/05/2011 18:17, Erik Bray a ?crit :
>>>>
>>>> orig['OI_VIS'].data['VISAMP'][:] = the_value
>>>
>>> Thanks a lot to you and Fr?d?ric.
>>>
>>> I actually must type :
>>> orig['OI_VIS'].data.field('VISAMP')[:] = the_value
>>>
>>> But it works.
>>
>>
>> I used to have related issues and what Jean-Luc wrote here is the
>> solution.
>>
>> This actually makes me wonder why PyFITS doesn't allow a "standard"
>> scheme of updating values in a named array.
>>
>> Sometimes, I want to write a simple function or something that takes
>> in a numpy named array and do some processing. ?And reuse it for the
>> array read via PyFITS. ? Since the "right" way of accessing the values
>> in PyFITS bintable is to use the field method, I often need to write a
>> wrapper to take care of those cases separately. ?Or convert the array
>> to a standard numpy array before feeding to the function.
>>
>> Is there any reason PyFITS require us to use the field method? ?I
>> guess I should make a feature request.
>
> I'm not sure I understand. ?Are you saying mapping-like indexing should be
> possible on tables? ?(i.e. data['COLUMN_NAME']) ?Because that is already
> implemented in PyFITS 2.4.0 and up. ?data['COLUMN_NAME'] is exactly the same
> as data.field('COLUMN_NAME').
>
> Erik
>
>


Oh I see... it's been changed...  I definitely had the issue with an
older version of PyFITS.  Thanks for letting me know.


-- 
Taro Sato
Department of Astronomy & Physics
Saint Mary's University? ? ? ? ? ?? email: taro at ap.smu.ca
Halifax, NS B3H 3C3? ? ? ? ? ? ? ?? phone: (902)420-5027
Canada? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? web: http://ap.smu.ca/~taro


From erik.tollerud at gmail.com  Wed Jun  1 21:24:02 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Wed, 1 Jun 2011 18:24:02 -0700
Subject: [AstroPy] Q re. function minimization
In-Reply-To: <4DE4FAA4.6020604@ucla.edu>
References: <4DE4FAA4.6020604@ucla.edu>
Message-ID: <BANLkTi=jMOPiHvng_A4Tdue3EChh=_Q56g@mail.gmail.com>

I've found that writing wrapper class for scipy.optimize can do all
the things you're asking about.  pymodelfit
(http://packages.python.org/PyModelFit/) does exactly that for the
model-data minimization problem.  That may not be the sort of
minimization you want, but the source code for that might be helpful
if you need to roll your own functions or classes to do this.



On Tue, May 31, 2011 at 7:26 AM, Ian Crossfield <ianc at ucla.edu> wrote:
> What functions or wrappers do people use for minimizing a function in
> multidimensional parameter space? ?I have some experience with the
> scipy.optimize subfunctions but I've never been entirely happy with
> them; there also appear to be options in sage and a Python MPFIT. ?What
> I'd like is a method that lets one minimize a function but also allows
> things like passed keyword arguments (unsupported in scipy.optimize) and
> above all a way to minimize while holding specified parameters fixed --
> so that one can minimize over all parameters, and then repeat with some
> parameters held constant without having to write an entirely new
> function for each peculiar set of parameters.
>
> Is anything that offers these features this out there? ?How do people
> deal with this?
>
> Thanks,
>
> --
> Ian Crossfield
> UCLA Astronomy
> KH 3-145J
> http://www.astro.ucla.edu/~ianc/
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 
Erik Tollerud


From wkerzendorf at googlemail.com  Thu Jun  2 00:13:22 2011
From: wkerzendorf at googlemail.com (Wolfgang Kerzendorf)
Date: Thu, 02 Jun 2011 14:13:22 +1000
Subject: [AstroPy] Q re. function minimization
In-Reply-To: <4DE4FAA4.6020604@ucla.edu>
References: <4DE4FAA4.6020604@ucla.edu>
Message-ID: <4DE70DE2.3060908@gmail.com>

I can recommend pyminuit that was mentioned by Neil. I don't even find 
it flaky and you can fix parameters and do three different types of 
optimization.

Cheers
   W
On 1/06/11 12:26 AM, Ian Crossfield wrote:
> What functions or wrappers do people use for minimizing a function in
> multidimensional parameter space?  I have some experience with the
> scipy.optimize subfunctions but I've never been entirely happy with
> them; there also appear to be options in sage and a Python MPFIT.  What
> I'd like is a method that lets one minimize a function but also allows
> things like passed keyword arguments (unsupported in scipy.optimize) and
> above all a way to minimize while holding specified parameters fixed --
> so that one can minimize over all parameters, and then repeat with some
> parameters held constant without having to write an entirely new
> function for each peculiar set of parameters.
>
> Is anything that offers these features this out there?  How do people
> deal with this?
>
> Thanks,
>



From gr.maravelias at gmail.com  Fri Jun  3 17:17:09 2011
From: gr.maravelias at gmail.com (Grigoris Maravelias)
Date: Sat, 04 Jun 2011 00:17:09 +0300
Subject: [AstroPy] upgrade matplotlib within scisoft
Message-ID: <4DE94F55.7040006@gmail.com>

Hello list !

I am a newbie in python/pylab stuff but I really enjoy the advantages of 
these tools! I would like to ask about a puzzling problem that I try to 
search and fix for some days now. I found out that matplotlib1.0.1 can 
keep a window open so as to plot continuously in the same window. I have 
made some scripts that take advantage of the iraf/pyraf inside 
scisoft7.5 (in fedora 14) but there the matplotlib0.99 version is 
included which can plot only once in a script. How can I upgrade to the 
matplotlib1.0.1?

Perhaps the answer is obvious but I am not able to see it...

Thanks,

Grigoris


From marquett at iap.fr  Sun Jun  5 09:59:11 2011
From: marquett at iap.fr (Jean-Baptiste Marquette)
Date: Sun, 5 Jun 2011 23:59:11 +1000
Subject: [AstroPy] Oblique rectangles in APLpy
In-Reply-To: <F8C6429468E840919F2319546E3329FE@gmail.com>
References: <BEE6840A-DD5A-4635-98AF-A3F828D9F9A4@iap.fr>
	<F8C6429468E840919F2319546E3329FE@gmail.com>
Message-ID: <217F3CB2-82F1-4980-A2A0-2AEA49F1CE57@iap.fr>

Hi Tom,

I don't find the correct syntax for the coordinates of the corners.

        Polygon = [[RaMin, DecMin], [RaMin, DecMax], [RaMax, DecMin], [RaMax, DecMax]]

doesn't work.

Please give an example.

Cheers
JB

Le 31 mai 2011 ? 02:57, Thomas Robitaille a ?crit :

> Hi Jean-Baptiste,
> 
> The easiest way to do this would be to figure out the coordinates of the corners of the rectangles in Galactic coordinates, and use show_polygons to plot them. Note that you can use the fk52gal function in aplpy.wcs_util to facilitate the conversion from J2000 to Galactic coordinates. Let me know if this does not work!
> 
> Cheers,
> Tom
> 
> 
> 
> On Monday, May 30, 2011 at 6:51 AM, Marquette Jean-Baptiste wrote:
> 
>> Dear APLpy gurus,
>> 
>> I have observational rectangle fields with J2000 coordinates I would
>> like to plot on images in galactic coordinates. But the
>> show_rectangles routine doesn't allow to plot those rectangles
>> obliquely.
>> Any hint welcome.
>> 
>> Cheers
>> Jean-Baptiste Marquette
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>> http://mail.scipy.org/mailman/listinfo/astropy
> 
> 




From thomas.robitaille at gmail.com  Sun Jun  5 10:11:17 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Sun, 5 Jun 2011 10:11:17 -0400
Subject: [AstroPy] Oblique rectangles in APLpy
In-Reply-To: <217F3CB2-82F1-4980-A2A0-2AEA49F1CE57@iap.fr>
References: <BEE6840A-DD5A-4635-98AF-A3F828D9F9A4@iap.fr>
	<F8C6429468E840919F2319546E3329FE@gmail.com>
	<217F3CB2-82F1-4980-A2A0-2AEA49F1CE57@iap.fr>
Message-ID: <088891E5128742AB908D8AACBA840688@gmail.com>

Hi JB,

The polygon has to be expressed as a 2xN numpy array:

polygon = np.array([[RaMin, RaMin, RaMax, RaMax],[DecMin, DecMax, DecMax, DecMin]])

and show_polygons takes a list of polygons

f.show_polygons([polygon])

Let me know if this doesn't work for you

Cheers,
Tom


On Sunday, June 5, 2011 at 9:59 AM, Jean-Baptiste Marquette wrote:

> Hi Tom,
> 
> I don't find the correct syntax for the coordinates of the corners.
> 
>  Polygon = [[RaMin, DecMin], [RaMin, DecMax], [RaMax, DecMin], [RaMax, DecMax]]
> 
> doesn't work.
> 
> Please give an example.
> 
> Cheers
> JB
> 
> Le 31 mai 2011 ? 02:57, Thomas Robitaille a ?crit :
> 
> > Hi Jean-Baptiste,
> > 
> > The easiest way to do this would be to figure out the coordinates of the corners of the rectangles in Galactic coordinates, and use show_polygons to plot them. Note that you can use the fk52gal function in aplpy.wcs_util to facilitate the conversion from J2000 to Galactic coordinates. Let me know if this does not work!
> > 
> > Cheers,
> > Tom
> > 
> > 
> > 
> > On Monday, May 30, 2011 at 6:51 AM, Marquette Jean-Baptiste wrote:
> > 
> > > Dear APLpy gurus,
> > > 
> > > I have observational rectangle fields with J2000 coordinates I would
> > > like to plot on images in galactic coordinates. But the
> > > show_rectangles routine doesn't allow to plot those rectangles
> > > obliquely.
> > > Any hint welcome.
> > > 
> > > Cheers
> > > Jean-Baptiste Marquette
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > http://mail.scipy.org/mailman/listinfo/astropy




From xavier.gnata at gmail.com  Tue Jun  7 10:44:34 2011
From: xavier.gnata at gmail.com (Xavier Gnata)
Date: Tue, 7 Jun 2011 16:44:34 +0200
Subject: [AstroPy] Plan to port astropy to python3?
Message-ID: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>

Hi,

well everything is mainly in the title.
numpy,scipy,matplotlib and pyfits are now working with python3.

Is there already a plan to port pywcs to python3 ?

Xavier


From perry at stsci.edu  Tue Jun  7 11:15:25 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 7 Jun 2011 11:15:25 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
Message-ID: <E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>

That's something we are thinking about now. So far we haven't seen any  
major shift by science users in general to Python 3, so if there are  
significant efforts out there that wish to use Python 3, let us know.  
If we don't hear of any, we may delay future Python 3 ports until the  
user base is growing. It is more likely we will support it with more  
general libraries such as pywcs than with our applications.

Perry

On Jun 7, 2011, at 10:44 AM, Xavier Gnata wrote:

> Hi,
>
> well everything is mainly in the title.
> numpy,scipy,matplotlib and pyfits are now working with python3.
>
> Is there already a plan to port pywcs to python3 ?
>
> Xavier
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From William.T.Bridgman at nasa.gov  Tue Jun  7 11:22:59 2011
From: William.T.Bridgman at nasa.gov (Bridgman, William T.)
Date: Tue, 7 Jun 2011 11:22:59 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
Message-ID: <EF6AE4D7-BE19-45AA-BF1A-80B055F8464C@nasa.gov>

This is the first I've heard that my main components of interest are  
ALL working on Python 3.

Which means I will probably try a test installation of Python 3 on one  
of my systems this summer.

I want to make more use of threading on multi-core systems since  
graphical rendering is a lot of 'embarrassingly parallel' type  
processing.
http://en.wikipedia.org/wiki/Embarrassingly_parallel

Tom

On Jun 7, 2011, at 11:15 AM, Perry Greenfield wrote:

> That's something we are thinking about now. So far we haven't seen any
> major shift by science users in general to Python 3, so if there are
> significant efforts out there that wish to use Python 3, let us know.
> If we don't hear of any, we may delay future Python 3 ports until the
> user base is growing. It is more likely we will support it with more
> general libraries such as pywcs than with our applications.
>
> Perry
>
> On Jun 7, 2011, at 10:44 AM, Xavier Gnata wrote:
>
>> Hi,
>>
>> well everything is mainly in the title.
>> numpy,scipy,matplotlib and pyfits are now working with python3.
>>
>> Is there already a plan to port pywcs to python3 ?
>>
>> Xavier
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

--
Dr. William T."Tom" Bridgman               Scientific Visualization  
Studio
Global Science & Technology, Inc.          NASA/Goddard Space Flight  
Center
Email: William.T.Bridgman at nasa.gov         Code 610.3
Phone: 301-286-1346                        Greenbelt, MD 20771
FAX:   301-286-1634                        http://svs.gsfc.nasa.gov/






From perry at stsci.edu  Tue Jun  7 11:39:48 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 7 Jun 2011 11:39:48 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <EF6AE4D7-BE19-45AA-BF1A-80B055F8464C@nasa.gov>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<EF6AE4D7-BE19-45AA-BF1A-80B055F8464C@nasa.gov>
Message-ID: <00E09ECE-3958-45FE-8973-FF1BED5820EE@stsci.edu>


On Jun 7, 2011, at 11:22 AM, Bridgman, William T. wrote:

> This is the first I've heard that my main components of interest are
> ALL working on Python 3.
>
> Which means I will probably try a test installation of Python 3 on one
> of my systems this summer.
>
> I want to make more use of threading on multi-core systems since
> graphical rendering is a lot of 'embarrassingly parallel' type
> processing.
> http://en.wikipedia.org/wiki/Embarrassingly_parallel
>
> Tom
>
Threading in Python won't necessarily exploit multi-core systems very  
well. You may want to look at other alternatives in Python.

Perry




From thomas.robitaille at gmail.com  Tue Jun  7 11:43:43 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 7 Jun 2011 11:43:43 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
Message-ID: <0430C335B65F49E6804724C6B0E8A1A4@gmail.com>

We'd be interested in getting APLpy working in Python 3, but pyfits and pywcs are crucial dependencies, so we'll have to wait until those are ported to make the move. ATpy should already work in Python 3, but FITS and VO I/O isn't supported because of the pyfits and vo dependencies. In the end, one could argue that it's up to us to support Python 3 soon, because there won't be a large astronomy user base in Python 3 until we do (otherwise it's a vicious circle). The fact that numpy/scipy/matplotlib have moved means we should probably start thinking about it too.

Cheers,
Thomas


On Tuesday, June 7, 2011 at 11:15 AM, Perry Greenfield wrote:

> That's something we are thinking about now. So far we haven't seen any 
> major shift by science users in general to Python 3, so if there are 
> significant efforts out there that wish to use Python 3, let us know. 
> If we don't hear of any, we may delay future Python 3 ports until the 
> user base is growing. It is more likely we will support it with more 
> general libraries such as pywcs than with our applications.
> 
> Perry
> 
> On Jun 7, 2011, at 10:44 AM, Xavier Gnata wrote:
> 
> > Hi,
> > 
> > well everything is mainly in the title.
> > numpy,scipy,matplotlib and pyfits are now working with python3.
> > 
> > Is there already a plan to port pywcs to python3 ?
> > 
> > Xavier
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> http://mail.scipy.org/mailman/listinfo/astropy




From perry at stsci.edu  Tue Jun  7 11:50:57 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 7 Jun 2011 11:50:57 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <0430C335B65F49E6804724C6B0E8A1A4@gmail.com>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<0430C335B65F49E6804724C6B0E8A1A4@gmail.com>
Message-ID: <CE809F57-488B-4F24-AED0-67D10A683584@stsci.edu>

I think I've heard enough to make sure that pywcs will be ported. Mike  
is on vacation for a little while and has some other work waiting for  
him when he gets back, so it may take a month or two for it to happen.

Perry

On Jun 7, 2011, at 11:43 AM, Thomas Robitaille wrote:

> We'd be interested in getting APLpy working in Python 3, but pyfits  
> and pywcs are crucial dependencies, so we'll have to wait until  
> those are ported to make the move. ATpy should already work in  
> Python 3, but FITS and VO I/O isn't supported because of the pyfits  
> and vo dependencies. In the end, one could argue that it's up to us  
> to support Python 3 soon, because there won't be a large astronomy  
> user base in Python 3 until we do (otherwise it's a vicious circle).  
> The fact that numpy/scipy/matplotlib have moved means we should  
> probably start thinking about it too.
>
> Cheers,
> Thomas
>
>
> On Tuesday, June 7, 2011 at 11:15 AM, Perry Greenfield wrote:
>
>> That's something we are thinking about now. So far we haven't seen  
>> any
>> major shift by science users in general to Python 3, so if there are
>> significant efforts out there that wish to use Python 3, let us know.
>> If we don't hear of any, we may delay future Python 3 ports until the
>> user base is growing. It is more likely we will support it with more
>> general libraries such as pywcs than with our applications.
>>
>> Perry
>>
>> On Jun 7, 2011, at 10:44 AM, Xavier Gnata wrote:
>>
>>> Hi,
>>>
>>> well everything is mainly in the title.
>>> numpy,scipy,matplotlib and pyfits are now working with python3.
>>>
>>> Is there already a plan to port pywcs to python3 ?
>>>
>>> Xavier
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>> http://mail.scipy.org/mailman/listinfo/astropy
>
>



From mrdavis at stsci.edu  Tue Jun  7 11:52:05 2011
From: mrdavis at stsci.edu (Matt Davis)
Date: Tue, 7 Jun 2011 11:52:05 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <00E09ECE-3958-45FE-8973-FF1BED5820EE@stsci.edu>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<EF6AE4D7-BE19-45AA-BF1A-80B055F8464C@nasa.gov>
	<00E09ECE-3958-45FE-8973-FF1BED5820EE@stsci.edu>
Message-ID: <21BB23F7-853B-4441-B98D-F66DAD7AF627@stsci.edu>

Python 3 is adding some new concurrency tools, something Erik brought up in a local list a while ago. To quote Erik:

Python 3.2 added a new package called 'concurrent' that has an even higher-level API built on top of the threading, multiprocessing, and other related modules.  See:

http://docs.python.org/dev/whatsnew/3.2.html#pep-3148-the-concurrent-futures-module

http://docs.python.org/dev/library/concurrent.futures.html#module-concurrent.futures

- Matt

On Jun 7, 2011, at 11:39 AM, Perry Greenfield wrote:

> 
> On Jun 7, 2011, at 11:22 AM, Bridgman, William T. wrote:
> 
>> This is the first I've heard that my main components of interest are
>> ALL working on Python 3.
>> 
>> Which means I will probably try a test installation of Python 3 on one
>> of my systems this summer.
>> 
>> I want to make more use of threading on multi-core systems since
>> graphical rendering is a lot of 'embarrassingly parallel' type
>> processing.
>> http://en.wikipedia.org/wiki/Embarrassingly_parallel
>> 
>> Tom
>> 
> Threading in Python won't necessarily exploit multi-core systems very  
> well. You may want to look at other alternatives in Python.
> 
> Perry
> 
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110607/1c60bf2e/attachment.html>

From erwin at mpe.mpg.de  Tue Jun  7 12:00:54 2011
From: erwin at mpe.mpg.de (Peter Erwin)
Date: Tue, 7 Jun 2011 18:00:54 +0200
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <00E09ECE-3958-45FE-8973-FF1BED5820EE@stsci.edu>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<EF6AE4D7-BE19-45AA-BF1A-80B055F8464C@nasa.gov>
	<00E09ECE-3958-45FE-8973-FF1BED5820EE@stsci.edu>
Message-ID: <91EF0A00-2F80-427E-AC20-A0E85D21DD60@mpe.mpg.de>


On Jun 7, 2011, at 5:39 PM, Perry Greenfield wrote:

> 
> On Jun 7, 2011, at 11:22 AM, Bridgman, William T. wrote:
> 
>> This is the first I've heard that my main components of interest are
>> ALL working on Python 3.
>> 
>> Which means I will probably try a test installation of Python 3 on one
>> of my systems this summer.
>> 
>> I want to make more use of threading on multi-core systems since
>> graphical rendering is a lot of 'embarrassingly parallel' type
>> processing.
>> http://en.wikipedia.org/wiki/Embarrassingly_parallel
>> 
>> Tom
>> 
> Threading in Python won't necessarily exploit multi-core systems very  
> well. You may want to look at other alternatives in Python.
> 


I believe the multiprocessing library (introduced in Python 2.6) is designed to
take advantage of mulitple cores (it spawns & communicates with separate processes,
rather than Python threads, which have to coexist within the parent Python
process).

(I haven't tried using it yet, though.)

=============================================================
Peter Erwin                   Max-Planck-Insitute for Extraterrestrial 
erwin at mpe.mpg.de              Physics, Giessenbachstrasse
tel. +49 (0)89 30000 3695     85748 Garching, Germany
fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin





From matthewturk at gmail.com  Tue Jun  7 12:01:48 2011
From: matthewturk at gmail.com (Matthew Turk)
Date: Tue, 7 Jun 2011 09:01:48 -0700
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <21BB23F7-853B-4441-B98D-F66DAD7AF627@stsci.edu>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<EF6AE4D7-BE19-45AA-BF1A-80B055F8464C@nasa.gov>
	<00E09ECE-3958-45FE-8973-FF1BED5820EE@stsci.edu>
	<21BB23F7-853B-4441-B98D-F66DAD7AF627@stsci.edu>
Message-ID: <BANLkTikXSu9fQBEHVGmUggFrFjnhH_=UqA@mail.gmail.com>

Hi Matt,

It's not clear yet how well the new, improved global interpreter lock
(which was improved in Python 3.2) will perform.  David Beazley has
done a number of studies of how it performs.  This is the most recent
one I could find:

http://dabeaz.blogspot.com/2010/02/revisiting-thread-priorities-and-new.html

There are interesting things going on in the Cython space, including
threading, "nogil" and OpenMP directives.  Cython is already agnostic
to 2.x versus 3.x, as well, so any extensions written in Cython should
be fine.  For process-based parallelism, mpi4py is written largely if
not completely in Cython and optimistically it will either work as is
or will be portable soon.

On the front of porting to 3.x, I spent some time porting the package
I work on (yt) to 3.0; it has a number of both hand-coded C extensions
as well as Cython-based extensions, and overall I found the experience
very straightforward.  The 2to3 tool worked as advertised, and
modifications to hand-coded C extensions such as registering functions
in the module definition, registering types and creating modules were
all relatively simple to modify.  To keep a 2- and 3-compatible, the C
extensions have to be peppered with #ifdefs, but otherwise I didn't
find it to be too big of a hassle.

-Matt

On Tue, Jun 7, 2011 at 8:52 AM, Matt Davis <mrdavis at stsci.edu> wrote:
> Python 3 is adding some new concurrency tools, something Erik brought up in
> a local list a while ago. To quote Erik:
> Python 3.2 added a new package called 'concurrent' that has an even
> higher-level API built on top of the threading, multiprocessing, and other
> related modules. ?See:
> http://docs.python.org/dev/whatsnew/3.2.html#pep-3148-the-concurrent-futures-module
>
> http://docs.python.org/dev/library/concurrent.futures.html#module-concurrent.futures
> - Matt
>
> On Jun 7, 2011, at 11:39 AM, Perry Greenfield wrote:
>
> On Jun 7, 2011, at 11:22 AM, Bridgman, William T. wrote:
>
> This is the first I've heard that my main components of interest are
>
> ALL working on Python 3.
>
> Which means I will probably try a test installation of Python 3 on one
>
> of my systems this summer.
>
> I want to make more use of threading on multi-core systems since
>
> graphical rendering is a lot of 'embarrassingly parallel' type
>
> processing.
>
> http://en.wikipedia.org/wiki/Embarrassingly_parallel
>
> Tom
>
> Threading in Python won't necessarily exploit multi-core systems very
> well. You may want to look at other alternatives in Python.
>
> Perry
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>


From William.T.Bridgman at nasa.gov  Tue Jun  7 12:06:14 2011
From: William.T.Bridgman at nasa.gov (Bridgman, William T.)
Date: Tue, 7 Jun 2011 12:06:14 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <91EF0A00-2F80-427E-AC20-A0E85D21DD60@mpe.mpg.de>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<EF6AE4D7-BE19-45AA-BF1A-80B055F8464C@nasa.gov>
	<00E09ECE-3958-45FE-8973-FF1BED5820EE@stsci.edu>
	<91EF0A00-2F80-427E-AC20-A0E85D21DD60@mpe.mpg.de>
Message-ID: <414D8B19-AAE8-4991-BBA9-028327AD6D99@nasa.gov>

Yes, I should have said 'multiprocessing' instead of 'threading'.   
I've had some successful experiments with python 2.6 & 2.7 using  
multiprocessing but my understanding is that Python 3 improves on this.

I have encountered weirdness when reading/writing .npz files in  
multiple processes but could not reliably reproduce it.

Tom
On Jun 7, 2011, at 12:00 PM, Peter Erwin wrote:

>
> On Jun 7, 2011, at 5:39 PM, Perry Greenfield wrote:
>
>>
>> On Jun 7, 2011, at 11:22 AM, Bridgman, William T. wrote:
>>
>>> This is the first I've heard that my main components of interest are
>>> ALL working on Python 3.
>>>
>>> Which means I will probably try a test installation of Python 3 on  
>>> one
>>> of my systems this summer.
>>>
>>> I want to make more use of threading on multi-core systems since
>>> graphical rendering is a lot of 'embarrassingly parallel' type
>>> processing.
>>> http://en.wikipedia.org/wiki/Embarrassingly_parallel
>>>
>>> Tom
>>>
>> Threading in Python won't necessarily exploit multi-core systems very
>> well. You may want to look at other alternatives in Python.
>>
>
>
> I believe the multiprocessing library (introduced in Python 2.6) is  
> designed to
> take advantage of mulitple cores (it spawns & communicates with  
> separate processes,
> rather than Python threads, which have to coexist within the parent  
> Python
> process).
>
> (I haven't tried using it yet, though.)
>
> =============================================================
> Peter Erwin                   Max-Planck-Insitute for Extraterrestrial
> erwin at mpe.mpg.de              Physics, Giessenbachstrasse
> tel. +49 (0)89 30000 3695     85748 Garching, Germany
> fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

--
Dr. William T."Tom" Bridgman               Scientific Visualization  
Studio
Global Science & Technology, Inc.          NASA/Goddard Space Flight  
Center
Email: William.T.Bridgman at nasa.gov         Code 610.3
Phone: 301-286-1346                        Greenbelt, MD 20771
FAX:   301-286-1634                        http://svs.gsfc.nasa.gov/






From sienkiew at stsci.edu  Tue Jun  7 14:00:58 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Tue, 07 Jun 2011 14:00:58 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <0430C335B65F49E6804724C6B0E8A1A4@gmail.com>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<0430C335B65F49E6804724C6B0E8A1A4@gmail.com>
Message-ID: <4DEE675A.3070203@stsci.edu>

Thomas Robitaille wrote:
>  In the end, one could argue that it's up to us to support Python 3 soon, because there won't be a large astronomy user base in Python 3 until we do (otherwise it's a vicious circle). The fact that numpy/scipy/matplotlib have moved means we should probably start thinking about it too.
>   


Can you tell me where to find a matplotlib release that works with 
python 3?  According to their web site, it only works in python 2:

http://matplotlib.sourceforge.net/users/installing.html#build-requirements

Mark S.



From oneaufs at gmail.com  Tue Jun  7 14:06:33 2011
From: oneaufs at gmail.com (Prasanth)
Date: Tue, 7 Jun 2011 23:36:33 +0530
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <4DEE675A.3070203@stsci.edu>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<0430C335B65F49E6804724C6B0E8A1A4@gmail.com>
	<4DEE675A.3070203@stsci.edu>
Message-ID: <BANLkTikdY+vz_yDpjr1_zQRH7cd=HTzBBA@mail.gmail.com>

Hello,

This is a work in progress:
https://github.com/matplotlib/matplotlib-py3/wiki.

Prasanth

On Tue, Jun 7, 2011 at 11:30 PM, Mark Sienkiewicz <sienkiew at stsci.edu>wrote:

> Thomas Robitaille wrote:
> >  In the end, one could argue that it's up to us to support Python 3 soon,
> because there won't be a large astronomy user base in Python 3 until we do
> (otherwise it's a vicious circle). The fact that numpy/scipy/matplotlib have
> moved means we should probably start thinking about it too.
> >
>
>
> Can you tell me where to find a matplotlib release that works with
> python 3?  According to their web site, it only works in python 2:
>
> http://matplotlib.sourceforge.net/users/installing.html#build-requirements
>
> Mark S.
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110607/61e3c716/attachment.html>

From erwin at mpe.mpg.de  Tue Jun  7 14:49:29 2011
From: erwin at mpe.mpg.de (Peter Erwin)
Date: Tue, 7 Jun 2011 20:49:29 +0200
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <BANLkTikdY+vz_yDpjr1_zQRH7cd=HTzBBA@mail.gmail.com>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<0430C335B65F49E6804724C6B0E8A1A4@gmail.com>
	<4DEE675A.3070203@stsci.edu>
	<BANLkTikdY+vz_yDpjr1_zQRH7cd=HTzBBA@mail.gmail.com>
Message-ID: <CC81B0D1-AE7D-4512-BC1B-60A1A022A84F@mpe.mpg.de>

As Prasanth points out, the "Python 3 support" in matplotlib is obviously still in the
bleeding-edge development stage. I have the impression the same is true for pyfits
(the latest official release, 2.4.0, only lists "Python 2.4 to 2.7") and also for ipython.

So Xavier Gnata's "numpy,scipy,matplotlib and pyfits are now working with python3"
should perhaps be interpreted as "these packages are either working with Python 3,
or there is active development on Python 3 versions" ... which is arguably reason enough
to start thinking about Python 3 versions of other things.

(The official Mac OS X binary releases for numpy and scipy are still Python 2 only, which
precludes my experimenting with them, as I don't really want to go to the trouble
of compiling from source, given that the 2.x versions install quite nicely.)

   -- Peter

On Jun 7, 2011, at 8:06 PM, Prasanth wrote:

> Hello,
> 
> This is a work in progress: https://github.com/matplotlib/matplotlib-py3/wiki.
> 
> Prasanth
> 
> On Tue, Jun 7, 2011 at 11:30 PM, Mark Sienkiewicz <sienkiew at stsci.edu> wrote:
> Thomas Robitaille wrote:
> >  In the end, one could argue that it's up to us to support Python 3 soon, because there won't be a large astronomy user base in Python 3 until we do (otherwise it's a vicious circle). The fact that numpy/scipy/matplotlib have moved means we should probably start thinking about it too.
> >
> 
> 
> Can you tell me where to find a matplotlib release that works with
> python 3?  According to their web site, it only works in python 2:
> 
> http://matplotlib.sourceforge.net/users/installing.html#build-requirements
> 
> Mark S.
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

=============================================================
Peter Erwin                   Max-Planck-Insitute for Extraterrestrial 
erwin at mpe.mpg.de              Physics, Giessenbachstrasse
tel. +49 (0)89 30000 3695     85748 Garching, Germany
fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin





From thomas.robitaille at gmail.com  Tue Jun  7 15:23:38 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 7 Jun 2011 15:23:38 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <CC81B0D1-AE7D-4512-BC1B-60A1A022A84F@mpe.mpg.de>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>
	<0430C335B65F49E6804724C6B0E8A1A4@gmail.com>
	<4DEE675A.3070203@stsci.edu>
	<BANLkTikdY+vz_yDpjr1_zQRH7cd=HTzBBA@mail.gmail.com>
	<CC81B0D1-AE7D-4512-BC1B-60A1A022A84F@mpe.mpg.de>
Message-ID: <2F28809F8863473B91D69723945403D2@gmail.com>

Matplotlib is indeed not compiling nicely with Python 3.2. However, just for info, I can confirm Scipy 0.9.0 and Numpy 1.6.0 do compile with Python 3.2 out of the box.

Tom



On Tuesday, June 7, 2011 at 2:49 PM, Peter Erwin wrote:
> As Prasanth points out, the "Python 3 support" in matplotlib is obviously still in the
> bleeding-edge development stage. I have the impression the same is true for pyfits
> (the latest official release, 2.4.0, only lists "Python 2.4 to 2.7") and also for ipython.
> 
> So Xavier Gnata's "numpy,scipy,matplotlib and pyfits are now working with python3"
> should perhaps be interpreted as "these packages are either working with Python 3,
> or there is active development on Python 3 versions" ... which is arguably reason enough
> to start thinking about Python 3 versions of other things.
> 
> (The official Mac OS X binary releases for numpy and scipy are still Python 2 only, which
> precludes my experimenting with them, as I don't really want to go to the trouble
> of compiling from source, given that the 2.x versions install quite nicely.)
> 
>  -- Peter
> 
> On Jun 7, 2011, at 8:06 PM, Prasanth wrote:
> 
> > Hello,
> > 
> > This is a work in progress: https://github.com/matplotlib/matplotlib-py3/wiki.
> > 
> > Prasanth
> > 
> > On Tue, Jun 7, 2011 at 11:30 PM, Mark Sienkiewicz <sienkiew at stsci.edu (mailto:sienkiew at stsci.edu)> wrote:
> > Thomas Robitaille wrote:
> > >  In the end, one could argue that it's up to us to support Python 3 soon, because there won't be a large astronomy user base in Python 3 until we do (otherwise it's a vicious circle). The fact that numpy/scipy/matplotlib have moved means we should probably start thinking about it too.
> > 
> > 
> > Can you tell me where to find a matplotlib release that works with
> > python 3? According to their web site, it only works in python 2:
> > 
> > http://matplotlib.sourceforge.net/users/installing.html#build-requirements
> > 
> > Mark S.
> > 
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy
> > 
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy
> 
> =============================================================
> Peter Erwin Max-Planck-Insitute for Extraterrestrial 
> erwin at mpe.mpg.de (mailto:erwin at mpe.mpg.de) Physics, Giessenbachstrasse
> tel. +49 (0)89 30000 3695 85748 Garching, Germany
> fax +49 (0)89 30000 3495 http://www.mpe.mpg.de/~erwin
> 
> 
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> http://mail.scipy.org/mailman/listinfo/astropy




From astropython at gmail.com  Tue Jun  7 16:22:24 2011
From: astropython at gmail.com (Astronomical Python)
Date: Tue, 7 Jun 2011 16:22:24 -0400
Subject: [AstroPy] APLpy 0.9.6 Release
Message-ID: <BANLkTim+Dp5vHpDBGCL5VcLFVUOVHwuPTQ@mail.gmail.com>

We are pleased to announce the release of APLpy 0.9.6, which includes many
new features, bug fixes, and improvements.

APLpy is a Python module that makes it easy to interactively produce
publication-quality plots of astronomical images.

Note that we have now migrated the website and source code repository to
GitHub. The website is now located at http://aplpy.github.com/, while the
source code can be found at https://github.com/aplpy/aplpy. In addition,
APLpy is now released under the MIT license.

One of the main new features in this release is the ability to directly
plot RGB images with embedded AVM meta-data. AVM is a standard for
embedding astronomy-related information in images, and includes spatial
information similar to WCS. Many recent outreach images, e.g. from Spitzer
or ESO, include this information. Therefore, it is possible to plot such
files directly without the need for a FITS file. Additionally, the
aplpy.make_rgb_image function can optionally embed AVM meta-data into the
images it produces. More information can be found here:

http://aplpy.github.com/documentation/howto_avm.html

This release also includes methods to show lines, arrows, and polygons, as
well as a 'close' method that can be used to free up memory when plotting
many images. A full list of changes can be found in the CHANGES file.

Please do not hesitate to let us know if you encounter any problems with
this release,

Cheers,

Thomas Robitaille, Eli Bressert, and Adam Ginsburg


From sienkiew at stsci.edu  Wed Jun  8 11:26:02 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Wed, 08 Jun 2011 11:26:02 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <CC81B0D1-AE7D-4512-BC1B-60A1A022A84F@mpe.mpg.de>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>	<0430C335B65F49E6804724C6B0E8A1A4@gmail.com>	<4DEE675A.3070203@stsci.edu>	<BANLkTikdY+vz_yDpjr1_zQRH7cd=HTzBBA@mail.gmail.com>
	<CC81B0D1-AE7D-4512-BC1B-60A1A022A84F@mpe.mpg.de>
Message-ID: <4DEF948A.7000801@stsci.edu>

Peter Erwin wrote:
> So Xavier Gnata's "numpy,scipy,matplotlib and pyfits are now working with python3"
> should perhaps be interpreted as "these packages are either working with Python 3,
> or there is active development on Python 3 versions" ... which is arguably reason enough
> to start thinking about Python 3 versions of other things.
>   

I thought to ask because I saw yet another statement that implied that 
numpy/scipy/matplotlib were all ready for python 3, but in truth 
matplotlib isn't there yet.

The release notes for numpy 1.6 and scipy 0.9 both explicitly state that 
they DO support python 3, even though the FAQs say that they do not.  
They do work, mostly: numpy 1.6 has 1 erroring test and scipy 0.9 has 1 
error and 1 failure.

I just finished setting up a preliminary python 3 test environment 
yesterday, and I found that about half of the python packages that I 
normally provide are not available as a release package for python 3.  
This includes matplotlib, but also ipython, PIL, Pmw, even Sphinx.  (I 
wonder how they did the python 3 docs - maybe with the pre-release sphinx?)

I had one big idea as a result of all of this:  If you care about python 
3, put a statement on your web page about whether your package 
does/does-not work in python 3.  Even for packages that have python 3 
support, finding this information sometimes feels like you have to look 
in the bottom of a locked filing cabinet stuck in a disused lavatory 
with a sign on the door saying "Beware of the Leopard".

As far as I know, none of the material in astrolib is currently ready 
for python 3.  Should I put a notice on the wiki to that effect?

Mark S.



From stefan.czesla at hs.uni-hamburg.de  Thu Jun  9 12:54:27 2011
From: stefan.czesla at hs.uni-hamburg.de (Stefan Czesla)
Date: Thu, 09 Jun 2011 18:54:27 +0200
Subject: [AstroPy] PyAstronomy
Message-ID: <4DF0FAC3.6050505@hs.uni-hamburg.de>

Dear all,

we would like to let you know about our recent release of a -- hopefully --
useful contribution to Python's astronomy community, namely, our PyAstronomy
package (yes, there have been more inspired names...). It consists of
several subpackages providing the following functionality:

  - funcFit: An XSPEC like fitting interface for scipy and pymc
  - modelSuite: Currently, models for transit light-curves and Rossiter 
McLaughlin effect
                (calculating and fitting)
  - pyTiming: A collection of periodograms
  - pyasl: Our tiny contribution to porting the IDL AstroLib to Python.

If you are interested, you can download the code and browse the 
documentation at:

http://www.hs.uni-hamburg.de/DE/Ins/Per/Czesla/PyA/PyA/index.html

We did not yet set up a public repository or tracker, but we appreciate your
feedback. Feel free to e-mail us to:

stefan.czesla at hs.uni-hamburg.de

Best regards,
Stefan & Sebastian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stefan_czesla.vcf
Type: text/x-vcard
Size: 224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20110609/4ae774ee/attachment.vcf>

From mperrin at stsci.edu  Thu Jun  9 18:25:36 2011
From: mperrin at stsci.edu (Marshall Perrin)
Date: Thu, 9 Jun 2011 22:25:36 +0000
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF0FAC3.6050505@hs.uni-hamburg.de>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
Message-ID: <C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>

On Jun 9, 2011, at 12:54 PM, Stefan Czesla wrote:
Dear all,

we would like to let you know about our recent release of a -- hopefully --
useful contribution to Python's astronomy community, namely, our PyAstronomy
package (yes, there have been more inspired names...). It consists of
several subpackages providing the following functionality:

- funcFit: An XSPEC like fitting interface for scipy and pymc
- modelSuite: Currently, models for transit light-curves and Rossiter McLaughlin effect
              (calculating and fitting)
- pyTiming: A collection of periodograms
- pyasl: Our tiny contribution to porting the IDL AstroLib to Python.


Hopefully without sounding too critical of you in particular, I'm going to ask: do we as a community really need yet another separate python library for astronomy and yet another attempt at building a core set of routines ported from the IDL library?

There is now astrolib (https://trac6.assembla.com/astrolib ), also pyAstroLib (which appears to be something entirely different than astrolib? http://sourceforge.net/projects/pyastrolib/develop), astropysics (http://packages.python.org/Astropysics/) , now your pyAstronomy, and probably more.  It looks like PyAstronomy contains wavelength conversions (already implemented in pyastrolib and astropysics) and coordinate conversions and precession (already in coords, astropysics, pySLALIB, & more).

I'm going to be provocative here:  As a community, we are doing something wrong if everyone wants to start their own new module rather than contributing to a common shared open-source core.  We are clearly doing something wrong if people repeatedly implement the same basic functions rather than building on what's already there. What do we need to do differently? How can we make it easier to use a shared repository and shared namespace for all this?

I'm not arguing that somehow there has to be one true library that everyone uses, one ring to rule them all. But wouldn't it be better if there were a more straightforward choice that community members could easily contribute to, leading to a more coherent overall system? I would assert that astrolib is probably the best candidate for growing into this, but I don't really care which one the community rallies behind. I'd just like to have some sense that we might actually be converging towards an IDL-like unified core rather than just proliferating little packages.   Currently we're just forming dwarf galaxies; how can we get them to accrete together to build a grand design spiral?

 - Marshall



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110609/b13e9ab7/attachment.html>

From wkerzendorf at googlemail.com  Thu Jun  9 21:57:42 2011
From: wkerzendorf at googlemail.com (Wolfgang Kerzendorf)
Date: Fri, 10 Jun 2011 11:57:42 +1000
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
Message-ID: <4DF17A16.5010906@gmail.com>

I very much agree with what marshall said. I am really in favour of a 
modular design rather than these monolithic packages. For example: I use 
pyfits for my fits files, it can't do coordinate transformation and it 
can't do light curve fitting, but it's fast and good at what it does. In 
addition, it puts the data in a format that other packages understand: 
numpy arrays. The same with ephem, pywcs, aplpy, atpy and others (I'm 
sorry if I didn't mention your package). I think we need to build an 
ecosystem of these small tools, that are bug free and easy to use before 
going to big packages (Sage is a good example of this working). One big 
point is that they need to be interoperable.
I'm going to reiterate what I said on this topic before: we need 
standard base classes so that if someone writes a new tool it can easily 
interact with other tools.
Andy Casey and me are currently working on a base class for 1d spectra. 
It's very easy to use and very simple. We will release it into the wild 
in a month or two and hope that this will bring out the discussions 
about APIs for base classes.

Cheers
    Wolfgang

On 10/06/11 8:25 AM, Marshall Perrin wrote:
> On Jun 9, 2011, at 12:54 PM, Stefan Czesla wrote:
>> Dear all,
>>
>> we would like to let you know about our recent release of a -- 
>> hopefully --
>> useful contribution to Python's astronomy community, namely, our 
>> PyAstronomy
>> package (yes, there have been more inspired names...). It consists of
>> several subpackages providing the following functionality:
>>
>> - funcFit: An XSPEC like fitting interface for scipy and pymc
>> - modelSuite: Currently, models for transit light-curves and Rossiter 
>> McLaughlin effect
>>               (calculating and fitting)
>> - pyTiming: A collection of periodograms
>> - pyasl: Our tiny contribution to porting the IDL AstroLib to Python.
>
>
> Hopefully without sounding too critical of you in particular, I'm 
> going to ask: do we as a community really need /yet another/ separate 
> python library for astronomy and yet another attempt at building a 
> core set of routines ported from the IDL library?
>
> There is now astrolib (https://trac6.assembla.com/astrolib ), also 
> pyAstroLib (which appears to be something entirely different than 
> astrolib? http://sourceforge.net/projects/pyastrolib/develop), 
> astropysics (http://packages.python.org/Astropysics/) , now your 
> pyAstronomy, and probably more.  It looks like PyAstronomy contains 
> wavelength conversions (already implemented in pyastrolib and 
> astropysics) and coordinate conversions and precession (already in 
> coords, astropysics, pySLALIB, & more).
>
> I'm going to be provocative here: /As a community, we are doing 
> something wrong /if everyone wants to start their own new module 
> rather than contributing to a common shared open-source core.  We are 
> clearly doing something wrong if people repeatedly implement the same 
> basic functions rather than building on what's already there. What do 
> we need to do differently? How can we make it easier to use a shared 
> repository and shared namespace for all this?
>
> I'm not arguing that somehow there has to be one true library that 
> everyone uses, one ring to rule them all. But wouldn't it be better if 
> there were a more straightforward choice that community members could 
> easily contribute to, leading to a more coherent overall system? I 
> would assert that astrolib is probably the best candidate for growing 
> into this, but I don't really care which one the community rallies 
> behind. I'd just like to have some sense that we might actually be 
> converging towards an IDL-like unified core rather than just 
> proliferating little packages.   Currently we're just forming dwarf 
> galaxies; how can we get them to accrete together to build a grand 
> design spiral?
>
>  - Marshall
>
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110610/97181b6f/attachment.html>

From taro at ap.smu.ca  Thu Jun  9 22:26:00 2011
From: taro at ap.smu.ca (Taro Sato)
Date: Thu, 9 Jun 2011 23:26:00 -0300
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
Message-ID: <BANLkTin+s-Uo7_EcYO4Vew5f7-W-a3J8QQ@mail.gmail.com>

On Thu, Jun 9, 2011 at 7:25 PM, Marshall Perrin <mperrin at stsci.edu> wrote:
> I'm going to be provocative here:  As a community, we are doing something
> wrong if everyone wants to start their own new module rather than
> contributing to a common shared open-source core.  We are clearly doing
> something wrong if people repeatedly implement the same basic functions
> rather than building on what's already there. What do we need to do
> differently? How can we make it easier to use a shared repository and shared
> namespace for all this?
> I'm not arguing that somehow there has to be one true library that everyone
> uses, one ring to rule them all. But wouldn't it be better if there were a
> more straightforward choice that community members could easily contribute
> to, leading to a more coherent overall system? I would assert that astrolib
> is probably the best candidate for growing into this, but I don't really
> care which one the community rallies behind. I'd just like to have some
> sense that we might actually be converging towards an IDL-like unified core
> rather than just proliferating little packages.   Currently we're just
> forming dwarf galaxies; how can we get them to accrete together to build a
> grand design spiral?
>  - Marshall
>


I agree mostly with what Marshall says, but as a (sort of) an
outsider, it's still quite obscure what projects are considered to be
the "main" community efforts.  I still don't really know which
packages should be the de facto standards to do certain things for my
own work (aside from PyFITS, PyRAF, stsci_python... something that
comes out of STSci).

So Instead of relying on somebody else's work which can get abandoned
any time, I've been rolling out my own ports of misc. tools whenever
necessary.  Seeing that can lead to redundant efforts & reinventions
of wheels, I even wanted to join the community effort if such a thing
existed and contacted somebody at STSci, but nothing came out of it
(that was several years ago though...).

My point is, if you claim there *are* community efforts that we could
contribute to, we would appreciate which ones they are and advertise
as such.


-- 
Taro Sato
Department of Astronomy & Physics
Saint Mary's University? ? ? ? ? ?? email: taro at ap.smu.ca
Halifax, NS B3H 3C3? ? ? ? ? ? ? ?? phone: (902)420-5027
Canada? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? web: http://ap.smu.ca/~taro


From thomas.robitaille at gmail.com  Thu Jun  9 22:58:39 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Thu, 9 Jun 2011 22:58:39 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTin+s-Uo7_EcYO4Vew5f7-W-a3J8QQ@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<BANLkTin+s-Uo7_EcYO4Vew5f7-W-a3J8QQ@mail.gmail.com>
Message-ID: <B23E2E2618E34CA98738121100202C71@gmail.com>

> I agree mostly with what Marshall says, but as a (sort of) an
> outsider, it's still quite obscure what projects are considered to be
> the "main" community efforts. I still don't really know which
> packages should be the de facto standards to do certain things for my
> own work (aside from PyFITS, PyRAF, stsci_python... something that
> comes out of STSci).
> My point is, if you claim there *are* community efforts that we could
> contribute to, we would appreciate which ones they are and advertise
> as such.
I don't think it matters too much if a package is *the* main one, but as a rule of thumb, it'd be better to contribute to *any* of the existing packages than to write a new one. So if there are three WCS libraries, then it's best to contribute to the one you prefer than to say that none are the 'main' one and create a new one!

Another rule of thumb is that if a package is on GitHub, then it's very easy to contribute to it via pull requests, and developers really appreciate it when users fix bugs or add features this way. Packages hosted on custom-hosted SVN servers are more difficult because the only mechanism in place is to send a patch by email and the process is much less transparent and your email might just get ignored.

Maybe we need some kind of voting system where users can list what packages they use, and it would then be more obvious which are the most widely used ones? (like http://osx.iusethis.com/). Any volunteers to set up such a system?

Cheers,
Tom




From thomas.robitaille at gmail.com  Thu Jun  9 23:12:44 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Thu, 9 Jun 2011 23:12:44 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
Message-ID: <E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>

I just wanted to also add that (in agreement with Marshall) I'm all in favor of many small modules that accomplish a particular task well, rather than packages that aim for a 'do-it-all' approach and fall short. It's always possible to bundle small packages together afterwards, and I don't mean merge development, but instead just bundling the packages for installation (kind of like EPD). I think that is the easiest approach for all of us.

Maybe in the long run, a specific set of core packages will emerge as essential and we can then talk about truly merging them into a scipy-like package, but for now, I think the race is still on. And after all, there's nothing to say we *have* to achieve the same setup as in IDL.

Cheers,
Tom





From stefan.czesla at hs.uni-hamburg.de  Fri Jun 10 05:57:52 2011
From: stefan.czesla at hs.uni-hamburg.de (Stefan Czesla)
Date: Fri, 10 Jun 2011 11:57:52 +0200
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF17A16.5010906@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<4DF17A16.5010906@gmail.com>
Message-ID: <4DF1EAA0.6090300@hs.uni-hamburg.de>

Dear all,

with great interest, I follow the discussion on how packet proliferation 
for astronomy
should take place. I would like to comment on some issues. Marshall 
kicked off the
discussion with a little provocation (not too critical btw.). I shall 
first say that PyAstronomy
(and the other mentioned projects, except pyastrolib, as I know), is not an
"attempt at building a core set of routines ported from the IDL 
library"; that is the smallest
part at least of PyA. Nonetheless, I agree with Marshall that something 
is wrong with
the development model currently pursued by "the community".

People will always tend to
implement there own small wavelength conversion routines etc. until they 
do not find a solution
that convinces them that it is worth to be installed and "learned". Not 
until they are convinced that
the latter is more convenient than doing it with a quick hack, we will 
see a successful project covering that.
I agree that currently no particular project has gained enough ground to 
take this role. Taro is quite right in
pointing out that there is no de facto standard yet in Python, and as 
the community has not gathered behind
one project, I do not know whether we can already speak of real 
community projects. Considering that Taro
probably did also not exactly find what was needed, the choice of
"rolling out [] own ports" seems quite natural.

Behind forming a beautiful spiral stands the question of what makes a 
successful open-source project - not
raised for the first time here. Although I do not know, I believe that 
documentation and in particular examples (i.e., simplicity and
usability) is a key to convince users to give it a try.  From the 
developer side, I assume that project leadership is crucial.
As long as the state is fuzzy like now, releasing "just another" project 
does at least not hurt anybody.

So - I am curious to see how this evolves,

yours,
Stefan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: stefan_czesla.vcf
Type: text/x-vcard
Size: 224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20110610/cb647a26/attachment.vcf>

From perry at stsci.edu  Fri Jun 10 09:48:55 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Fri, 10 Jun 2011 09:48:55 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
Message-ID: <75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>


On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:

> I just wanted to also add that (in agreement with Marshall) I'm all  
> in favor of many small modules that accomplish a particular task  
> well, rather than packages that aim for a 'do-it-all' approach and  
> fall short. It's always possible to bundle small packages together  
> afterwards, and I don't mean merge development, but instead just  
> bundling the packages for installation (kind of like EPD). I think  
> that is the easiest approach for all of us.
>
> Maybe in the long run, a specific set of core packages will emerge  
> as essential and we can then talk about truly merging them into a  
> scipy-like package, but for now, I think the race is still on. And  
> after all, there's nothing to say we *have* to achieve the same  
> setup as in IDL.

It sure seems to me that the time is ripe to start trying to coalesce  
some of the ongoing efforts.

Mind you, I don't think it is necessarily good to start with only one  
version. Allowing a few different approaches initially has its  
benefits. You get to see more approaches and ideas in play and having  
experience with them is very helpful in deciding which one is more  
productive. And sometimes there is room for more than one in the long  
run. The different approaches may have their own niches. But it is  
hard to imagine any long-term need for more than two or three  
different approaches.

Early on there are some pragmatic needs for different approaches. For  
example, having a fairly "literal" translation of IDL tools into  
Python has its benefit. It is very useful for those that would like to  
migrate IDL code, and given the existing IDL versions, make it much  
easier to test their correctness. But I don't see this as a substitute  
for a good set of modular tools that have a better object design and  
consistent interfaces with other modules. Doing this is more work and  
will take more time. So a need for both approaches is likely. Some  
could argue the same for replacing IRAF tasks.

All this is much easier said than done of course.

I wish STScI had more resources to devote to this than we've actually  
had. We've been planning to do more on this front than we've actually  
done. Things come up repeatedly that ruin these plans. But it may be  
worth saying where some of our current efforts are going that may  
overlap some of these other efforts.

1) We've been planning (along with Gemini, and particularly James  
Turner), to try to develop some Sage-like installation package that  
would make it easy to install all the basic tools for most  
astronomers. We had hoped to have a beta version out, but one of the  
people working on this left at the end of last year, and we've not  
been able to replace that person. We are going to continue this effort  
with existing staff though. Hopefully in a few months we'll have  
something to try out.

2) There is a recognition that a more serious effort needs to be made  
to replace IRAF functionality. Perhaps one of the benefits of a JWST  
delay is that it will allow us to do some of that work more  
explicitly. But we would not do it by replacing IRAF tasks one-for-one  
but coming up with an entirely different approach which has to start  
from the bottom up (the end result could have applications that mostly  
emulate IRAF tasks, but also provide much more modular tools).

3) More specifically, we are currently focussing on how to handle WCS  
issues in a more general way than they are handled in FITS. If there  
is interest, perhaps we should say more about the intended approach.  
This is particularly important for replacing spectrographic tools in  
IRAF, and this is where we are starting our effort.

4) We need a way of saving these WCS models, and FITS is not the way.  
We are looking at an alternate data format, not just for WCS models,  
but for data in general [gasp!]. Work has begun on this too.

5) A lot of our recent work has been on pysynphot and ETCs. We plan on  
making the computational part of our ETCs a released tool. But I'm  
also wondering if we can generalize the pysynphot spectral models for  
more general use in spectral tools.

6) We have been working on a framework for making pipelines easier to  
build and configure. That won't be ready for at least a few months,  
but could well be of general interest and use.

But besides these things, I would like to see if we can't begin the  
effort of narrowing some of the underlying libraries being used. FITS  
WCS is one obvious area that seems ripe for that.

But the community ought to identify one or two areas that are of the  
most interest in consolidating (let's start small). What should we  
start with? Focus is important in making any progress in this area.

Perry





From erik.tollerud at gmail.com  Fri Jun 10 17:32:32 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Fri, 10 Jun 2011 14:32:32 -0700
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
Message-ID: <BANLkTikyj3eZmDPLNTmb8NtRBE+sTM5f+A@mail.gmail.com>

A few comments on the excellent points Perry brings up:

> Early on there are some pragmatic needs for different approaches. For
> example, having a fairly "literal" translation of IDL tools into
> Python has its benefit. It is very useful for those that would like to
> migrate IDL code, and given the existing IDL versions, make it much
> easier to test their correctness. But I don't see this as a substitute
> for a good set of modular tools that have a better object design and
> consistent interfaces with other modules. Doing this is more work and
> will take more time. So a need for both approaches is likely. Some
> could argue the same for replacing IRAF tasks.
>
> All this is much easier said than done of course.

I 100% agree that the goal should be to make things work in a way
people are familiar with in IDL, while eventually switching to a clear
object-oriented/pythonic approach.  In my experience, programming
re-writing algorithms in object-based approaches is actually not
actually more time-consuming, once you get used to it - the key is to
then *also* provide a more functional-like interface to make the
transition easier/more practical.


> 1) We've been planning (along with Gemini, and particularly James
> Turner), to try to develop some Sage-like installation package that
> would make it easy to install all the basic tools for most
> astronomers. We had hoped to have a beta version out, but one of the
> people working on this left at the end of last year, and we've not
> been able to replace that person. We are going to continue this effort
> with existing staff though. Hopefully in a few months we'll have
> something to try out.

You may already be too far in to change this, but I think it's
important to make install tools work within the existing python
ecosystem.  For example, I already have a tool written in astropysics
that installs needed astronomy-related packages, but it uses the PyPI
RPC interface to easily identify packages, and then downloads and
installs them just like, for example, pip
(http://pypi.python.org/pypi/pip) does, which has been clearly
endorsed as the future of python package installers.



> 2) There is a recognition that a more serious effort needs to be made
> to replace IRAF functionality. Perhaps one of the benefits of a JWST
> delay is that it will allow us to do some of that work more
> explicitly. But we would not do it by replacing IRAF tasks one-for-one
> but coming up with an entirely different approach which has to start
> from the bottom up (the end result could have applications that mostly
> emulate IRAF tasks, but also provide much more modular tools).

+1 !

> 3) More specifically, we are currently focussing on how to handle WCS
> issues in a more general way than they are handled in FITS. If there
> is interest, perhaps we should say more about the intended approach.
> This is particularly important for replacing spectrographic tools in
> IRAF, and this is where we are starting our effort.
>
> 4) We need a way of saving these WCS models, and FITS is not the way.
> We are looking at an alternate data format, not just for WCS models,
> but for data in general [gasp!]. Work has begun on this too.

I'm very curious what you have in mind here - are you thinking of a
new format that's more adaptable, or more tightly coupling the data
format to the intended applications?  And is there a plan yet for what
the format is supposed to look like (i.e. XML or otherwise
standards-based or something different?)


> 5) A lot of our recent work has been on pysynphot and ETCs. We plan on
> making the computational part of our ETCs a released tool. But I'm
> also wondering if we can generalize the pysynphot spectral models for
> more general use in spectral tools.

I think spectra might be a good "focus area" where there's room for
consolidation as you mention below...


> 6) We have been working on a framework for making pipelines easier to
> build and configure. That won't be ready for at least a few months,
> but could well be of general interest and use.

Have you looked at my astropysics.utils.pipeline module?  That's
already working, although that may not be exactly what you had in
mind.  I'm working on a traits-based GUI interface for it as well,
although it's not working yet.  Regardless, one important thing to
think about here is making sure it's compatible with the parallel
computing era that we're going into now (it could be nade much easier
with the new parallel architecture of the 0.11 version of ipython).


> But the community ought to identify one or two areas that are of the
> most interest in consolidating (let's start small). What should we
> start with? Focus is important in making any progress in this area.

See my comments above with regard to spectra.  I also think perhaps
some agreement on some basic interfaces for model-fitting to make them
cross-compatible is also important, as a lot of people seem to be
duplicating functionality there, and it's a pretty common need (albiet
with a lot of complicated and application-specific implementation
details).  I've started at this with pymodelfit, but that's certainly
not the only way it could be done...

Also, while I agree focus is important, I think there is also strong
need for a "vision" - that is, a plan for how to proceed on a larger
scale. Then the starting pieces in the direction of that vision can be
determined.  I think my person "vision" for how to deal with this
consolidation was apparent from my previous message, so I won't repeat
it here (although I emphasis that the vision does *not* have to be
based on astropysics - some other package that fulfills the same aims
and uses github or other robust development models would work just as
well).


-- 
Erik Tollerud


From erik.tollerud at gmail.com  Fri Jun 10 17:47:53 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Fri, 10 Jun 2011 14:47:53 -0700
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF1EAA0.6090300@hs.uni-hamburg.de>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<4DF17A16.5010906@gmail.com> <4DF1EAA0.6090300@hs.uni-hamburg.de>
Message-ID: <BANLkTi=OHC41AMrfxihB9xF1UpsCQT5Vkw@mail.gmail.com>

I definitely agree with Marshall's starting point that consolidation
would be great, but I'd like to weigh in with a slightly different
point-of-view of the "small modules" approach.  The danger of
*independent* small modules is that it can lead to *huge* versioning
problems and reproducibility issues.  This is one of the most
frustrating problems I fight with on a seemingly daily basis with IDL.

Certain applications are much better for small modules - for example,
pyfits is great as a stand-alone tool because it really is just used
by other modules and there's little need for the other modules to
"talk back" to pyfits, as it were. pywcs is similar (although not
quite as much so, given the need for coordinate systems).  On the
other hand, consider the situation of a set of classes designed to
represent photometric bands (and associated photometry routines).  It
makes a lot of sense to represent these bands in a similar way to how
you would represent a spectrum.  And if you want to have another class
for an SED, you need to both have the SED understand spectra and
photometry, *and* there are many use cases where you want to treat the
SED as a spectrum of sorts.  This is where small modules for each of
these things suddenly become frustrating because you spend more time
dealing with module/package management then you do with the actual
science.

This *could* by solved via an EPD-like approach, but those require
that each of the sub-modules be fairly carefully/consistently
packaged, and it seems like that just doesn't happen in astronomy.  It
also makes it much harder to coordinate development, because the
versioning problems are only solved at every release.  Most python
packages don't really have that complicated dependencies, unlike the
astronomy case where conceivable a given module depends on many
different modules in the scenario outlined above.

So the vision I have for astropysics is to house all such modules in
one *package*, while still implementing them as separate modules
within that package.  As Thomas points out, combining this with the
excellent capabilities of git and github means different people can
work on different modules on different branches, while still keeping
it a single package. This is akin to the idea of the Goddard IDL
library (an approach that is clearly working fairly well with the
community), while still being pythonic and up-to-date in terms of project
management.

With this in mind, I agree with Wolfgang's suggestion of a consistent
set of base classes... But note that this is also already implemented
in many places in astropysics, given that it was my original design
philosophy (in fact, Wolfgang, I would strongly urge you to look at
the astropysics.spec.Spectrum class before writing your own - I
designed it with exactly the idea you had in mind.).

I think the key to addresses most of these concerns is a
community-driven package using the tools and methods that open source
software people have spent decades refining.  I think these models are
something that python-based astronomy would do well to learn from and
make use of for our own needs. Of course I personally like the idea of
doing this
with astropysics, given that it's my baby, but the broader ideas for the
community certainly don't have to be tied to that particular package.



On Fri, Jun 10, 2011 at 2:57 AM, Stefan Czesla
<stefan.czesla at hs.uni-hamburg.de> wrote:
> Dear all,
>
> with great interest, I follow the discussion on how packet proliferation for
> astronomy
> should take place. I would like to comment on some issues. Marshall kicked
> off the
> discussion with a little provocation (not too critical btw.). I shall first
> say that PyAstronomy
> (and the other mentioned projects, except pyastrolib, as I know), is not an
> "attempt at building a core set of routines ported from the IDL library";
> that is the smallest
> part at least of PyA. Nonetheless, I agree with Marshall that something is
> wrong with
> the development model currently pursued by "the community".
>
> People will always tend to
> implement there own small wavelength conversion routines etc. until they do
> not find a solution
> that convinces them that it is worth to be installed and "learned". Not
> until they are convinced that
> the latter is more convenient than doing it with a quick hack, we will see a
> successful project covering that.
> I agree that currently no particular project has gained enough ground to
> take this role. Taro is quite right in
> pointing out that there is no de facto standard yet in Python, and as the
> community has not gathered behind
> one project, I do not know whether we can already speak of real community
> projects. Considering that Taro
> probably did also not exactly find what was needed, the choice of
> "rolling out [] own ports" seems quite natural.
>
> Behind forming a beautiful spiral stands the question of what makes a
> successful open-source project - not
> raised for the first time here. Although I do not know, I believe that
> documentation and in particular examples (i.e., simplicity and
> usability) is a key to convince users to give it a try. ?From the developer
> side, I assume that project leadership is crucial.
> As long as the state is fuzzy like now, releasing "just another" project
> does at least not hurt anybody.
>
> So - I am curious to see how this evolves,
>
> yours,
> Stefan
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>



-- 
Erik Tollerud


From embray at stsci.edu  Fri Jun 10 19:30:20 2011
From: embray at stsci.edu (Erik Bray)
Date: Fri, 10 Jun 2011 19:30:20 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTi=OHC41AMrfxihB9xF1UpsCQT5Vkw@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<4DF17A16.5010906@gmail.com>
	<4DF1EAA0.6090300@hs.uni-hamburg.de>
	<BANLkTi=OHC41AMrfxihB9xF1UpsCQT5Vkw@mail.gmail.com>
Message-ID: <4DF2A90C.6030809@stsci.edu>

On 06/10/2011 05:47 PM, Erik Tollerud wrote:
 > So the vision I have for astropysics is to house all such modules in
 > one *package*, while still implementing them as separate modules
 > within that package.  As Thomas points out, combining this with the
 > excellent capabilities of git and github means different people can
 > work on different modules on different branches, while still keeping
 > it a single package. This is akin to the idea of the Goddard IDL
 > library (an approach that is clearly working fairly well with the
 > community), while still being pythonic and up-to-date in terms of project
 > management.

+1 to this.  Grouping closely related functionality to a medium-sized 
package is probably best from a project management perspective.  Think 
scikits.astronomy or scikits.photometry or the like (not saying it has 
to be part of scikits, though they provide a good example).


From perry at stsci.edu  Fri Jun 10 21:23:17 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Fri, 10 Jun 2011 21:23:17 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTikyj3eZmDPLNTmb8NtRBE+sTM5f+A@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<BANLkTikyj3eZmDPLNTmb8NtRBE+sTM5f+A@mail.gmail.com>
Message-ID: <6253EEFB-CA11-4671-9451-5072B31FF744@stsci.edu>


On Jun 10, 2011, at 5:32 PM, Erik Tollerud wrote:
> [...]
>> 1) We've been planning (along with Gemini, and particularly James
>> Turner), to try to develop some Sage-like installation package that
>> would make it easy to install all the basic tools for most
>> astronomers. We had hoped to have a beta version out, but one of the
>> people working on this left at the end of last year, and we've not
>> been able to replace that person. We are going to continue this  
>> effort
>> with existing staff though. Hopefully in a few months we'll have
>> something to try out.
>
> You may already be too far in to change this, but I think it's
> important to make install tools work within the existing python
> ecosystem.  For example, I already have a tool written in astropysics
> that installs needed astronomy-related packages, but it uses the PyPI
> RPC interface to easily identify packages, and then downloads and
> installs them just like, for example, pip
> (http://pypi.python.org/pypi/pip) does, which has been clearly
> endorsed as the future of python package installers.
>
We are planning to do that as well, but it doesn't solve the larger  
issue. Not everything people need to use is in that ecosystem, and  
some of the things in it don't install easily on lots of platforms.  
Unlike the Sage approach, this has to allow updates and additions so I  
wouldn't want people to read too much into the "Sage-like"
>
>
>> 3) More specifically, we are currently focussing on how to handle WCS
>> issues in a more general way than they are handled in FITS. If there
>> is interest, perhaps we should say more about the intended approach.
>> This is particularly important for replacing spectrographic tools in
>> IRAF, and this is where we are starting our effort.
>>
>> 4) We need a way of saving these WCS models, and FITS is not the way.
>> We are looking at an alternate data format, not just for WCS models,
>> but for data in general [gasp!]. Work has begun on this too.
>
> I'm very curious what you have in mind here - are you thinking of a
> new format that's more adaptable, or more tightly coupling the data
> format to the intended applications?  And is there a plan yet for what
> the format is supposed to look like (i.e. XML or otherwise
> standards-based or something different?)
>
Yes to more adaptable, no to tightly coupling (though I suppose that  
is subject to interpretation). We have some ideas. Ideally it could  
map to more than one representation on disk but represent a common  
abstract object system. Perhaps I will circulate a bit more on some of  
the ideas in the near future.
>
>> 5) A lot of our recent work has been on pysynphot and ETCs. We plan  
>> on
>> making the computational part of our ETCs a released tool. But I'm
>> also wondering if we can generalize the pysynphot spectral models for
>> more general use in spectral tools.
>
> I think spectra might be a good "focus area" where there's room for
> consolidation as you mention below...
>
That's a good possibility.
>
>> 6) We have been working on a framework for making pipelines easier to
>> build and configure. That won't be ready for at least a few months,
>> but could well be of general interest and use.
>
> Have you looked at my astropysics.utils.pipeline module?  That's
> already working, although that may not be exactly what you had in
> mind.  I'm working on a traits-based GUI interface for it as well,
> although it's not working yet.  Regardless, one important thing to
> think about here is making sure it's compatible with the parallel
> computing era that we're going into now (it could be nade much easier
> with the new parallel architecture of the 0.11 version of ipython).
>
yes, but this has a somewhat different goal. Reviewing that again, it  
doesn't seem like there is that much overlap in what it does. I think  
pipeline is a fairly overloaded term these days.
>
>> But the community ought to identify one or two areas that are of the
>> most interest in consolidating (let's start small). What should we
>> start with? Focus is important in making any progress in this area.
>
> See my comments above with regard to spectra.  I also think perhaps
> some agreement on some basic interfaces for model-fitting to make them
> cross-compatible is also important, as a lot of people seem to be
> duplicating functionality there, and it's a pretty common need (albiet
> with a lot of complicated and application-specific implementation
> details).  I've started at this with pymodelfit, but that's certainly
> not the only way it could be done...
>
> Also, while I agree focus is important, I think there is also strong
> need for a "vision" - that is, a plan for how to proceed on a larger
> scale. Then the starting pieces in the direction of that vision can be
> determined.  I think my person "vision" for how to deal with this
> consolidation was apparent from my previous message, so I won't repeat
> it here (although I emphasis that the vision does *not* have to be
> based on astropysics - some other package that fulfills the same aims
> and uses github or other robust development models would work just as
> well).

Perhaps. I think vision makes sense on small to medium scales.  
Astronomy software is a pretty large area and I'm not sure anyone in  
particular can do a vision for everything in that regard. I think  
there is room for both broad (top down?) and narrow approaches (bottom  
up coalescing). I've seen lots of failures with the pure top down  
approach. But maybe I'm too small minded :-)

Perry




From jh at physics.ucf.edu  Sat Jun 11 02:03:02 2011
From: jh at physics.ucf.edu (Joe Harrington)
Date: Sat, 11 Jun 2011 02:03:02 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <mailman.1.1307725201.22420.astropy@scipy.org>
	(astropy-request@scipy.org)
References: <mailman.1.1307725201.22420.astropy@scipy.org>
Message-ID: <wl8r570aort.fsf@physics.ucf.edu>

Responding quickly to Perry...

I like the matplotlib approach to the IDL problem: Implement it right
first.  Then write compatibility wrappers that look like IDL.  These
keep old code alive without requiring maintenance of two full
codebases.

I dislike the SAGE approach.  It installs too much, and not enough,
both at the same time!  Base classes, small and independent packages,
meta-packages that have dependencies on a selection of real packages
and thus look like monolithic systems you can easily extend, all done
at the package-manager level.  Tests.  Docs.  A small start but with a
big and straightforward design.  This is the way to go.

--jh--


From oneaufs at gmail.com  Sat Jun 11 02:42:46 2011
From: oneaufs at gmail.com (Prasanth)
Date: Sat, 11 Jun 2011 12:12:46 +0530
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <wl8r570aort.fsf@physics.ucf.edu>
References: <mailman.1.1307725201.22420.astropy@scipy.org>
	<wl8r570aort.fsf@physics.ucf.edu>
Message-ID: <BANLkTimfu1yxs=6Rf9n5ByGWpd74hs_=XA@mail.gmail.com>

Hello,

I would like to put forward a suggestion, as briefly mentioned by others
above, that if community participation is to be effective, then a system
like Github would be very useful. I gather that most of today's established
software have historically accepted contributions in the form of patches and
tarballs, but one of the reasons why Git, and perhaps other systems too, was
invented was because at one stage it became too difficult to do
this effectively.

I like Git, and Github, but there are other services such as assembla.com,
which offers both SVN and Git, and bitbucket.org that offers Mercurial. I
think that moving to a distributed version control system such as Git or
Mercurial, and using websites such as Github, will bring down the barrier
for making contributions.

As was mentioned earlier, an email can get ignored, perhaps unintentionally
or perhaps it just doesn't make it to the top of the to-do list of the
concerned person, where as a pull request is there for anyone to see. It can
be useful to other "forks" of the repository even if the main repository
takes a lot of time to respond. Everyone can see the developments taking
place, and can collaborate and contribute to each individual line of
development.

It is of-course hard to decide on which system to use. Python source code
now uses Mercurial, mainly because they needed a system that was robust on
Windows and the majority of developers liked Mercurial. If it is decided
that an open system for accepting contributions is the way to go, then maybe
we can have such a discussion and voting.

No matter what system is used, I strongly feel that we need to use these
tools like the Linux and core Python communities. With these tools if I want
to make a contribution, I can just, as they say at Github, "Go Fork and
Multiply".

Prasanth

On Sat, Jun 11, 2011 at 11:33 AM, Joe Harrington <jh at physics.ucf.edu> wrote:

> Responding quickly to Perry...
>
> I like the matplotlib approach to the IDL problem: Implement it right
> first.  Then write compatibility wrappers that look like IDL.  These
> keep old code alive without requiring maintenance of two full
> codebases.
>
> I dislike the SAGE approach.  It installs too much, and not enough,
> both at the same time!  Base classes, small and independent packages,
> meta-packages that have dependencies on a selection of real packages
> and thus look like monolithic systems you can easily extend, all done
> at the package-manager level.  Tests.  Docs.  A small start but with a
> big and straightforward design.  This is the way to go.
>
> --jh--
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110611/f9700644/attachment.html>

From perry at stsci.edu  Sat Jun 11 10:54:27 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Sat, 11 Jun 2011 10:54:27 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <wl8r570aort.fsf@physics.ucf.edu>
References: <mailman.1.1307725201.22420.astropy@scipy.org>
	<wl8r570aort.fsf@physics.ucf.edu>
Message-ID: <FDDA7D02-54CF-4890-B820-49C70A41D37A@stsci.edu>


On Jun 11, 2011, at 2:03 AM, Joe Harrington wrote:

> Responding quickly to Perry...
>
> I like the matplotlib approach to the IDL problem: Implement it right
> first.  Then write compatibility wrappers that look like IDL.  These
> keep old code alive without requiring maintenance of two full
> codebases.
>
Ideally, yes. But the simple fact is that it is much simpler to do a  
literal translation. When the 'right' solution doesn't appear quickly  
enough there are many who are willing to do the easy thing (and may  
not have time or the expertise to do the right thing). Are we to say  
"no, you must wait for the right approach"? And I do mean that it is  
much easier. Doing it right means understanding most of the uses of  
the objects involved in order to figure out how best to do the design  
and even then a few iterations may be needed to get a good design. So  
I can see where the literal approach can solve some people's problem  
in the meantime.

It's easy to point to matplotlib to say that was the right way to do  
it. It misses the fact that there were several years of failed  
approaches before it appeared. And even in matplotlib, if you were to  
ask John Hunter, would have done some things differently now (and it  
has had some significant refactorings besides).

> I dislike the SAGE approach.  It installs too much, and not enough,
> both at the same time!  Base classes, small and independent packages,
> meta-packages that have dependencies on a selection of real packages
> and thus look like monolithic systems you can easily extend, all done
> at the package-manager level.  Tests.  Docs.  A small start but with a
> big and straightforward design.  This is the way to go.
>
Well, note I said Sage-like, not Sage. Sage itself I think is  
essentially monolithic. We aren't talking about anything like its  
size. We would like to use some of its tools and approaches. And we  
intend for it to allow upgrades and additions. They have had good  
success with installation though. Am I 100% certain this is the right  
approach? No, but it's what we are going to try given our resources.

As I've said before, this is a notoriously hard problem. If it wasn't  
it would have been solved already by many other in many other fields.  
There will be no perfect solution (I feel confident in this  
prediction). No one is stopping others from trying approaches they  
feel are superior.

Perry





From mdroe at stsci.edu  Mon Jun 13 09:01:14 2011
From: mdroe at stsci.edu (Michael Droettboom)
Date: Mon, 13 Jun 2011 09:01:14 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <2F28809F8863473B91D69723945403D2@gmail.com>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>	<0430C335B65F49E6804724C6B0E8A1A4@gmail.com>	<4DEE675A.3070203@stsci.edu>	<BANLkTikdY+vz_yDpjr1_zQRH7cd=HTzBBA@mail.gmail.com>	<CC81B0D1-AE7D-4512-BC1B-60A1A022A84F@mpe.mpg.de>
	<2F28809F8863473B91D69723945403D2@gmail.com>
Message-ID: <4DF60A1A.5040608@stsci.edu>

Are you referring to master, or the py3k branch? [1]

https://github.com/matplotlib/matplotlib-py3

Mike

On 06/07/2011 03:23 PM, Thomas Robitaille wrote:
> Matplotlib is indeed not compiling nicely with Python 3.2. However, just for info, I can confirm Scipy 0.9.0 and Numpy 1.6.0 do compile with Python 3.2 out of the box.
>
> Tom
>
>
>
> On Tuesday, June 7, 2011 at 2:49 PM, Peter Erwin wrote:
>> As Prasanth points out, the "Python 3 support" in matplotlib is obviously still in the
>> bleeding-edge development stage. I have the impression the same is true for pyfits
>> (the latest official release, 2.4.0, only lists "Python 2.4 to 2.7") and also for ipython.
>>
>> So Xavier Gnata's "numpy,scipy,matplotlib and pyfits are now working with python3"
>> should perhaps be interpreted as "these packages are either working with Python 3,
>> or there is active development on Python 3 versions" ... which is arguably reason enough
>> to start thinking about Python 3 versions of other things.
>>
>> (The official Mac OS X binary releases for numpy and scipy are still Python 2 only, which
>> precludes my experimenting with them, as I don't really want to go to the trouble
>> of compiling from source, given that the 2.x versions install quite nicely.)
>>
>>   -- Peter
>>
>> On Jun 7, 2011, at 8:06 PM, Prasanth wrote:
>>
>>> Hello,
>>>
>>> This is a work in progress: https://github.com/matplotlib/matplotlib-py3/wiki.
>>>
>>> Prasanth
>>>
>>> On Tue, Jun 7, 2011 at 11:30 PM, Mark Sienkiewicz<sienkiew at stsci.edu (mailto:sienkiew at stsci.edu)>  wrote:
>>> Thomas Robitaille wrote:
>>>>   In the end, one could argue that it's up to us to support Python 3 soon, because there won't be a large astronomy user base in Python 3 until we do (otherwise it's a vicious circle). The fact that numpy/scipy/matplotlib have moved means we should probably start thinking about it too.
>>>
>>> Can you tell me where to find a matplotlib release that works with
>>> python 3? According to their web site, it only works in python 2:
>>>
>>> http://matplotlib.sourceforge.net/users/installing.html#build-requirements
>>>
>>> Mark S.
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>> http://mail.scipy.org/mailman/listinfo/astropy
>> =============================================================
>> Peter Erwin Max-Planck-Insitute for Extraterrestrial
>> erwin at mpe.mpg.de (mailto:erwin at mpe.mpg.de) Physics, Giessenbachstrasse
>> tel. +49 (0)89 30000 3695 85748 Garching, Germany
>> fax +49 (0)89 30000 3495 http://www.mpe.mpg.de/~erwin
>>
>>
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>> http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA



From mdroe at stsci.edu  Mon Jun 13 09:02:31 2011
From: mdroe at stsci.edu (Michael Droettboom)
Date: Mon, 13 Jun 2011 09:02:31 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <4DEF948A.7000801@stsci.edu>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>	<E8100678-48B7-46D4-A6EC-A70314697527@stsci.edu>	<0430C335B65F49E6804724C6B0E8A1A4@gmail.com>	<4DEE675A.3070203@stsci.edu>	<BANLkTikdY+vz_yDpjr1_zQRH7cd=HTzBBA@mail.gmail.com>	<CC81B0D1-AE7D-4512-BC1B-60A1A022A84F@mpe.mpg.de>
	<4DEF948A.7000801@stsci.edu>
Message-ID: <4DF60A67.7070200@stsci.edu>

On 06/08/2011 11:26 AM, Mark Sienkiewicz wrote:
> Peter Erwin wrote:
>> So Xavier Gnata's "numpy,scipy,matplotlib and pyfits are now working with python3"
>> should perhaps be interpreted as "these packages are either working with Python 3,
>> or there is active development on Python 3 versions" ... which is arguably reason enough
>> to start thinking about Python 3 versions of other things.
>>
> I thought to ask because I saw yet another statement that implied that
> numpy/scipy/matplotlib were all ready for python 3, but in truth
> matplotlib isn't there yet.
It is there and passing all regression tests in the git repo, just not 
in the latest release.  This py3 support will most likely be in the next 
release, though there is no schedule for that.

Mike

-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA



From mdroe at stsci.edu  Mon Jun 13 09:03:19 2011
From: mdroe at stsci.edu (Michael Droettboom)
Date: Mon, 13 Jun 2011 09:03:19 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
Message-ID: <4DF60A97.4020108@stsci.edu>

On 06/07/2011 10:44 AM, Xavier Gnata wrote:
> Hi,
>
> well everything is mainly in the title.
> numpy,scipy,matplotlib and pyfits are now working with python3.
>
Sorry for the delayed response (I was on vacation last week).

pywcs in SVN supports Python 3 (it has for many months), but some 
features that rely on pyfits will be unavailable.

Mike

-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA



From embray at stsci.edu  Mon Jun 13 11:39:29 2011
From: embray at stsci.edu (Erik Bray)
Date: Mon, 13 Jun 2011 11:39:29 -0400
Subject: [AstroPy] Plan to port astropy to python3?
In-Reply-To: <4DF60A97.4020108@stsci.edu>
References: <BANLkTikAqLbxr9wHyn11j-n3WN25JK2+Dw@mail.gmail.com>
	<4DF60A97.4020108@stsci.edu>
Message-ID: <4DF62F31.2030306@stsci.edu>

On 06/13/2011 09:03 AM, Michael Droettboom wrote:
> On 06/07/2011 10:44 AM, Xavier Gnata wrote:
>> Hi,
>>
>> well everything is mainly in the title.
>> numpy,scipy,matplotlib and pyfits are now working with python3.
>>
> Sorry for the delayed response (I was on vacation last week).
>
> pywcs in SVN supports Python 3 (it has for many months), but some
> features that rely on pyfits will be unavailable.

Why would that be?  Pyfits is supporting Python 3 fairly reliably at 
this point.  I haven't tested it with pywcs in Python 3 yet, but I 
expect it should be mostly compatible at this point.

Erik


From sienkiew at stsci.edu  Mon Jun 13 11:46:42 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Mon, 13 Jun 2011 11:46:42 -0400
Subject: [AstroPy] Organizing AstroPy (was Re: Proliferating py-astro-libs)
In-Reply-To: <C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
Message-ID: <4DF630E2.5020107@stsci.edu>

Marshall Perrin wrote:
> I'm going to be provocative here:  /As a community, we are doing 
> something wrong /if everyone wants to start their own new module 
> rather than contributing to a common shared open-source core.  We are 
> clearly doing something wrong if people repeatedly implement the same 
> basic functions rather than building on what's already there. What do 
> we need to do differently? How can we make it easier to use a shared 
> repository and shared namespace for all this?
...
>  Currently we're just forming dwarf galaxies; how can we get them to 
> accrete together to build a grand design spiral?


If you make a universe that contains a lot of hydrogen, all you have to 
do is wait, and you will get spiral galaxies. Our problem is more like 
trying to arrange the stars onto the faces of a cube and keep them there 
when they start to drift away.  It is not a naturally occurring state, 
but you can do it, if you apply enough energy in the right places.

If you make a universe that contains people who have similar software 
needs and expect each of them to do their own programming, you naturally 
get hundreds of disjoint pieces that do not fit together nicely, contain 
a lot of duplication, and are mostly unmaintained.  Except for small 
groups, communities and software projects are not self-organizing.  But 
you can organize the work into a single well-coordinated project, if you 
apply enough energy in the right places.

The most effective approach would require:

- one person willing to coordinate the project -- but maybe as many 5 at 
most.  (Unless you subdivide the project into independent parts, it 
doesn't work as well with more.)  Basically, we need a Guido -- he acted 
as BDFL for python at the request of (and with the consent of) the 
python community.  He provided valuable organization that could not have 
existed without somebody working at it.

- some small number of people to act as core developers.  These people 
design the system and coordinate contributions.  This is also a lot of 
work, and you're looking for people with software design skills.  Maybe 
3 to 10 people.

- a bunch of people willing be contributing developers.  These people 
contribute subsets.  This is less work than being a core developer, but 
still a bunch more than hacking together a custom solution for your 
specific needs and putting it up on your web site.

- users who are interested enough to overcome the learning curve.  Why 
should I use your simple coordinate transformation when I can write my 
own faster than I can figure out how to use yours?  But, implement and 
document enough useful capabilities in your package, and it is worth my 
time to use them instead of writing my own copy of everything.  
(Remember:  The user of your package only knows what you tell them -- as 
far as the user knows, anything that isn't documented doesn't exist.)

This rough model is typical of successful free software projects.  Think 
of examples like python, perl, linux, GCC, gnuplot, freebsd, or gnome.  
They all work this way.


I'm informed that this model is more top-down than this community likes, 
but you have to consider whether the benefits of organizing are greater 
than the costs of failing to organize.  After all, if you reject the 
possibility that someone will coordinate the activities, then the 
activities will necessarily be un-coordinated.

In fact, I think that this community _wants_ some organization -- if 
not, we would not be having this discussion.  (again - check the mailing 
list archives.)

In an organization like what I am describing, the "management" layers 
actually work for the "lower" levels.  As a developer who makes an 
occasional contribution, I demand that the organizer give me the 
information necessary to make my contribution fit into the integrated 
whole.  If you think in terms of traditional business management models, 
it may look like the organizer is "assigning work", but in fact, the 
organizer is helping me by telling me how my piece can fit.

In other words, make it easy for me to contribute an integrated part.  
(I'm speaking of coming up with the right software, not mechanism for 
sending my contribution.)

In the python community, the acronymn "BDFL" stands for "Benevolent 
Dictator For Life", but in fact, Guido does not dictate anything.  He 
aids the community in resolving differences.  When the python community 
does not reach consensus, somebody (Guido) has to decide.  We all go 
along with his decision because we are better off staying organized 
together even when some specific choice does not go the way I like.  
(See my opinions on python 3, for example.)


We need an organizer to help the community reach consensus and to make 
choices when consensus is not achievable.  There is another problem, though:

Who can do it?

I can't answer that, but I've seen plenty of open source projects fail 
because they can't get an organizer.  If nobody loves the idea of 
AstroPy enough to be the organizer (maybe even to take it on independent 
of their primary job), then maybe we should accept that and lower our 
expectations.  There is a less labor-intensive alternative, where 
AstroPy becomes just a collection of different packages with no unified 
model and no attempt to make them fit nicely together.  This also needs 
an organizer, but with a lesser workload.  It doesn't make nearly as 
good a product, but it is better to succeed at the "best you can do with 
the resources available" than to fail at "the ideal result".


It all starts with a big decision.  Will we organize AstroPy?  Who will 
volunteer to work for us as the organizer?  Is our community willing to 
buy in to this development model?  If not, what is the alternative?

Mark S.

p.s.  I normally don't bother saying this because it is usually obvious, 
but:  These comments are mine, based on decades of involvement in 
programming and software engineering.  I do not speak for STScI.



From tevaughan at gmail.com  Mon Jun 13 12:22:14 2011
From: tevaughan at gmail.com (Thomas Vaughan)
Date: Mon, 13 Jun 2011 16:22:14 +0000
Subject: [AstroPy] pysynphot
Message-ID: <BANLkTikikFZgNaCKgof_cYYMrYL-wKTEYA@mail.gmail.com>

On Tue, May 17, 2011 at 18:22, Vicki Laidler <laidler at stsci.edu> wrote:
>
> Indeed, astrolib includes pysynphot, which is presently being used
> under the hood of the HST Exposure Time Calculators. It's intended as
> a replacement for the IRAF STSDAS synphot package, and is largely
> complete. I'm the primary developer for pysynphot at this time.

Thanks for responding to my query last month. Pysynphot looks like the
right tool to me. I'm just now getting round to looking at the
software, and I have a couple of questions.

> You would need to create a throughput file (wavelength and throughput in dimensionless units between 0 and 1) for each band in your instrument, and a spectrum file or array (wavelength and flux) or black body as a function of temperature.

My plan is (rather than to use blackbody spectra) to use the
flux-calibrated template library of Pickles (1998) and match these up
to the spectral identifications in the Tycho-2 Spectral Type Catalog.

> Once you have those, you can simulate an observation and predict the
> magnitude, something like this:
>
> import pysynphot as psyn
> bp = psyn.FileBandpass('your_throughput_file.fits')
> sp = psyn.FileSpectrum('spectrum_file.fits')
> obs = psyn.Observation(sp, bp)
> obs.effstim('abmag') ?#gives you the observed magnitude

That looks nice and easy. I don't have stuff in FITS format, but I
suppose that your library can read a text file with a couple of
columns.

One issue that I've had right up front is in installing pysynphot-0.7.
 I looked at the README file, and installed the listed prerequisites.
Nevertheless, on attempted installation I got an error message about a
missing library that seemed to be named as if it were part of the
stsci_python package.

So I grabbed stsci-python-2.11.  That seems to have installed well
enough.  It came with a pysynphot.  A recursive diff shows that
pysynphot-0.7 is different from the pysynphot in stsci-python-2.11.
Which is better to use?

-- 
Thomas E. Vaughan


From jturner at gemini.edu  Mon Jun 13 13:25:13 2011
From: jturner at gemini.edu (James Turner)
Date: Mon, 13 Jun 2011 13:25:13 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
Message-ID: <4DF647F9.2060409@gemini.edu>

It seems that several of us would really like to improve
collaboration on Python libraries but have been struggling to pull
it off. I've raised the same issue on this list in past months, but my
focus has unavoidably been on other things and since I'm wary of
shouting a lot without contributing much, I haven't really been able
to keep the discussion alive...

I tend to agree with Mark and Stefan about the question of leadership.
Perry & co. at Space Telescope deserve recognition for getting us this
far with things like PyFITS and PyRAF. Others have taken the initiative
with things like astronomical plotting and documentation sprints. We're
still lacking a bit of coherence though, which (as Mark suggests) is
likely to involve one or a few dedicated, energetic, knowledgeable,
hands-on developer(s) who can glue things together. Those people need
to be employed by someone, though, to ensure stability & continuity
(fortunately there's already a bit of that going on at STScI, eg. with
Mike and Matplotlib). Personally, I have the motivation but have not
had the time/independence (and might not be assertive enough).
Apparently we do have several energetic authors in the community now
(like Thomas & Eli), but each with their own project.

A couple of years ago, a number of us at the observatories submitted a
white paper to the Decadal Survey, pointing out the need for more
co-ordinated funding so that we can have people who focus on cross-
institutional platform development & support. The report from the
committee did give a nod to our concerns and their importance, but
stopped short of making any recommendation, which basically means "good
luck with that". Meanwhile, at Gemini we have had our own problems to
deal with, which make it very difficult for me to propose something
internally beyond working with STScI on the distribution of
dependencies that Perry mentioned. Perhaps someone obtaining a grant
for this is not out of the question though.

I would like it if we could get together organically behind Astrolib,
but sometimes it's difficult to get people away from their immediate
priorities to focus on that, even within my own institution. If we
could get people dedicated to it, though, it could become indispensable
enough to attract and co-ordinate more effort. I'm just not sure how we
get started at this point and my personal options for tackling the
problem seem limited given the overarching funding transition at Gemini
and the intense focus on projects that are needed to make that work...

Cheers,

James.


On 10/06/11 09:48, Perry Greenfield wrote:
>
> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>
>> I just wanted to also add that (in agreement with Marshall) I'm all
>> in favor of many small modules that accomplish a particular task
>> well, rather than packages that aim for a 'do-it-all' approach and
>> fall short. It's always possible to bundle small packages together
>> afterwards, and I don't mean merge development, but instead just
>> bundling the packages for installation (kind of like EPD). I think
>> that is the easiest approach for all of us.
>>
>> Maybe in the long run, a specific set of core packages will emerge
>> as essential and we can then talk about truly merging them into a
>> scipy-like package, but for now, I think the race is still on. And
>> after all, there's nothing to say we *have* to achieve the same
>> setup as in IDL.
>
> It sure seems to me that the time is ripe to start trying to coalesce
> some of the ongoing efforts.
>
> Mind you, I don't think it is necessarily good to start with only one
> version. Allowing a few different approaches initially has its
> benefits. You get to see more approaches and ideas in play and having
> experience with them is very helpful in deciding which one is more
> productive. And sometimes there is room for more than one in the long
> run. The different approaches may have their own niches. But it is
> hard to imagine any long-term need for more than two or three
> different approaches.
>
> Early on there are some pragmatic needs for different approaches. For
> example, having a fairly "literal" translation of IDL tools into
> Python has its benefit. It is very useful for those that would like to
> migrate IDL code, and given the existing IDL versions, make it much
> easier to test their correctness. But I don't see this as a substitute
> for a good set of modular tools that have a better object design and
> consistent interfaces with other modules. Doing this is more work and
> will take more time. So a need for both approaches is likely. Some
> could argue the same for replacing IRAF tasks.
>
> All this is much easier said than done of course.
>
> I wish STScI had more resources to devote to this than we've actually
> had. We've been planning to do more on this front than we've actually
> done. Things come up repeatedly that ruin these plans. But it may be
> worth saying where some of our current efforts are going that may
> overlap some of these other efforts.
>
> 1) We've been planning (along with Gemini, and particularly James
> Turner), to try to develop some Sage-like installation package that
> would make it easy to install all the basic tools for most
> astronomers. We had hoped to have a beta version out, but one of the
> people working on this left at the end of last year, and we've not
> been able to replace that person. We are going to continue this effort
> with existing staff though. Hopefully in a few months we'll have
> something to try out.
>
> 2) There is a recognition that a more serious effort needs to be made
> to replace IRAF functionality. Perhaps one of the benefits of a JWST
> delay is that it will allow us to do some of that work more
> explicitly. But we would not do it by replacing IRAF tasks one-for-one
> but coming up with an entirely different approach which has to start
> from the bottom up (the end result could have applications that mostly
> emulate IRAF tasks, but also provide much more modular tools).
>
> 3) More specifically, we are currently focussing on how to handle WCS
> issues in a more general way than they are handled in FITS. If there
> is interest, perhaps we should say more about the intended approach.
> This is particularly important for replacing spectrographic tools in
> IRAF, and this is where we are starting our effort.
>
> 4) We need a way of saving these WCS models, and FITS is not the way.
> We are looking at an alternate data format, not just for WCS models,
> but for data in general [gasp!]. Work has begun on this too.
>
> 5) A lot of our recent work has been on pysynphot and ETCs. We plan on
> making the computational part of our ETCs a released tool. But I'm
> also wondering if we can generalize the pysynphot spectral models for
> more general use in spectral tools.
>
> 6) We have been working on a framework for making pipelines easier to
> build and configure. That won't be ready for at least a few months,
> but could well be of general interest and use.
>
> But besides these things, I would like to see if we can't begin the
> effort of narrowing some of the underlying libraries being used. FITS
> WCS is one obvious area that seems ripe for that.
>
> But the community ought to identify one or two areas that are of the
> most interest in consolidating (let's start small). What should we
> start with? Focus is important in making any progress in this area.
>
> Perry
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


From jturner at gemini.edu  Mon Jun 13 13:43:53 2011
From: jturner at gemini.edu (James Turner)
Date: Mon, 13 Jun 2011 13:43:53 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF647F9.2060409@gemini.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu>
Message-ID: <4DF64C59.3010501@gemini.edu>

Maybe we should hold an AstroPy conference, where we can discuss
co-ordination, get to know each other better and even sit down and
work on libraries together (like at SciPy). That might help generate
a bit of momentum. Some of us have had meetings before that were
full of ideas that didn't go anywhere, but I don't think it has to
be that way if active people on the ground are talking to one another
rather than having institutions present their plans and try to
negotiate at a high level.


On 13/06/11 13:25, James Turner wrote:
> It seems that several of us would really like to improve
> collaboration on Python libraries but have been struggling to pull
> it off. I've raised the same issue on this list in past months, but my
> focus has unavoidably been on other things and since I'm wary of
> shouting a lot without contributing much, I haven't really been able
> to keep the discussion alive...
>
> I tend to agree with Mark and Stefan about the question of leadership.
> Perry & co. at Space Telescope deserve recognition for getting us this
> far with things like PyFITS and PyRAF. Others have taken the initiative
> with things like astronomical plotting and documentation sprints. We're
> still lacking a bit of coherence though, which (as Mark suggests) is
> likely to involve one or a few dedicated, energetic, knowledgeable,
> hands-on developer(s) who can glue things together. Those people need
> to be employed by someone, though, to ensure stability & continuity
> (fortunately there's already a bit of that going on at STScI, eg. with
> Mike and Matplotlib). Personally, I have the motivation but have not
> had the time/independence (and might not be assertive enough).
> Apparently we do have several energetic authors in the community now
> (like Thomas & Eli), but each with their own project.
>
> A couple of years ago, a number of us at the observatories submitted a
> white paper to the Decadal Survey, pointing out the need for more
> co-ordinated funding so that we can have people who focus on cross-
> institutional platform development & support. The report from the
> committee did give a nod to our concerns and their importance, but
> stopped short of making any recommendation, which basically means "good
> luck with that". Meanwhile, at Gemini we have had our own problems to
> deal with, which make it very difficult for me to propose something
> internally beyond working with STScI on the distribution of
> dependencies that Perry mentioned. Perhaps someone obtaining a grant
> for this is not out of the question though.
>
> I would like it if we could get together organically behind Astrolib,
> but sometimes it's difficult to get people away from their immediate
> priorities to focus on that, even within my own institution. If we
> could get people dedicated to it, though, it could become indispensable
> enough to attract and co-ordinate more effort. I'm just not sure how we
> get started at this point and my personal options for tackling the
> problem seem limited given the overarching funding transition at Gemini
> and the intense focus on projects that are needed to make that work...
>
> Cheers,
>
> James.
>
>
> On 10/06/11 09:48, Perry Greenfield wrote:
>>
>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>
>>> I just wanted to also add that (in agreement with Marshall) I'm all
>>> in favor of many small modules that accomplish a particular task
>>> well, rather than packages that aim for a 'do-it-all' approach and
>>> fall short. It's always possible to bundle small packages together
>>> afterwards, and I don't mean merge development, but instead just
>>> bundling the packages for installation (kind of like EPD). I think
>>> that is the easiest approach for all of us.
>>>
>>> Maybe in the long run, a specific set of core packages will emerge
>>> as essential and we can then talk about truly merging them into a
>>> scipy-like package, but for now, I think the race is still on. And
>>> after all, there's nothing to say we *have* to achieve the same
>>> setup as in IDL.
>>
>> It sure seems to me that the time is ripe to start trying to coalesce
>> some of the ongoing efforts.
>>
>> Mind you, I don't think it is necessarily good to start with only one
>> version. Allowing a few different approaches initially has its
>> benefits. You get to see more approaches and ideas in play and having
>> experience with them is very helpful in deciding which one is more
>> productive. And sometimes there is room for more than one in the long
>> run. The different approaches may have their own niches. But it is
>> hard to imagine any long-term need for more than two or three
>> different approaches.
>>
>> Early on there are some pragmatic needs for different approaches. For
>> example, having a fairly "literal" translation of IDL tools into
>> Python has its benefit. It is very useful for those that would like to
>> migrate IDL code, and given the existing IDL versions, make it much
>> easier to test their correctness. But I don't see this as a substitute
>> for a good set of modular tools that have a better object design and
>> consistent interfaces with other modules. Doing this is more work and
>> will take more time. So a need for both approaches is likely. Some
>> could argue the same for replacing IRAF tasks.
>>
>> All this is much easier said than done of course.
>>
>> I wish STScI had more resources to devote to this than we've actually
>> had. We've been planning to do more on this front than we've actually
>> done. Things come up repeatedly that ruin these plans. But it may be
>> worth saying where some of our current efforts are going that may
>> overlap some of these other efforts.
>>
>> 1) We've been planning (along with Gemini, and particularly James
>> Turner), to try to develop some Sage-like installation package that
>> would make it easy to install all the basic tools for most
>> astronomers. We had hoped to have a beta version out, but one of the
>> people working on this left at the end of last year, and we've not
>> been able to replace that person. We are going to continue this effort
>> with existing staff though. Hopefully in a few months we'll have
>> something to try out.
>>
>> 2) There is a recognition that a more serious effort needs to be made
>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
>> delay is that it will allow us to do some of that work more
>> explicitly. But we would not do it by replacing IRAF tasks one-for-one
>> but coming up with an entirely different approach which has to start
>> from the bottom up (the end result could have applications that mostly
>> emulate IRAF tasks, but also provide much more modular tools).
>>
>> 3) More specifically, we are currently focussing on how to handle WCS
>> issues in a more general way than they are handled in FITS. If there
>> is interest, perhaps we should say more about the intended approach.
>> This is particularly important for replacing spectrographic tools in
>> IRAF, and this is where we are starting our effort.
>>
>> 4) We need a way of saving these WCS models, and FITS is not the way.
>> We are looking at an alternate data format, not just for WCS models,
>> but for data in general [gasp!]. Work has begun on this too.
>>
>> 5) A lot of our recent work has been on pysynphot and ETCs. We plan on
>> making the computational part of our ETCs a released tool. But I'm
>> also wondering if we can generalize the pysynphot spectral models for
>> more general use in spectral tools.
>>
>> 6) We have been working on a framework for making pipelines easier to
>> build and configure. That won't be ready for at least a few months,
>> but could well be of general interest and use.
>>
>> But besides these things, I would like to see if we can't begin the
>> effort of narrowing some of the underlying libraries being used. FITS
>> WCS is one obvious area that seems ripe for that.
>>
>> But the community ought to identify one or two areas that are of the
>> most interest in consolidating (let's start small). What should we
>> start with? Focus is important in making any progress in this area.
>>
>> Perry
>>
>>
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy


-- 
James E.H. Turner
Gemini Observatory Southern Operations Centre,
Casilla 603,		Tel. (+56) 51 205609
La Serena, Chile.	Fax. (+56) 51 205650


From perry at stsci.edu  Mon Jun 13 14:00:39 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Mon, 13 Jun 2011 14:00:39 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF64C59.3010501@gemini.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
Message-ID: <F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>

That's a good idea.

Perry

On Jun 13, 2011, at 1:43 PM, James Turner wrote:

> Maybe we should hold an AstroPy conference, where we can discuss
> co-ordination, get to know each other better and even sit down and
> work on libraries together (like at SciPy). That might help generate
> a bit of momentum. Some of us have had meetings before that were
> full of ideas that didn't go anywhere, but I don't think it has to
> be that way if active people on the ground are talking to one another
> rather than having institutions present their plans and try to
> negotiate at a high level.
>
>
> On 13/06/11 13:25, James Turner wrote:
>> It seems that several of us would really like to improve
>> collaboration on Python libraries but have been struggling to pull
>> it off. I've raised the same issue on this list in past months, but  
>> my
>> focus has unavoidably been on other things and since I'm wary of
>> shouting a lot without contributing much, I haven't really been able
>> to keep the discussion alive...
>>
>> I tend to agree with Mark and Stefan about the question of  
>> leadership.
>> Perry & co. at Space Telescope deserve recognition for getting us  
>> this
>> far with things like PyFITS and PyRAF. Others have taken the  
>> initiative
>> with things like astronomical plotting and documentation sprints.  
>> We're
>> still lacking a bit of coherence though, which (as Mark suggests) is
>> likely to involve one or a few dedicated, energetic, knowledgeable,
>> hands-on developer(s) who can glue things together. Those people need
>> to be employed by someone, though, to ensure stability & continuity
>> (fortunately there's already a bit of that going on at STScI, eg.  
>> with
>> Mike and Matplotlib). Personally, I have the motivation but have not
>> had the time/independence (and might not be assertive enough).
>> Apparently we do have several energetic authors in the community now
>> (like Thomas & Eli), but each with their own project.
>>
>> A couple of years ago, a number of us at the observatories  
>> submitted a
>> white paper to the Decadal Survey, pointing out the need for more
>> co-ordinated funding so that we can have people who focus on cross-
>> institutional platform development & support. The report from the
>> committee did give a nod to our concerns and their importance, but
>> stopped short of making any recommendation, which basically means  
>> "good
>> luck with that". Meanwhile, at Gemini we have had our own problems to
>> deal with, which make it very difficult for me to propose something
>> internally beyond working with STScI on the distribution of
>> dependencies that Perry mentioned. Perhaps someone obtaining a grant
>> for this is not out of the question though.
>>
>> I would like it if we could get together organically behind Astrolib,
>> but sometimes it's difficult to get people away from their immediate
>> priorities to focus on that, even within my own institution. If we
>> could get people dedicated to it, though, it could become  
>> indispensable
>> enough to attract and co-ordinate more effort. I'm just not sure  
>> how we
>> get started at this point and my personal options for tackling the
>> problem seem limited given the overarching funding transition at  
>> Gemini
>> and the intense focus on projects that are needed to make that  
>> work...
>>
>> Cheers,
>>
>> James.
>>
>>
>> On 10/06/11 09:48, Perry Greenfield wrote:
>>>
>>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>
>>>> I just wanted to also add that (in agreement with Marshall) I'm all
>>>> in favor of many small modules that accomplish a particular task
>>>> well, rather than packages that aim for a 'do-it-all' approach and
>>>> fall short. It's always possible to bundle small packages together
>>>> afterwards, and I don't mean merge development, but instead just
>>>> bundling the packages for installation (kind of like EPD). I think
>>>> that is the easiest approach for all of us.
>>>>
>>>> Maybe in the long run, a specific set of core packages will emerge
>>>> as essential and we can then talk about truly merging them into a
>>>> scipy-like package, but for now, I think the race is still on. And
>>>> after all, there's nothing to say we *have* to achieve the same
>>>> setup as in IDL.
>>>
>>> It sure seems to me that the time is ripe to start trying to  
>>> coalesce
>>> some of the ongoing efforts.
>>>
>>> Mind you, I don't think it is necessarily good to start with only  
>>> one
>>> version. Allowing a few different approaches initially has its
>>> benefits. You get to see more approaches and ideas in play and  
>>> having
>>> experience with them is very helpful in deciding which one is more
>>> productive. And sometimes there is room for more than one in the  
>>> long
>>> run. The different approaches may have their own niches. But it is
>>> hard to imagine any long-term need for more than two or three
>>> different approaches.
>>>
>>> Early on there are some pragmatic needs for different approaches.  
>>> For
>>> example, having a fairly "literal" translation of IDL tools into
>>> Python has its benefit. It is very useful for those that would  
>>> like to
>>> migrate IDL code, and given the existing IDL versions, make it much
>>> easier to test their correctness. But I don't see this as a  
>>> substitute
>>> for a good set of modular tools that have a better object design and
>>> consistent interfaces with other modules. Doing this is more work  
>>> and
>>> will take more time. So a need for both approaches is likely. Some
>>> could argue the same for replacing IRAF tasks.
>>>
>>> All this is much easier said than done of course.
>>>
>>> I wish STScI had more resources to devote to this than we've  
>>> actually
>>> had. We've been planning to do more on this front than we've  
>>> actually
>>> done. Things come up repeatedly that ruin these plans. But it may be
>>> worth saying where some of our current efforts are going that may
>>> overlap some of these other efforts.
>>>
>>> 1) We've been planning (along with Gemini, and particularly James
>>> Turner), to try to develop some Sage-like installation package that
>>> would make it easy to install all the basic tools for most
>>> astronomers. We had hoped to have a beta version out, but one of the
>>> people working on this left at the end of last year, and we've not
>>> been able to replace that person. We are going to continue this  
>>> effort
>>> with existing staff though. Hopefully in a few months we'll have
>>> something to try out.
>>>
>>> 2) There is a recognition that a more serious effort needs to be  
>>> made
>>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
>>> delay is that it will allow us to do some of that work more
>>> explicitly. But we would not do it by replacing IRAF tasks one-for- 
>>> one
>>> but coming up with an entirely different approach which has to start
>>> from the bottom up (the end result could have applications that  
>>> mostly
>>> emulate IRAF tasks, but also provide much more modular tools).
>>>
>>> 3) More specifically, we are currently focussing on how to handle  
>>> WCS
>>> issues in a more general way than they are handled in FITS. If there
>>> is interest, perhaps we should say more about the intended approach.
>>> This is particularly important for replacing spectrographic tools in
>>> IRAF, and this is where we are starting our effort.
>>>
>>> 4) We need a way of saving these WCS models, and FITS is not the  
>>> way.
>>> We are looking at an alternate data format, not just for WCS models,
>>> but for data in general [gasp!]. Work has begun on this too.
>>>
>>> 5) A lot of our recent work has been on pysynphot and ETCs. We  
>>> plan on
>>> making the computational part of our ETCs a released tool. But I'm
>>> also wondering if we can generalize the pysynphot spectral models  
>>> for
>>> more general use in spectral tools.
>>>
>>> 6) We have been working on a framework for making pipelines easier  
>>> to
>>> build and configure. That won't be ready for at least a few months,
>>> but could well be of general interest and use.
>>>
>>> But besides these things, I would like to see if we can't begin the
>>> effort of narrowing some of the underlying libraries being used.  
>>> FITS
>>> WCS is one obvious area that seems ripe for that.
>>>
>>> But the community ought to identify one or two areas that are of the
>>> most interest in consolidating (let's start small). What should we
>>> start with? Focus is important in making any progress in this area.
>>>
>>> Perry
>>>
>>>
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>
>
> -- 
> James E.H. Turner
> Gemini Observatory Southern Operations Centre,
> Casilla 603,		Tel. (+56) 51 205609
> La Serena, Chile.	Fax. (+56) 51 205650
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From kellecruz at gmail.com  Mon Jun 13 14:59:12 2011
From: kellecruz at gmail.com (Kelle Cruz)
Date: Mon, 13 Jun 2011 14:59:12 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
Message-ID: <BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>

I think a sit-down is desperately needed to resolve these issues, figure out
the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
progress to occur.

I'd be happy to participate as a non-python/programming expert and maybe
provide the voice of the "users".

I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that seems
to be where most of the movers and shakers in this game are located.

kelle

On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu> wrote:

> That's a good idea.
>
> Perry
>
> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>
> > Maybe we should hold an AstroPy conference, where we can discuss
> > co-ordination, get to know each other better and even sit down and
> > work on libraries together (like at SciPy). That might help generate
> > a bit of momentum. Some of us have had meetings before that were
> > full of ideas that didn't go anywhere, but I don't think it has to
> > be that way if active people on the ground are talking to one another
> > rather than having institutions present their plans and try to
> > negotiate at a high level.
> >
> >
> > On 13/06/11 13:25, James Turner wrote:
> >> It seems that several of us would really like to improve
> >> collaboration on Python libraries but have been struggling to pull
> >> it off. I've raised the same issue on this list in past months, but
> >> my
> >> focus has unavoidably been on other things and since I'm wary of
> >> shouting a lot without contributing much, I haven't really been able
> >> to keep the discussion alive...
> >>
> >> I tend to agree with Mark and Stefan about the question of
> >> leadership.
> >> Perry & co. at Space Telescope deserve recognition for getting us
> >> this
> >> far with things like PyFITS and PyRAF. Others have taken the
> >> initiative
> >> with things like astronomical plotting and documentation sprints.
> >> We're
> >> still lacking a bit of coherence though, which (as Mark suggests) is
> >> likely to involve one or a few dedicated, energetic, knowledgeable,
> >> hands-on developer(s) who can glue things together. Those people need
> >> to be employed by someone, though, to ensure stability & continuity
> >> (fortunately there's already a bit of that going on at STScI, eg.
> >> with
> >> Mike and Matplotlib). Personally, I have the motivation but have not
> >> had the time/independence (and might not be assertive enough).
> >> Apparently we do have several energetic authors in the community now
> >> (like Thomas & Eli), but each with their own project.
> >>
> >> A couple of years ago, a number of us at the observatories
> >> submitted a
> >> white paper to the Decadal Survey, pointing out the need for more
> >> co-ordinated funding so that we can have people who focus on cross-
> >> institutional platform development & support. The report from the
> >> committee did give a nod to our concerns and their importance, but
> >> stopped short of making any recommendation, which basically means
> >> "good
> >> luck with that". Meanwhile, at Gemini we have had our own problems to
> >> deal with, which make it very difficult for me to propose something
> >> internally beyond working with STScI on the distribution of
> >> dependencies that Perry mentioned. Perhaps someone obtaining a grant
> >> for this is not out of the question though.
> >>
> >> I would like it if we could get together organically behind Astrolib,
> >> but sometimes it's difficult to get people away from their immediate
> >> priorities to focus on that, even within my own institution. If we
> >> could get people dedicated to it, though, it could become
> >> indispensable
> >> enough to attract and co-ordinate more effort. I'm just not sure
> >> how we
> >> get started at this point and my personal options for tackling the
> >> problem seem limited given the overarching funding transition at
> >> Gemini
> >> and the intense focus on projects that are needed to make that
> >> work...
> >>
> >> Cheers,
> >>
> >> James.
> >>
> >>
> >> On 10/06/11 09:48, Perry Greenfield wrote:
> >>>
> >>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
> >>>
> >>>> I just wanted to also add that (in agreement with Marshall) I'm all
> >>>> in favor of many small modules that accomplish a particular task
> >>>> well, rather than packages that aim for a 'do-it-all' approach and
> >>>> fall short. It's always possible to bundle small packages together
> >>>> afterwards, and I don't mean merge development, but instead just
> >>>> bundling the packages for installation (kind of like EPD). I think
> >>>> that is the easiest approach for all of us.
> >>>>
> >>>> Maybe in the long run, a specific set of core packages will emerge
> >>>> as essential and we can then talk about truly merging them into a
> >>>> scipy-like package, but for now, I think the race is still on. And
> >>>> after all, there's nothing to say we *have* to achieve the same
> >>>> setup as in IDL.
> >>>
> >>> It sure seems to me that the time is ripe to start trying to
> >>> coalesce
> >>> some of the ongoing efforts.
> >>>
> >>> Mind you, I don't think it is necessarily good to start with only
> >>> one
> >>> version. Allowing a few different approaches initially has its
> >>> benefits. You get to see more approaches and ideas in play and
> >>> having
> >>> experience with them is very helpful in deciding which one is more
> >>> productive. And sometimes there is room for more than one in the
> >>> long
> >>> run. The different approaches may have their own niches. But it is
> >>> hard to imagine any long-term need for more than two or three
> >>> different approaches.
> >>>
> >>> Early on there are some pragmatic needs for different approaches.
> >>> For
> >>> example, having a fairly "literal" translation of IDL tools into
> >>> Python has its benefit. It is very useful for those that would
> >>> like to
> >>> migrate IDL code, and given the existing IDL versions, make it much
> >>> easier to test their correctness. But I don't see this as a
> >>> substitute
> >>> for a good set of modular tools that have a better object design and
> >>> consistent interfaces with other modules. Doing this is more work
> >>> and
> >>> will take more time. So a need for both approaches is likely. Some
> >>> could argue the same for replacing IRAF tasks.
> >>>
> >>> All this is much easier said than done of course.
> >>>
> >>> I wish STScI had more resources to devote to this than we've
> >>> actually
> >>> had. We've been planning to do more on this front than we've
> >>> actually
> >>> done. Things come up repeatedly that ruin these plans. But it may be
> >>> worth saying where some of our current efforts are going that may
> >>> overlap some of these other efforts.
> >>>
> >>> 1) We've been planning (along with Gemini, and particularly James
> >>> Turner), to try to develop some Sage-like installation package that
> >>> would make it easy to install all the basic tools for most
> >>> astronomers. We had hoped to have a beta version out, but one of the
> >>> people working on this left at the end of last year, and we've not
> >>> been able to replace that person. We are going to continue this
> >>> effort
> >>> with existing staff though. Hopefully in a few months we'll have
> >>> something to try out.
> >>>
> >>> 2) There is a recognition that a more serious effort needs to be
> >>> made
> >>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
> >>> delay is that it will allow us to do some of that work more
> >>> explicitly. But we would not do it by replacing IRAF tasks one-for-
> >>> one
> >>> but coming up with an entirely different approach which has to start
> >>> from the bottom up (the end result could have applications that
> >>> mostly
> >>> emulate IRAF tasks, but also provide much more modular tools).
> >>>
> >>> 3) More specifically, we are currently focussing on how to handle
> >>> WCS
> >>> issues in a more general way than they are handled in FITS. If there
> >>> is interest, perhaps we should say more about the intended approach.
> >>> This is particularly important for replacing spectrographic tools in
> >>> IRAF, and this is where we are starting our effort.
> >>>
> >>> 4) We need a way of saving these WCS models, and FITS is not the
> >>> way.
> >>> We are looking at an alternate data format, not just for WCS models,
> >>> but for data in general [gasp!]. Work has begun on this too.
> >>>
> >>> 5) A lot of our recent work has been on pysynphot and ETCs. We
> >>> plan on
> >>> making the computational part of our ETCs a released tool. But I'm
> >>> also wondering if we can generalize the pysynphot spectral models
> >>> for
> >>> more general use in spectral tools.
> >>>
> >>> 6) We have been working on a framework for making pipelines easier
> >>> to
> >>> build and configure. That won't be ready for at least a few months,
> >>> but could well be of general interest and use.
> >>>
> >>> But besides these things, I would like to see if we can't begin the
> >>> effort of narrowing some of the underlying libraries being used.
> >>> FITS
> >>> WCS is one obvious area that seems ripe for that.
> >>>
> >>> But the community ought to identify one or two areas that are of the
> >>> most interest in consolidating (let's start small). What should we
> >>> start with? Focus is important in making any progress in this area.
> >>>
> >>> Perry
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> AstroPy mailing list
> >>> AstroPy at scipy.org
> >>> http://mail.scipy.org/mailman/listinfo/astropy
> >
> >
> > --
> > James E.H. Turner
> > Gemini Observatory Southern Operations Centre,
> > Casilla 603,          Tel. (+56) 51 205609
> > La Serena, Chile.     Fax. (+56) 51 205650
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org
> > http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 
Kelle Cruz, PhD ? http://kellecruz.com/
917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110613/32ce0153/attachment.html>

From aldcroft at head.cfa.harvard.edu  Mon Jun 13 15:07:13 2011
From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft)
Date: Mon, 13 Jun 2011 15:07:13 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
Message-ID: <BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>

What about a splinter meeting at SciPy2011?  I guess the question is
how many interested parties will NOT be there this year.

- Tom

On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com> wrote:
> I think a sit-down is desperately needed to resolve these issues, figure out
> the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
> progress to occur.
> I'd be happy to participate as a non-python/programming expert and maybe
> provide the voice of the "users".
> I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
> deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
> Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that seems
> to be where most of the movers and shakers in this game are located.
> kelle
>
> On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu> wrote:
>>
>> That's a good idea.
>>
>> Perry
>>
>> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>>
>> > Maybe we should hold an AstroPy conference, where we can discuss
>> > co-ordination, get to know each other better and even sit down and
>> > work on libraries together (like at SciPy). That might help generate
>> > a bit of momentum. Some of us have had meetings before that were
>> > full of ideas that didn't go anywhere, but I don't think it has to
>> > be that way if active people on the ground are talking to one another
>> > rather than having institutions present their plans and try to
>> > negotiate at a high level.
>> >
>> >
>> > On 13/06/11 13:25, James Turner wrote:
>> >> It seems that several of us would really like to improve
>> >> collaboration on Python libraries but have been struggling to pull
>> >> it off. I've raised the same issue on this list in past months, but
>> >> my
>> >> focus has unavoidably been on other things and since I'm wary of
>> >> shouting a lot without contributing much, I haven't really been able
>> >> to keep the discussion alive...
>> >>
>> >> I tend to agree with Mark and Stefan about the question of
>> >> leadership.
>> >> Perry & co. at Space Telescope deserve recognition for getting us
>> >> this
>> >> far with things like PyFITS and PyRAF. Others have taken the
>> >> initiative
>> >> with things like astronomical plotting and documentation sprints.
>> >> We're
>> >> still lacking a bit of coherence though, which (as Mark suggests) is
>> >> likely to involve one or a few dedicated, energetic, knowledgeable,
>> >> hands-on developer(s) who can glue things together. Those people need
>> >> to be employed by someone, though, to ensure stability & continuity
>> >> (fortunately there's already a bit of that going on at STScI, eg.
>> >> with
>> >> Mike and Matplotlib). Personally, I have the motivation but have not
>> >> had the time/independence (and might not be assertive enough).
>> >> Apparently we do have several energetic authors in the community now
>> >> (like Thomas & Eli), but each with their own project.
>> >>
>> >> A couple of years ago, a number of us at the observatories
>> >> submitted a
>> >> white paper to the Decadal Survey, pointing out the need for more
>> >> co-ordinated funding so that we can have people who focus on cross-
>> >> institutional platform development & support. The report from the
>> >> committee did give a nod to our concerns and their importance, but
>> >> stopped short of making any recommendation, which basically means
>> >> "good
>> >> luck with that". Meanwhile, at Gemini we have had our own problems to
>> >> deal with, which make it very difficult for me to propose something
>> >> internally beyond working with STScI on the distribution of
>> >> dependencies that Perry mentioned. Perhaps someone obtaining a grant
>> >> for this is not out of the question though.
>> >>
>> >> I would like it if we could get together organically behind Astrolib,
>> >> but sometimes it's difficult to get people away from their immediate
>> >> priorities to focus on that, even within my own institution. If we
>> >> could get people dedicated to it, though, it could become
>> >> indispensable
>> >> enough to attract and co-ordinate more effort. I'm just not sure
>> >> how we
>> >> get started at this point and my personal options for tackling the
>> >> problem seem limited given the overarching funding transition at
>> >> Gemini
>> >> and the intense focus on projects that are needed to make that
>> >> work...
>> >>
>> >> Cheers,
>> >>
>> >> James.
>> >>
>> >>
>> >> On 10/06/11 09:48, Perry Greenfield wrote:
>> >>>
>> >>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>> >>>
>> >>>> I just wanted to also add that (in agreement with Marshall) I'm all
>> >>>> in favor of many small modules that accomplish a particular task
>> >>>> well, rather than packages that aim for a 'do-it-all' approach and
>> >>>> fall short. It's always possible to bundle small packages together
>> >>>> afterwards, and I don't mean merge development, but instead just
>> >>>> bundling the packages for installation (kind of like EPD). I think
>> >>>> that is the easiest approach for all of us.
>> >>>>
>> >>>> Maybe in the long run, a specific set of core packages will emerge
>> >>>> as essential and we can then talk about truly merging them into a
>> >>>> scipy-like package, but for now, I think the race is still on. And
>> >>>> after all, there's nothing to say we *have* to achieve the same
>> >>>> setup as in IDL.
>> >>>
>> >>> It sure seems to me that the time is ripe to start trying to
>> >>> coalesce
>> >>> some of the ongoing efforts.
>> >>>
>> >>> Mind you, I don't think it is necessarily good to start with only
>> >>> one
>> >>> version. Allowing a few different approaches initially has its
>> >>> benefits. You get to see more approaches and ideas in play and
>> >>> having
>> >>> experience with them is very helpful in deciding which one is more
>> >>> productive. And sometimes there is room for more than one in the
>> >>> long
>> >>> run. The different approaches may have their own niches. But it is
>> >>> hard to imagine any long-term need for more than two or three
>> >>> different approaches.
>> >>>
>> >>> Early on there are some pragmatic needs for different approaches.
>> >>> For
>> >>> example, having a fairly "literal" translation of IDL tools into
>> >>> Python has its benefit. It is very useful for those that would
>> >>> like to
>> >>> migrate IDL code, and given the existing IDL versions, make it much
>> >>> easier to test their correctness. But I don't see this as a
>> >>> substitute
>> >>> for a good set of modular tools that have a better object design and
>> >>> consistent interfaces with other modules. Doing this is more work
>> >>> and
>> >>> will take more time. So a need for both approaches is likely. Some
>> >>> could argue the same for replacing IRAF tasks.
>> >>>
>> >>> All this is much easier said than done of course.
>> >>>
>> >>> I wish STScI had more resources to devote to this than we've
>> >>> actually
>> >>> had. We've been planning to do more on this front than we've
>> >>> actually
>> >>> done. Things come up repeatedly that ruin these plans. But it may be
>> >>> worth saying where some of our current efforts are going that may
>> >>> overlap some of these other efforts.
>> >>>
>> >>> 1) We've been planning (along with Gemini, and particularly James
>> >>> Turner), to try to develop some Sage-like installation package that
>> >>> would make it easy to install all the basic tools for most
>> >>> astronomers. We had hoped to have a beta version out, but one of the
>> >>> people working on this left at the end of last year, and we've not
>> >>> been able to replace that person. We are going to continue this
>> >>> effort
>> >>> with existing staff though. Hopefully in a few months we'll have
>> >>> something to try out.
>> >>>
>> >>> 2) There is a recognition that a more serious effort needs to be
>> >>> made
>> >>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
>> >>> delay is that it will allow us to do some of that work more
>> >>> explicitly. But we would not do it by replacing IRAF tasks one-for-
>> >>> one
>> >>> but coming up with an entirely different approach which has to start
>> >>> from the bottom up (the end result could have applications that
>> >>> mostly
>> >>> emulate IRAF tasks, but also provide much more modular tools).
>> >>>
>> >>> 3) More specifically, we are currently focussing on how to handle
>> >>> WCS
>> >>> issues in a more general way than they are handled in FITS. If there
>> >>> is interest, perhaps we should say more about the intended approach.
>> >>> This is particularly important for replacing spectrographic tools in
>> >>> IRAF, and this is where we are starting our effort.
>> >>>
>> >>> 4) We need a way of saving these WCS models, and FITS is not the
>> >>> way.
>> >>> We are looking at an alternate data format, not just for WCS models,
>> >>> but for data in general [gasp!]. Work has begun on this too.
>> >>>
>> >>> 5) A lot of our recent work has been on pysynphot and ETCs. We
>> >>> plan on
>> >>> making the computational part of our ETCs a released tool. But I'm
>> >>> also wondering if we can generalize the pysynphot spectral models
>> >>> for
>> >>> more general use in spectral tools.
>> >>>
>> >>> 6) We have been working on a framework for making pipelines easier
>> >>> to
>> >>> build and configure. That won't be ready for at least a few months,
>> >>> but could well be of general interest and use.
>> >>>
>> >>> But besides these things, I would like to see if we can't begin the
>> >>> effort of narrowing some of the underlying libraries being used.
>> >>> FITS
>> >>> WCS is one obvious area that seems ripe for that.
>> >>>
>> >>> But the community ought to identify one or two areas that are of the
>> >>> most interest in consolidating (let's start small). What should we
>> >>> start with? Focus is important in making any progress in this area.
>> >>>
>> >>> Perry
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> AstroPy mailing list
>> >>> AstroPy at scipy.org
>> >>> http://mail.scipy.org/mailman/listinfo/astropy
>> >
>> >
>> > --
>> > James E.H. Turner
>> > Gemini Observatory Southern Operations Centre,
>> > Casilla 603, ? ? ? ? ?Tel. (+56) 51 205609
>> > La Serena, Chile. ? ? Fax. (+56) 51 205650
>> > _______________________________________________
>> > AstroPy mailing list
>> > AstroPy at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/astropy
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
>
>
> --
> Kelle Cruz, PhD ??http://kellecruz.com/
> 917.725.1334 ??Hunter ext: 16486 ? AMNH ext: 3404
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>


From matthewturk at gmail.com  Mon Jun 13 15:11:13 2011
From: matthewturk at gmail.com (Matthew Turk)
Date: Mon, 13 Jun 2011 12:11:13 -0700
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
	<BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
Message-ID: <BANLkTinvOyspdCdzB_2+=tjv1XNcPSkSUA@mail.gmail.com>

Hi all,

I'd be very interested in participating in a discussion of these
issues.  I am not an observer, but I am a member of the astrophysical
simulations community concerned with issues of interop, open science,
and developing community-focused packages for scientific analysis, and
I believe that  increasing cross-talk between the observation
community and the simulations community would be of great benefit to
all concerned.

I won't be at SciPy (haven't been able to make it since 2009,
unfortunately!) but I think this kind of discussion might warrant its
own mini-workshop.

-Matt

On Mon, Jun 13, 2011 at 12:07 PM, Tom Aldcroft
<aldcroft at head.cfa.harvard.edu> wrote:
> What about a splinter meeting at SciPy2011? ?I guess the question is
> how many interested parties will NOT be there this year.
>
> - Tom
>
> On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com> wrote:
>> I think a sit-down is desperately needed to resolve these issues, figure out
>> the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
>> progress to occur.
>> I'd be happy to participate as a non-python/programming expert and maybe
>> provide the voice of the "users".
>> I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
>> deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
>> Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that seems
>> to be where most of the movers and shakers in this game are located.
>> kelle
>>
>> On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu> wrote:
>>>
>>> That's a good idea.
>>>
>>> Perry
>>>
>>> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>>>
>>> > Maybe we should hold an AstroPy conference, where we can discuss
>>> > co-ordination, get to know each other better and even sit down and
>>> > work on libraries together (like at SciPy). That might help generate
>>> > a bit of momentum. Some of us have had meetings before that were
>>> > full of ideas that didn't go anywhere, but I don't think it has to
>>> > be that way if active people on the ground are talking to one another
>>> > rather than having institutions present their plans and try to
>>> > negotiate at a high level.
>>> >
>>> >
>>> > On 13/06/11 13:25, James Turner wrote:
>>> >> It seems that several of us would really like to improve
>>> >> collaboration on Python libraries but have been struggling to pull
>>> >> it off. I've raised the same issue on this list in past months, but
>>> >> my
>>> >> focus has unavoidably been on other things and since I'm wary of
>>> >> shouting a lot without contributing much, I haven't really been able
>>> >> to keep the discussion alive...
>>> >>
>>> >> I tend to agree with Mark and Stefan about the question of
>>> >> leadership.
>>> >> Perry & co. at Space Telescope deserve recognition for getting us
>>> >> this
>>> >> far with things like PyFITS and PyRAF. Others have taken the
>>> >> initiative
>>> >> with things like astronomical plotting and documentation sprints.
>>> >> We're
>>> >> still lacking a bit of coherence though, which (as Mark suggests) is
>>> >> likely to involve one or a few dedicated, energetic, knowledgeable,
>>> >> hands-on developer(s) who can glue things together. Those people need
>>> >> to be employed by someone, though, to ensure stability & continuity
>>> >> (fortunately there's already a bit of that going on at STScI, eg.
>>> >> with
>>> >> Mike and Matplotlib). Personally, I have the motivation but have not
>>> >> had the time/independence (and might not be assertive enough).
>>> >> Apparently we do have several energetic authors in the community now
>>> >> (like Thomas & Eli), but each with their own project.
>>> >>
>>> >> A couple of years ago, a number of us at the observatories
>>> >> submitted a
>>> >> white paper to the Decadal Survey, pointing out the need for more
>>> >> co-ordinated funding so that we can have people who focus on cross-
>>> >> institutional platform development & support. The report from the
>>> >> committee did give a nod to our concerns and their importance, but
>>> >> stopped short of making any recommendation, which basically means
>>> >> "good
>>> >> luck with that". Meanwhile, at Gemini we have had our own problems to
>>> >> deal with, which make it very difficult for me to propose something
>>> >> internally beyond working with STScI on the distribution of
>>> >> dependencies that Perry mentioned. Perhaps someone obtaining a grant
>>> >> for this is not out of the question though.
>>> >>
>>> >> I would like it if we could get together organically behind Astrolib,
>>> >> but sometimes it's difficult to get people away from their immediate
>>> >> priorities to focus on that, even within my own institution. If we
>>> >> could get people dedicated to it, though, it could become
>>> >> indispensable
>>> >> enough to attract and co-ordinate more effort. I'm just not sure
>>> >> how we
>>> >> get started at this point and my personal options for tackling the
>>> >> problem seem limited given the overarching funding transition at
>>> >> Gemini
>>> >> and the intense focus on projects that are needed to make that
>>> >> work...
>>> >>
>>> >> Cheers,
>>> >>
>>> >> James.
>>> >>
>>> >>
>>> >> On 10/06/11 09:48, Perry Greenfield wrote:
>>> >>>
>>> >>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>> >>>
>>> >>>> I just wanted to also add that (in agreement with Marshall) I'm all
>>> >>>> in favor of many small modules that accomplish a particular task
>>> >>>> well, rather than packages that aim for a 'do-it-all' approach and
>>> >>>> fall short. It's always possible to bundle small packages together
>>> >>>> afterwards, and I don't mean merge development, but instead just
>>> >>>> bundling the packages for installation (kind of like EPD). I think
>>> >>>> that is the easiest approach for all of us.
>>> >>>>
>>> >>>> Maybe in the long run, a specific set of core packages will emerge
>>> >>>> as essential and we can then talk about truly merging them into a
>>> >>>> scipy-like package, but for now, I think the race is still on. And
>>> >>>> after all, there's nothing to say we *have* to achieve the same
>>> >>>> setup as in IDL.
>>> >>>
>>> >>> It sure seems to me that the time is ripe to start trying to
>>> >>> coalesce
>>> >>> some of the ongoing efforts.
>>> >>>
>>> >>> Mind you, I don't think it is necessarily good to start with only
>>> >>> one
>>> >>> version. Allowing a few different approaches initially has its
>>> >>> benefits. You get to see more approaches and ideas in play and
>>> >>> having
>>> >>> experience with them is very helpful in deciding which one is more
>>> >>> productive. And sometimes there is room for more than one in the
>>> >>> long
>>> >>> run. The different approaches may have their own niches. But it is
>>> >>> hard to imagine any long-term need for more than two or three
>>> >>> different approaches.
>>> >>>
>>> >>> Early on there are some pragmatic needs for different approaches.
>>> >>> For
>>> >>> example, having a fairly "literal" translation of IDL tools into
>>> >>> Python has its benefit. It is very useful for those that would
>>> >>> like to
>>> >>> migrate IDL code, and given the existing IDL versions, make it much
>>> >>> easier to test their correctness. But I don't see this as a
>>> >>> substitute
>>> >>> for a good set of modular tools that have a better object design and
>>> >>> consistent interfaces with other modules. Doing this is more work
>>> >>> and
>>> >>> will take more time. So a need for both approaches is likely. Some
>>> >>> could argue the same for replacing IRAF tasks.
>>> >>>
>>> >>> All this is much easier said than done of course.
>>> >>>
>>> >>> I wish STScI had more resources to devote to this than we've
>>> >>> actually
>>> >>> had. We've been planning to do more on this front than we've
>>> >>> actually
>>> >>> done. Things come up repeatedly that ruin these plans. But it may be
>>> >>> worth saying where some of our current efforts are going that may
>>> >>> overlap some of these other efforts.
>>> >>>
>>> >>> 1) We've been planning (along with Gemini, and particularly James
>>> >>> Turner), to try to develop some Sage-like installation package that
>>> >>> would make it easy to install all the basic tools for most
>>> >>> astronomers. We had hoped to have a beta version out, but one of the
>>> >>> people working on this left at the end of last year, and we've not
>>> >>> been able to replace that person. We are going to continue this
>>> >>> effort
>>> >>> with existing staff though. Hopefully in a few months we'll have
>>> >>> something to try out.
>>> >>>
>>> >>> 2) There is a recognition that a more serious effort needs to be
>>> >>> made
>>> >>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
>>> >>> delay is that it will allow us to do some of that work more
>>> >>> explicitly. But we would not do it by replacing IRAF tasks one-for-
>>> >>> one
>>> >>> but coming up with an entirely different approach which has to start
>>> >>> from the bottom up (the end result could have applications that
>>> >>> mostly
>>> >>> emulate IRAF tasks, but also provide much more modular tools).
>>> >>>
>>> >>> 3) More specifically, we are currently focussing on how to handle
>>> >>> WCS
>>> >>> issues in a more general way than they are handled in FITS. If there
>>> >>> is interest, perhaps we should say more about the intended approach.
>>> >>> This is particularly important for replacing spectrographic tools in
>>> >>> IRAF, and this is where we are starting our effort.
>>> >>>
>>> >>> 4) We need a way of saving these WCS models, and FITS is not the
>>> >>> way.
>>> >>> We are looking at an alternate data format, not just for WCS models,
>>> >>> but for data in general [gasp!]. Work has begun on this too.
>>> >>>
>>> >>> 5) A lot of our recent work has been on pysynphot and ETCs. We
>>> >>> plan on
>>> >>> making the computational part of our ETCs a released tool. But I'm
>>> >>> also wondering if we can generalize the pysynphot spectral models
>>> >>> for
>>> >>> more general use in spectral tools.
>>> >>>
>>> >>> 6) We have been working on a framework for making pipelines easier
>>> >>> to
>>> >>> build and configure. That won't be ready for at least a few months,
>>> >>> but could well be of general interest and use.
>>> >>>
>>> >>> But besides these things, I would like to see if we can't begin the
>>> >>> effort of narrowing some of the underlying libraries being used.
>>> >>> FITS
>>> >>> WCS is one obvious area that seems ripe for that.
>>> >>>
>>> >>> But the community ought to identify one or two areas that are of the
>>> >>> most interest in consolidating (let's start small). What should we
>>> >>> start with? Focus is important in making any progress in this area.
>>> >>>
>>> >>> Perry
>>> >>>
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> AstroPy mailing list
>>> >>> AstroPy at scipy.org
>>> >>> http://mail.scipy.org/mailman/listinfo/astropy
>>> >
>>> >
>>> > --
>>> > James E.H. Turner
>>> > Gemini Observatory Southern Operations Centre,
>>> > Casilla 603, ? ? ? ? ?Tel. (+56) 51 205609
>>> > La Serena, Chile. ? ? Fax. (+56) 51 205650
>>> > _______________________________________________
>>> > AstroPy mailing list
>>> > AstroPy at scipy.org
>>> > http://mail.scipy.org/mailman/listinfo/astropy
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>>
>> --
>> Kelle Cruz, PhD ??http://kellecruz.com/
>> 917.725.1334 ??Hunter ext: 16486 ? AMNH ext: 3404
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From perry at stsci.edu  Mon Jun 13 15:15:28 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Mon, 13 Jun 2011 15:15:28 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
	<BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
Message-ID: <D3C94402-208C-4FEF-B158-0EE12D495E5F@stsci.edu>

It may be too late. Those attending may have already made travel plans  
that preclude an extra day or two (I think it deserves at least a  
day). And it may be too late for those that weren't planning to go.

Perry

On Jun 13, 2011, at 3:07 PM, Tom Aldcroft wrote:

> What about a splinter meeting at SciPy2011?  I guess the question is
> how many interested parties will NOT be there this year.
>
> - Tom
>
> On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com>  
> wrote:
>> I think a sit-down is desperately needed to resolve these issues,  
>> figure out
>> the mgmt structure (aka, pecking order), for the BDFL to emerge,  
>> and for
>> progress to occur.
>> I'd be happy to participate as a non-python/programming expert and  
>> maybe
>> provide the voice of the "users".
>> I propose there be a Splinter Meeting at AAS in Austin. (Splinter  
>> Meeting
>> deadline is Dec 1.) Or else someone will have to organize at CfA  
>> (Tom A?
>> Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell,  
>> that seems
>> to be where most of the movers and shakers in this game are located.
>> kelle
>>
>> On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu>  
>> wrote:
>>>
>>> That's a good idea.
>>>
>>> Perry
>>>
>>> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>>>
>>>> Maybe we should hold an AstroPy conference, where we can discuss
>>>> co-ordination, get to know each other better and even sit down and
>>>> work on libraries together (like at SciPy). That might help  
>>>> generate
>>>> a bit of momentum. Some of us have had meetings before that were
>>>> full of ideas that didn't go anywhere, but I don't think it has to
>>>> be that way if active people on the ground are talking to one  
>>>> another
>>>> rather than having institutions present their plans and try to
>>>> negotiate at a high level.
>>>>
>>>>
>>>> On 13/06/11 13:25, James Turner wrote:
>>>>> It seems that several of us would really like to improve
>>>>> collaboration on Python libraries but have been struggling to pull
>>>>> it off. I've raised the same issue on this list in past months,  
>>>>> but
>>>>> my
>>>>> focus has unavoidably been on other things and since I'm wary of
>>>>> shouting a lot without contributing much, I haven't really been  
>>>>> able
>>>>> to keep the discussion alive...
>>>>>
>>>>> I tend to agree with Mark and Stefan about the question of
>>>>> leadership.
>>>>> Perry & co. at Space Telescope deserve recognition for getting us
>>>>> this
>>>>> far with things like PyFITS and PyRAF. Others have taken the
>>>>> initiative
>>>>> with things like astronomical plotting and documentation sprints.
>>>>> We're
>>>>> still lacking a bit of coherence though, which (as Mark  
>>>>> suggests) is
>>>>> likely to involve one or a few dedicated, energetic,  
>>>>> knowledgeable,
>>>>> hands-on developer(s) who can glue things together. Those people  
>>>>> need
>>>>> to be employed by someone, though, to ensure stability &  
>>>>> continuity
>>>>> (fortunately there's already a bit of that going on at STScI, eg.
>>>>> with
>>>>> Mike and Matplotlib). Personally, I have the motivation but have  
>>>>> not
>>>>> had the time/independence (and might not be assertive enough).
>>>>> Apparently we do have several energetic authors in the community  
>>>>> now
>>>>> (like Thomas & Eli), but each with their own project.
>>>>>
>>>>> A couple of years ago, a number of us at the observatories
>>>>> submitted a
>>>>> white paper to the Decadal Survey, pointing out the need for more
>>>>> co-ordinated funding so that we can have people who focus on  
>>>>> cross-
>>>>> institutional platform development & support. The report from the
>>>>> committee did give a nod to our concerns and their importance, but
>>>>> stopped short of making any recommendation, which basically means
>>>>> "good
>>>>> luck with that". Meanwhile, at Gemini we have had our own  
>>>>> problems to
>>>>> deal with, which make it very difficult for me to propose  
>>>>> something
>>>>> internally beyond working with STScI on the distribution of
>>>>> dependencies that Perry mentioned. Perhaps someone obtaining a  
>>>>> grant
>>>>> for this is not out of the question though.
>>>>>
>>>>> I would like it if we could get together organically behind  
>>>>> Astrolib,
>>>>> but sometimes it's difficult to get people away from their  
>>>>> immediate
>>>>> priorities to focus on that, even within my own institution. If we
>>>>> could get people dedicated to it, though, it could become
>>>>> indispensable
>>>>> enough to attract and co-ordinate more effort. I'm just not sure
>>>>> how we
>>>>> get started at this point and my personal options for tackling the
>>>>> problem seem limited given the overarching funding transition at
>>>>> Gemini
>>>>> and the intense focus on projects that are needed to make that
>>>>> work...
>>>>>
>>>>> Cheers,
>>>>>
>>>>> James.
>>>>>
>>>>>
>>>>> On 10/06/11 09:48, Perry Greenfield wrote:
>>>>>>
>>>>>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>>>>
>>>>>>> I just wanted to also add that (in agreement with Marshall)  
>>>>>>> I'm all
>>>>>>> in favor of many small modules that accomplish a particular task
>>>>>>> well, rather than packages that aim for a 'do-it-all' approach  
>>>>>>> and
>>>>>>> fall short. It's always possible to bundle small packages  
>>>>>>> together
>>>>>>> afterwards, and I don't mean merge development, but instead just
>>>>>>> bundling the packages for installation (kind of like EPD). I  
>>>>>>> think
>>>>>>> that is the easiest approach for all of us.
>>>>>>>
>>>>>>> Maybe in the long run, a specific set of core packages will  
>>>>>>> emerge
>>>>>>> as essential and we can then talk about truly merging them  
>>>>>>> into a
>>>>>>> scipy-like package, but for now, I think the race is still on.  
>>>>>>> And
>>>>>>> after all, there's nothing to say we *have* to achieve the same
>>>>>>> setup as in IDL.
>>>>>>
>>>>>> It sure seems to me that the time is ripe to start trying to
>>>>>> coalesce
>>>>>> some of the ongoing efforts.
>>>>>>
>>>>>> Mind you, I don't think it is necessarily good to start with only
>>>>>> one
>>>>>> version. Allowing a few different approaches initially has its
>>>>>> benefits. You get to see more approaches and ideas in play and
>>>>>> having
>>>>>> experience with them is very helpful in deciding which one is  
>>>>>> more
>>>>>> productive. And sometimes there is room for more than one in the
>>>>>> long
>>>>>> run. The different approaches may have their own niches. But it  
>>>>>> is
>>>>>> hard to imagine any long-term need for more than two or three
>>>>>> different approaches.
>>>>>>
>>>>>> Early on there are some pragmatic needs for different approaches.
>>>>>> For
>>>>>> example, having a fairly "literal" translation of IDL tools into
>>>>>> Python has its benefit. It is very useful for those that would
>>>>>> like to
>>>>>> migrate IDL code, and given the existing IDL versions, make it  
>>>>>> much
>>>>>> easier to test their correctness. But I don't see this as a
>>>>>> substitute
>>>>>> for a good set of modular tools that have a better object  
>>>>>> design and
>>>>>> consistent interfaces with other modules. Doing this is more work
>>>>>> and
>>>>>> will take more time. So a need for both approaches is likely.  
>>>>>> Some
>>>>>> could argue the same for replacing IRAF tasks.
>>>>>>
>>>>>> All this is much easier said than done of course.
>>>>>>
>>>>>> I wish STScI had more resources to devote to this than we've
>>>>>> actually
>>>>>> had. We've been planning to do more on this front than we've
>>>>>> actually
>>>>>> done. Things come up repeatedly that ruin these plans. But it  
>>>>>> may be
>>>>>> worth saying where some of our current efforts are going that may
>>>>>> overlap some of these other efforts.
>>>>>>
>>>>>> 1) We've been planning (along with Gemini, and particularly James
>>>>>> Turner), to try to develop some Sage-like installation package  
>>>>>> that
>>>>>> would make it easy to install all the basic tools for most
>>>>>> astronomers. We had hoped to have a beta version out, but one  
>>>>>> of the
>>>>>> people working on this left at the end of last year, and we've  
>>>>>> not
>>>>>> been able to replace that person. We are going to continue this
>>>>>> effort
>>>>>> with existing staff though. Hopefully in a few months we'll have
>>>>>> something to try out.
>>>>>>
>>>>>> 2) There is a recognition that a more serious effort needs to be
>>>>>> made
>>>>>> to replace IRAF functionality. Perhaps one of the benefits of a  
>>>>>> JWST
>>>>>> delay is that it will allow us to do some of that work more
>>>>>> explicitly. But we would not do it by replacing IRAF tasks one- 
>>>>>> for-
>>>>>> one
>>>>>> but coming up with an entirely different approach which has to  
>>>>>> start
>>>>>> from the bottom up (the end result could have applications that
>>>>>> mostly
>>>>>> emulate IRAF tasks, but also provide much more modular tools).
>>>>>>
>>>>>> 3) More specifically, we are currently focussing on how to handle
>>>>>> WCS
>>>>>> issues in a more general way than they are handled in FITS. If  
>>>>>> there
>>>>>> is interest, perhaps we should say more about the intended  
>>>>>> approach.
>>>>>> This is particularly important for replacing spectrographic  
>>>>>> tools in
>>>>>> IRAF, and this is where we are starting our effort.
>>>>>>
>>>>>> 4) We need a way of saving these WCS models, and FITS is not the
>>>>>> way.
>>>>>> We are looking at an alternate data format, not just for WCS  
>>>>>> models,
>>>>>> but for data in general [gasp!]. Work has begun on this too.
>>>>>>
>>>>>> 5) A lot of our recent work has been on pysynphot and ETCs. We
>>>>>> plan on
>>>>>> making the computational part of our ETCs a released tool. But  
>>>>>> I'm
>>>>>> also wondering if we can generalize the pysynphot spectral models
>>>>>> for
>>>>>> more general use in spectral tools.
>>>>>>
>>>>>> 6) We have been working on a framework for making pipelines  
>>>>>> easier
>>>>>> to
>>>>>> build and configure. That won't be ready for at least a few  
>>>>>> months,
>>>>>> but could well be of general interest and use.
>>>>>>
>>>>>> But besides these things, I would like to see if we can't begin  
>>>>>> the
>>>>>> effort of narrowing some of the underlying libraries being used.
>>>>>> FITS
>>>>>> WCS is one obvious area that seems ripe for that.
>>>>>>
>>>>>> But the community ought to identify one or two areas that are  
>>>>>> of the
>>>>>> most interest in consolidating (let's start small). What should  
>>>>>> we
>>>>>> start with? Focus is important in making any progress in this  
>>>>>> area.
>>>>>>
>>>>>> Perry
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> AstroPy mailing list
>>>>>> AstroPy at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>>
>>>> --
>>>> James E.H. Turner
>>>> Gemini Observatory Southern Operations Centre,
>>>> Casilla 603,          Tel. (+56) 51 205609
>>>> La Serena, Chile.     Fax. (+56) 51 205650
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>>
>> --
>> Kelle Cruz, PhD ? http://kellecruz.com/
>> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From kellecruz at gmail.com  Mon Jun 13 15:23:05 2011
From: kellecruz at gmail.com (Kelle Cruz)
Date: Mon, 13 Jun 2011 15:23:05 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
	<BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
Message-ID: <BANLkTinLndKjGCt7OWWXrSuZ+X51t2LiZg@mail.gmail.com>

For those not in the know, like I was, SciPy 2011 is July 11-16 in Austin.
Registration is $200 for Academics and $150 for students.

The one thing I can say about this is that it is CRAZY hot in the Texas Hill
Country right now and it's only going to get worse. (My parents live halfway
between San Antonio and Austin.)  If you decided to go with this option, I
would likely not attend.

That said, if there are > 3 ppl on this list already planning on going, then
you guys should definitely get together to have something a bit more formal
than just a chat. You could plan the next meeting, evaluate the existing
packages, start to sketch out a roadmap, maybe even assign some tasks, or
create some leadership roles (e.g., Lead of Phot packages, Lead of Spectra
packages, Lead of Xray), etc.

kelle

On Mon, Jun 13, 2011 at 3:07 PM, Tom Aldcroft <aldcroft at head.cfa.harvard.edu
> wrote:

> What about a splinter meeting at SciPy2011?  I guess the question is
> how many interested parties will NOT be there this year.
>
> - Tom
>
> On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com> wrote:
> > I think a sit-down is desperately needed to resolve these issues, figure
> out
> > the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
> > progress to occur.
> > I'd be happy to participate as a non-python/programming expert and maybe
> > provide the voice of the "users".
> > I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
> > deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
> > Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that
> seems
> > to be where most of the movers and shakers in this game are located.
> > kelle
> >
> > On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu>
> wrote:
> >>
> >> That's a good idea.
> >>
> >> Perry
> >>
> >> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
> >>
> >> > Maybe we should hold an AstroPy conference, where we can discuss
> >> > co-ordination, get to know each other better and even sit down and
> >> > work on libraries together (like at SciPy). That might help generate
> >> > a bit of momentum. Some of us have had meetings before that were
> >> > full of ideas that didn't go anywhere, but I don't think it has to
> >> > be that way if active people on the ground are talking to one another
> >> > rather than having institutions present their plans and try to
> >> > negotiate at a high level.
> >> >
> >> >
> >> > On 13/06/11 13:25, James Turner wrote:
> >> >> It seems that several of us would really like to improve
> >> >> collaboration on Python libraries but have been struggling to pull
> >> >> it off. I've raised the same issue on this list in past months, but
> >> >> my
> >> >> focus has unavoidably been on other things and since I'm wary of
> >> >> shouting a lot without contributing much, I haven't really been able
> >> >> to keep the discussion alive...
> >> >>
> >> >> I tend to agree with Mark and Stefan about the question of
> >> >> leadership.
> >> >> Perry & co. at Space Telescope deserve recognition for getting us
> >> >> this
> >> >> far with things like PyFITS and PyRAF. Others have taken the
> >> >> initiative
> >> >> with things like astronomical plotting and documentation sprints.
> >> >> We're
> >> >> still lacking a bit of coherence though, which (as Mark suggests) is
> >> >> likely to involve one or a few dedicated, energetic, knowledgeable,
> >> >> hands-on developer(s) who can glue things together. Those people need
> >> >> to be employed by someone, though, to ensure stability & continuity
> >> >> (fortunately there's already a bit of that going on at STScI, eg.
> >> >> with
> >> >> Mike and Matplotlib). Personally, I have the motivation but have not
> >> >> had the time/independence (and might not be assertive enough).
> >> >> Apparently we do have several energetic authors in the community now
> >> >> (like Thomas & Eli), but each with their own project.
> >> >>
> >> >> A couple of years ago, a number of us at the observatories
> >> >> submitted a
> >> >> white paper to the Decadal Survey, pointing out the need for more
> >> >> co-ordinated funding so that we can have people who focus on cross-
> >> >> institutional platform development & support. The report from the
> >> >> committee did give a nod to our concerns and their importance, but
> >> >> stopped short of making any recommendation, which basically means
> >> >> "good
> >> >> luck with that". Meanwhile, at Gemini we have had our own problems to
> >> >> deal with, which make it very difficult for me to propose something
> >> >> internally beyond working with STScI on the distribution of
> >> >> dependencies that Perry mentioned. Perhaps someone obtaining a grant
> >> >> for this is not out of the question though.
> >> >>
> >> >> I would like it if we could get together organically behind Astrolib,
> >> >> but sometimes it's difficult to get people away from their immediate
> >> >> priorities to focus on that, even within my own institution. If we
> >> >> could get people dedicated to it, though, it could become
> >> >> indispensable
> >> >> enough to attract and co-ordinate more effort. I'm just not sure
> >> >> how we
> >> >> get started at this point and my personal options for tackling the
> >> >> problem seem limited given the overarching funding transition at
> >> >> Gemini
> >> >> and the intense focus on projects that are needed to make that
> >> >> work...
> >> >>
> >> >> Cheers,
> >> >>
> >> >> James.
> >> >>
> >> >>
> >> >> On 10/06/11 09:48, Perry Greenfield wrote:
> >> >>>
> >> >>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
> >> >>>
> >> >>>> I just wanted to also add that (in agreement with Marshall) I'm all
> >> >>>> in favor of many small modules that accomplish a particular task
> >> >>>> well, rather than packages that aim for a 'do-it-all' approach and
> >> >>>> fall short. It's always possible to bundle small packages together
> >> >>>> afterwards, and I don't mean merge development, but instead just
> >> >>>> bundling the packages for installation (kind of like EPD). I think
> >> >>>> that is the easiest approach for all of us.
> >> >>>>
> >> >>>> Maybe in the long run, a specific set of core packages will emerge
> >> >>>> as essential and we can then talk about truly merging them into a
> >> >>>> scipy-like package, but for now, I think the race is still on. And
> >> >>>> after all, there's nothing to say we *have* to achieve the same
> >> >>>> setup as in IDL.
> >> >>>
> >> >>> It sure seems to me that the time is ripe to start trying to
> >> >>> coalesce
> >> >>> some of the ongoing efforts.
> >> >>>
> >> >>> Mind you, I don't think it is necessarily good to start with only
> >> >>> one
> >> >>> version. Allowing a few different approaches initially has its
> >> >>> benefits. You get to see more approaches and ideas in play and
> >> >>> having
> >> >>> experience with them is very helpful in deciding which one is more
> >> >>> productive. And sometimes there is room for more than one in the
> >> >>> long
> >> >>> run. The different approaches may have their own niches. But it is
> >> >>> hard to imagine any long-term need for more than two or three
> >> >>> different approaches.
> >> >>>
> >> >>> Early on there are some pragmatic needs for different approaches.
> >> >>> For
> >> >>> example, having a fairly "literal" translation of IDL tools into
> >> >>> Python has its benefit. It is very useful for those that would
> >> >>> like to
> >> >>> migrate IDL code, and given the existing IDL versions, make it much
> >> >>> easier to test their correctness. But I don't see this as a
> >> >>> substitute
> >> >>> for a good set of modular tools that have a better object design and
> >> >>> consistent interfaces with other modules. Doing this is more work
> >> >>> and
> >> >>> will take more time. So a need for both approaches is likely. Some
> >> >>> could argue the same for replacing IRAF tasks.
> >> >>>
> >> >>> All this is much easier said than done of course.
> >> >>>
> >> >>> I wish STScI had more resources to devote to this than we've
> >> >>> actually
> >> >>> had. We've been planning to do more on this front than we've
> >> >>> actually
> >> >>> done. Things come up repeatedly that ruin these plans. But it may be
> >> >>> worth saying where some of our current efforts are going that may
> >> >>> overlap some of these other efforts.
> >> >>>
> >> >>> 1) We've been planning (along with Gemini, and particularly James
> >> >>> Turner), to try to develop some Sage-like installation package that
> >> >>> would make it easy to install all the basic tools for most
> >> >>> astronomers. We had hoped to have a beta version out, but one of the
> >> >>> people working on this left at the end of last year, and we've not
> >> >>> been able to replace that person. We are going to continue this
> >> >>> effort
> >> >>> with existing staff though. Hopefully in a few months we'll have
> >> >>> something to try out.
> >> >>>
> >> >>> 2) There is a recognition that a more serious effort needs to be
> >> >>> made
> >> >>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
> >> >>> delay is that it will allow us to do some of that work more
> >> >>> explicitly. But we would not do it by replacing IRAF tasks one-for-
> >> >>> one
> >> >>> but coming up with an entirely different approach which has to start
> >> >>> from the bottom up (the end result could have applications that
> >> >>> mostly
> >> >>> emulate IRAF tasks, but also provide much more modular tools).
> >> >>>
> >> >>> 3) More specifically, we are currently focussing on how to handle
> >> >>> WCS
> >> >>> issues in a more general way than they are handled in FITS. If there
> >> >>> is interest, perhaps we should say more about the intended approach.
> >> >>> This is particularly important for replacing spectrographic tools in
> >> >>> IRAF, and this is where we are starting our effort.
> >> >>>
> >> >>> 4) We need a way of saving these WCS models, and FITS is not the
> >> >>> way.
> >> >>> We are looking at an alternate data format, not just for WCS models,
> >> >>> but for data in general [gasp!]. Work has begun on this too.
> >> >>>
> >> >>> 5) A lot of our recent work has been on pysynphot and ETCs. We
> >> >>> plan on
> >> >>> making the computational part of our ETCs a released tool. But I'm
> >> >>> also wondering if we can generalize the pysynphot spectral models
> >> >>> for
> >> >>> more general use in spectral tools.
> >> >>>
> >> >>> 6) We have been working on a framework for making pipelines easier
> >> >>> to
> >> >>> build and configure. That won't be ready for at least a few months,
> >> >>> but could well be of general interest and use.
> >> >>>
> >> >>> But besides these things, I would like to see if we can't begin the
> >> >>> effort of narrowing some of the underlying libraries being used.
> >> >>> FITS
> >> >>> WCS is one obvious area that seems ripe for that.
> >> >>>
> >> >>> But the community ought to identify one or two areas that are of the
> >> >>> most interest in consolidating (let's start small). What should we
> >> >>> start with? Focus is important in making any progress in this area.
> >> >>>
> >> >>> Perry
> >> >>>
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> AstroPy mailing list
> >> >>> AstroPy at scipy.org
> >> >>> http://mail.scipy.org/mailman/listinfo/astropy
> >> >
> >> >
> >> > --
> >> > James E.H. Turner
> >> > Gemini Observatory Southern Operations Centre,
> >> > Casilla 603,          Tel. (+56) 51 205609
> >> > La Serena, Chile.     Fax. (+56) 51 205650
> >> > _______________________________________________
> >> > AstroPy mailing list
> >> > AstroPy at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/astropy
> >>
> >> _______________________________________________
> >> AstroPy mailing list
> >> AstroPy at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/astropy
> >
> >
> >
> > --
> > Kelle Cruz, PhD ? http://kellecruz.com/
> > 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
> >
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org
> > http://mail.scipy.org/mailman/listinfo/astropy
> >
> >
>



-- 
Kelle Cruz, PhD ? http://kellecruz.com/
917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110613/ad224ff8/attachment.html>

From tgrav at mac.com  Mon Jun 13 15:25:22 2011
From: tgrav at mac.com (Tommy Grav)
Date: Mon, 13 Jun 2011 15:25:22 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <D3C94402-208C-4FEF-B158-0EE12D495E5F@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
	<BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
	<D3C94402-208C-4FEF-B158-0EE12D495E5F@stsci.edu>
Message-ID: <C6C240BE-2AF3-4D8A-976E-0EB5C5937809@mac.com>

I think this discussion is very good. As an astronomer that does most of my 
programming in Python it certainly would be easier to contribute effort to 
a project that have some more unified community support similar to scipy  
and numpy. 

One quick question. I have not looked at pyRAF in a while, but it still is only
a wrapper of the IRAF routines, right? Has anyone considered migrating away
from the IRAF backend to build pure python or C/Fortran wrapped packages
of the same routines? To me the installation of of IRAF and then pyRAF on top
of that has always been a barrier I never have quite made it through. I know
that one can install the Stsci package, but last time I looked it installs itself
on the root directory and consists of way to much non-python related stuff.
I find that hard to deal with as I as a mac-user use Activestate python with
the framework structure, which to me is very clean and easy to keep organized.

Cheers
   Tommy




From rahman at astro.utoronto.ca  Mon Jun 13 15:25:41 2011
From: rahman at astro.utoronto.ca (Mubdi Rahman)
Date: Mon, 13 Jun 2011 20:25:41 +0100 (BST)
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTinvOyspdCdzB_2+=tjv1XNcPSkSUA@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
	<BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
	<BANLkTinvOyspdCdzB_2+=tjv1XNcPSkSUA@mail.gmail.com>
Message-ID: <20360.14542.qm@web29009.mail.ird.yahoo.com>

Hi everyone,

Considering that it may be especially difficult for some of us to get funding for a workshop/meeting that isn't directly related to our research, and leaving something like this to the AAS may be leaving it a bit too long, might it be worthwhile to investigate having an online/web conference? This way, there wouldn't necessarily be any travel, and some of us in the bigger centres could book web conferencing rooms while those in smaller centres could still participate with their webcam.?

We could use something like Cisco WebEx (which has an upper limit of 25 participants) or something similar? This way, we could do something in a month or two.

-Mubdi




>________________________________
>From: Matthew Turk <matthewturk at gmail.com>
>To: Tom Aldcroft <aldcroft at head.cfa.harvard.edu>
>Cc: astropy <astropy at scipy.org>
>Sent: Monday, June 13, 2011 3:11:13 PM
>Subject: Re: [AstroPy] Proliferating py-astro-libs
>
>Hi all,
>
>I'd be very interested in participating in a discussion of these
>issues.? I am not an observer, but I am a member of the astrophysical
>simulations community concerned with issues of interop, open science,
>and developing community-focused packages for scientific analysis, and
>I believe that? increasing cross-talk between the observation
>community and the simulations community would be of great benefit to
>all concerned.
>
>I won't be at SciPy (haven't been able to make it since 2009,
>unfortunately!) but I think this kind of discussion might warrant its
>own mini-workshop.
>
>-Matt
>
>On Mon, Jun 13, 2011 at 12:07 PM, Tom Aldcroft
><aldcroft at head.cfa.harvard.edu> wrote:
>> What about a splinter meeting at SciPy2011? ?I guess the question is
>> how many interested parties will NOT be there this year.
>>
>> - Tom
>>
>> On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com> wrote:
>>> I think a sit-down is desperately needed to resolve these issues, figure out
>>> the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
>>> progress to occur.
>>> I'd be happy to participate as a non-python/programming expert and maybe
>>> provide the voice of the "users".
>>> I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
>>> deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
>>> Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that seems
>>> to be where most of the movers and shakers in this game are located.
>>> kelle
>>>
>>> On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu> wrote:
>>>>
>>>> That's a good idea.
>>>>
>>>> Perry
>>>>
>>>> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>>>>
>>>> > Maybe we should hold an AstroPy conference, where we can discuss
>>>> > co-ordination, get to know each other better and even sit down and
>>>> > work on libraries together (like at SciPy). That might help generate
>>>> > a bit of momentum. Some of us have had meetings before that were
>>>> > full of ideas that didn't go anywhere, but I don't think it has to
>>>> > be that way if active people on the ground are talking to one another
>>>> > rather than having institutions present their plans and try to
>>>> > negotiate at a high level.
>>>> >
>>>> >
>>>> > On 13/06/11 13:25, James Turner wrote:
>>>> >> It seems that several of us would really like to improve
>>>> >> collaboration on Python libraries but have been struggling to pull
>>>> >> it off. I've raised the same issue on this list in past months, but
>>>> >> my
>>>> >> focus has unavoidably been on other things and since I'm wary of
>>>> >> shouting a lot without contributing much, I haven't really been able
>>>> >> to keep the discussion alive...
>>>> >>
>>>> >> I tend to agree with Mark and Stefan about the question of
>>>> >> leadership.
>>>> >> Perry & co. at Space Telescope deserve recognition for getting us
>>>> >> this
>>>> >> far with things like PyFITS and PyRAF. Others have taken the
>>>> >> initiative
>>>> >> with things like astronomical plotting and documentation sprints.
>>>> >> We're
>>>> >> still lacking a bit of coherence though, which (as Mark suggests) is
>>>> >> likely to involve one or a few dedicated, energetic, knowledgeable,
>>>> >> hands-on developer(s) who can glue things together. Those people need
>>>> >> to be employed by someone, though, to ensure stability & continuity
>>>> >> (fortunately there's already a bit of that going on at STScI, eg.
>>>> >> with
>>>> >> Mike and Matplotlib). Personally, I have the motivation but have not
>>>> >> had the time/independence (and might not be assertive enough).
>>>> >> Apparently we do have several energetic authors in the community now
>>>> >> (like Thomas & Eli), but each with their own project.
>>>> >>
>>>> >> A couple of years ago, a number of us at the observatories
>>>> >> submitted a
>>>> >> white paper to the Decadal Survey, pointing out the need for more
>>>> >> co-ordinated funding so that we can have people who focus on cross-
>>>> >> institutional platform development & support. The report from the
>>>> >> committee did give a nod to our concerns and their importance, but
>>>> >> stopped short of making any recommendation, which basically means
>>>> >> "good
>>>> >> luck with that". Meanwhile, at Gemini we have had our own problems to
>>>> >> deal with, which make it very difficult for me to propose something
>>>> >> internally beyond working with STScI on the distribution of
>>>> >> dependencies that Perry mentioned. Perhaps someone obtaining a grant
>>>> >> for this is not out of the question though.
>>>> >>
>>>> >> I would like it if we could get together organically behind Astrolib,
>>>> >> but sometimes it's difficult to get people away from their immediate
>>>> >> priorities to focus on that, even within my own institution. If we
>>>> >> could get people dedicated to it, though, it could become
>>>> >> indispensable
>>>> >> enough to attract and co-ordinate more effort. I'm just not sure
>>>> >> how we
>>>> >> get started at this point and my personal options for tackling the
>>>> >> problem seem limited given the overarching funding transition at
>>>> >> Gemini
>>>> >> and the intense focus on projects that are needed to make that
>>>> >> work...
>>>> >>
>>>> >> Cheers,
>>>> >>
>>>> >> James.
>>>> >>
>>>> >>
>>>> >> On 10/06/11 09:48, Perry Greenfield wrote:
>>>> >>>
>>>> >>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>> >>>
>>>> >>>> I just wanted to also add that (in agreement with Marshall) I'm all
>>>> >>>> in favor of many small modules that accomplish a particular task
>>>> >>>> well, rather than packages that aim for a 'do-it-all' approach and
>>>> >>>> fall short. It's always possible to bundle small packages together
>>>> >>>> afterwards, and I don't mean merge development, but instead just
>>>> >>>> bundling the packages for installation (kind of like EPD). I think
>>>> >>>> that is the easiest approach for all of us.
>>>> >>>>
>>>> >>>> Maybe in the long run, a specific set of core packages will emerge
>>>> >>>> as essential and we can then talk about truly merging them into a
>>>> >>>> scipy-like package, but for now, I think the race is still on. And
>>>> >>>> after all, there's nothing to say we *have* to achieve the same
>>>> >>>> setup as in IDL.
>>>> >>>
>>>> >>> It sure seems to me that the time is ripe to start trying to
>>>> >>> coalesce
>>>> >>> some of the ongoing efforts.
>>>> >>>
>>>> >>> Mind you, I don't think it is necessarily good to start with only
>>>> >>> one
>>>> >>> version. Allowing a few different approaches initially has its
>>>> >>> benefits. You get to see more approaches and ideas in play and
>>>> >>> having
>>>> >>> experience with them is very helpful in deciding which one is more
>>>> >>> productive. And sometimes there is room for more than one in the
>>>> >>> long
>>>> >>> run. The different approaches may have their own niches. But it is
>>>> >>> hard to imagine any long-term need for more than two or three
>>>> >>> different approaches.
>>>> >>>
>>>> >>> Early on there are some pragmatic needs for different approaches.
>>>> >>> For
>>>> >>> example, having a fairly "literal" translation of IDL tools into
>>>> >>> Python has its benefit. It is very useful for those that would
>>>> >>> like to
>>>> >>> migrate IDL code, and given the existing IDL versions, make it much
>>>> >>> easier to test their correctness. But I don't see this as a
>>>> >>> substitute
>>>> >>> for a good set of modular tools that have a better object design and
>>>> >>> consistent interfaces with other modules. Doing this is more work
>>>> >>> and
>>>> >>> will take more time. So a need for both approaches is likely. Some
>>>> >>> could argue the same for replacing IRAF tasks.
>>>> >>>
>>>> >>> All this is much easier said than done of course.
>>>> >>>
>>>> >>> I wish STScI had more resources to devote to this than we've
>>>> >>> actually
>>>> >>> had. We've been planning to do more on this front than we've
>>>> >>> actually
>>>> >>> done. Things come up repeatedly that ruin these plans. But it may be
>>>> >>> worth saying where some of our current efforts are going that may
>>>> >>> overlap some of these other efforts.
>>>> >>>
>>>> >>> 1) We've been planning (along with Gemini, and particularly James
>>>> >>> Turner), to try to develop some Sage-like installation package that
>>>> >>> would make it easy to install all the basic tools for most
>>>> >>> astronomers. We had hoped to have a beta version out, but one of the
>>>> >>> people working on this left at the end of last year, and we've not
>>>> >>> been able to replace that person. We are going to continue this
>>>> >>> effort
>>>> >>> with existing staff though. Hopefully in a few months we'll have
>>>> >>> something to try out.
>>>> >>>
>>>> >>> 2) There is a recognition that a more serious effort needs to be
>>>> >>> made
>>>> >>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
>>>> >>> delay is that it will allow us to do some of that work more
>>>> >>> explicitly. But we would not do it by replacing IRAF tasks one-for-
>>>> >>> one
>>>> >>> but coming up with an entirely different approach which has to start
>>>> >>> from the bottom up (the end result could have applications that
>>>> >>> mostly
>>>> >>> emulate IRAF tasks, but also provide much more modular tools).
>>>> >>>
>>>> >>> 3) More specifically, we are currently focussing on how to handle
>>>> >>> WCS
>>>> >>> issues in a more general way than they are handled in FITS. If there
>>>> >>> is interest, perhaps we should say more about the intended approach.
>>>> >>> This is particularly important for replacing spectrographic tools in
>>>> >>> IRAF, and this is where we are starting our effort.
>>>> >>>
>>>> >>> 4) We need a way of saving these WCS models, and FITS is not the
>>>> >>> way.
>>>> >>> We are looking at an alternate data format, not just for WCS models,
>>>> >>> but for data in general [gasp!]. Work has begun on this too.
>>>> >>>
>>>> >>> 5) A lot of our recent work has been on pysynphot and ETCs. We
>>>> >>> plan on
>>>> >>> making the computational part of our ETCs a released tool. But I'm
>>>> >>> also wondering if we can generalize the pysynphot spectral models
>>>> >>> for
>>>> >>> more general use in spectral tools.
>>>> >>>
>>>> >>> 6) We have been working on a framework for making pipelines easier
>>>> >>> to
>>>> >>> build and configure. That won't be ready for at least a few months,
>>>> >>> but could well be of general interest and use.
>>>> >>>
>>>> >>> But besides these things, I would like to see if we can't begin the
>>>> >>> effort of narrowing some of the underlying libraries being used.
>>>> >>> FITS
>>>> >>> WCS is one obvious area that seems ripe for that.
>>>> >>>
>>>> >>> But the community ought to identify one or two areas that are of the
>>>> >>> most interest in consolidating (let's start small). What should we
>>>> >>> start with? Focus is important in making any progress in this area.
>>>> >>>
>>>> >>> Perry
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> _______________________________________________
>>>> >>> AstroPy mailing list
>>>> >>> AstroPy at scipy.org
>>>> >>> http://mail.scipy.org/mailman/listinfo/astropy
>>>> >
>>>> >
>>>> > --
>>>> > James E.H. Turner
>>>> > Gemini Observatory Southern Operations Centre,
>>>> > Casilla 603, ? ? ? ? ?Tel. (+56) 51 205609
>>>> > La Serena, Chile. ? ? Fax. (+56) 51 205650
>>>> > _______________________________________________
>>>> > AstroPy mailing list
>>>> > AstroPy at scipy.org
>>>> > http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>>>
>>>
>>> --
>>> Kelle Cruz, PhD ??http://kellecruz.com/
>>> 917.725.1334 ??Hunter ext: 16486 ? AMNH ext: 3404
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>_______________________________________________
>AstroPy mailing list
>AstroPy at scipy.org
>http://mail.scipy.org/mailman/listinfo/astropy
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110613/3dd7f867/attachment.html>

From perry at stsci.edu  Mon Jun 13 15:29:45 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Mon, 13 Jun 2011 15:29:45 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <C6C240BE-2AF3-4D8A-976E-0EB5C5937809@mac.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
	<BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
	<D3C94402-208C-4FEF-B158-0EE12D495E5F@stsci.edu>
	<C6C240BE-2AF3-4D8A-976E-0EB5C5937809@mac.com>
Message-ID: <A7D51446-0B19-43FF-A0B4-48259F7DAFA7@stsci.edu>

Oh yes. This is increasingly becoming more important. There are bits  
and pieces, but this is  one area where we really want to start making  
progress. One package that is often cited as important is the twodspec  
package, so we are beginning to start working on replacing  the  
functionality there, and not just for space-based needs.

Perry


On Jun 13, 2011, at 3:25 PM, Tommy Grav wrote:

> I think this discussion is very good. As an astronomer that does  
> most of my
> programming in Python it certainly would be easier to contribute  
> effort to
> a project that have some more unified community support similar to  
> scipy
> and numpy.
>
> One quick question. I have not looked at pyRAF in a while, but it  
> still is only
> a wrapper of the IRAF routines, right? Has anyone considered  
> migrating away
> from the IRAF backend to build pure python or C/Fortran wrapped  
> packages
> of the same routines? To me the installation of of IRAF and then  
> pyRAF on top
> of that has always been a barrier I never have quite made it  
> through. I know
> that one can install the Stsci package, but last time I looked it  
> installs itself
> on the root directory and consists of way to much non-python related  
> stuff.
> I find that hard to deal with as I as a mac-user use Activestate  
> python with
> the framework structure, which to me is very clean and easy to keep  
> organized.
>
> Cheers
>   Tommy
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From tgrav at mac.com  Mon Jun 13 15:35:41 2011
From: tgrav at mac.com (Tommy Grav)
Date: Mon, 13 Jun 2011 15:35:41 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <A7D51446-0B19-43FF-A0B4-48259F7DAFA7@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
	<BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
	<D3C94402-208C-4FEF-B158-0EE12D495E5F@stsci.edu>
	<C6C240BE-2AF3-4D8A-976E-0EB5C5937809@mac.com>
	<A7D51446-0B19-43FF-A0B4-48259F7DAFA7@stsci.edu>
Message-ID: <2433AD3B-5711-49A7-B895-1B7240F1FF52@mac.com>

That is great to hear! One of the things I miss is the ability to use python to do simple profile plots of
stars and asteroids in CCD images (i can't recall the IRAF command at the moment, as it has been a
while since I used it). Maybe there are some existing package that someone can point me too? I guess one
way to contribute to the early effort is to review what exists and suggest what packages to assimilate and
which one needs changes/improvements. Is there a good list of the different astronomy packages out there?

Cheers
   Tommy



On Jun 13, 2011, at 3:29 PM, Perry Greenfield wrote:

> Oh yes. This is increasingly becoming more important. There are bits and pieces, but this is  one area where we really want to start making progress. One package that is often cited as important is the twodspec package, so we are beginning to start working on replacing  the functionality there, and not just for space-based needs.
> 
> Perry
> 
> 
> On Jun 13, 2011, at 3:25 PM, Tommy Grav wrote:
> 
>> I think this discussion is very good. As an astronomer that does most of my
>> programming in Python it certainly would be easier to contribute effort to
>> a project that have some more unified community support similar to scipy
>> and numpy.
>> 
>> One quick question. I have not looked at pyRAF in a while, but it still is only
>> a wrapper of the IRAF routines, right? Has anyone considered migrating away
>> from the IRAF backend to build pure python or C/Fortran wrapped packages
>> of the same routines? To me the installation of of IRAF and then pyRAF on top
>> of that has always been a barrier I never have quite made it through. I know
>> that one can install the Stsci package, but last time I looked it installs itself
>> on the root directory and consists of way to much non-python related stuff.
>> I find that hard to deal with as I as a mac-user use Activestate python with
>> the framework structure, which to me is very clean and easy to keep organized.
>> 
>> Cheers
>>  Tommy
>> 
>> 
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
> 



From jturner at gemini.edu  Mon Jun 13 15:35:50 2011
From: jturner at gemini.edu (James Turner)
Date: Mon, 13 Jun 2011 15:35:50 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <D3C94402-208C-4FEF-B158-0EE12D495E5F@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>
	<4DF64C59.3010501@gemini.edu>	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>	<BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
	<D3C94402-208C-4FEF-B158-0EE12D495E5F@stsci.edu>
Message-ID: <4DF66696.60600@gemini.edu>

Regarding splinter meetings, I'd say "why not?" -- but I was
thinking of something more dedicated, where people are focused just
on this particular problem and actually achieve something (eg.
contributing then and there to each other's project documentation),
rather than a brief meeting where people say enthusiastic things
and then go away :-). I've already done the latter a few times...

I will be at SciPy for the whole week though (largely so there is
time to talk to people) and will be happy to discuss this again (I
wish our distribution were ready so we could start by announcing it).

Kelle, wasn't there a AAS splinter meeting this year? Did anything
come of it? I probably wouldn't make it all the way from Chile just
for a brief session, but I imagine STScI (or this list) would fill
me in on the discussion and the AAS certainly might be a good place
to get wider input.

Cheers,

James.


On 13/06/11 15:15, Perry Greenfield wrote:
> It may be too late. Those attending may have already made travel plans
> that preclude an extra day or two (I think it deserves at least a
> day). And it may be too late for those that weren't planning to go.
>
> Perry
>
> On Jun 13, 2011, at 3:07 PM, Tom Aldcroft wrote:
>
>> What about a splinter meeting at SciPy2011?  I guess the question is
>> how many interested parties will NOT be there this year.
>>
>> - Tom
>>
>> On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz<kellecruz at gmail.com>
>> wrote:
>>> I think a sit-down is desperately needed to resolve these issues,
>>> figure out
>>> the mgmt structure (aka, pecking order), for the BDFL to emerge,
>>> and for
>>> progress to occur.
>>> I'd be happy to participate as a non-python/programming expert and
>>> maybe
>>> provide the voice of the "users".
>>> I propose there be a Splinter Meeting at AAS in Austin. (Splinter
>>> Meeting
>>> deadline is Dec 1.) Or else someone will have to organize at CfA
>>> (Tom A?
>>> Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell,
>>> that seems
>>> to be where most of the movers and shakers in this game are located.
>>> kelle
>>>
>>> On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield<perry at stsci.edu>
>>> wrote:
>>>>
>>>> That's a good idea.
>>>>
>>>> Perry
>>>>
>>>> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>>>>
>>>>> Maybe we should hold an AstroPy conference, where we can discuss
>>>>> co-ordination, get to know each other better and even sit down and
>>>>> work on libraries together (like at SciPy). That might help
>>>>> generate
>>>>> a bit of momentum. Some of us have had meetings before that were
>>>>> full of ideas that didn't go anywhere, but I don't think it has to
>>>>> be that way if active people on the ground are talking to one
>>>>> another
>>>>> rather than having institutions present their plans and try to
>>>>> negotiate at a high level.
>>>>>
>>>>>
>>>>> On 13/06/11 13:25, James Turner wrote:
>>>>>> It seems that several of us would really like to improve
>>>>>> collaboration on Python libraries but have been struggling to pull
>>>>>> it off. I've raised the same issue on this list in past months,
>>>>>> but
>>>>>> my
>>>>>> focus has unavoidably been on other things and since I'm wary of
>>>>>> shouting a lot without contributing much, I haven't really been
>>>>>> able
>>>>>> to keep the discussion alive...
>>>>>>
>>>>>> I tend to agree with Mark and Stefan about the question of
>>>>>> leadership.
>>>>>> Perry&  co. at Space Telescope deserve recognition for getting us
>>>>>> this
>>>>>> far with things like PyFITS and PyRAF. Others have taken the
>>>>>> initiative
>>>>>> with things like astronomical plotting and documentation sprints.
>>>>>> We're
>>>>>> still lacking a bit of coherence though, which (as Mark
>>>>>> suggests) is
>>>>>> likely to involve one or a few dedicated, energetic,
>>>>>> knowledgeable,
>>>>>> hands-on developer(s) who can glue things together. Those people
>>>>>> need
>>>>>> to be employed by someone, though, to ensure stability&
>>>>>> continuity
>>>>>> (fortunately there's already a bit of that going on at STScI, eg.
>>>>>> with
>>>>>> Mike and Matplotlib). Personally, I have the motivation but have
>>>>>> not
>>>>>> had the time/independence (and might not be assertive enough).
>>>>>> Apparently we do have several energetic authors in the community
>>>>>> now
>>>>>> (like Thomas&  Eli), but each with their own project.
>>>>>>
>>>>>> A couple of years ago, a number of us at the observatories
>>>>>> submitted a
>>>>>> white paper to the Decadal Survey, pointing out the need for more
>>>>>> co-ordinated funding so that we can have people who focus on
>>>>>> cross-
>>>>>> institutional platform development&  support. The report from the
>>>>>> committee did give a nod to our concerns and their importance, but
>>>>>> stopped short of making any recommendation, which basically means
>>>>>> "good
>>>>>> luck with that". Meanwhile, at Gemini we have had our own
>>>>>> problems to
>>>>>> deal with, which make it very difficult for me to propose
>>>>>> something
>>>>>> internally beyond working with STScI on the distribution of
>>>>>> dependencies that Perry mentioned. Perhaps someone obtaining a
>>>>>> grant
>>>>>> for this is not out of the question though.
>>>>>>
>>>>>> I would like it if we could get together organically behind
>>>>>> Astrolib,
>>>>>> but sometimes it's difficult to get people away from their
>>>>>> immediate
>>>>>> priorities to focus on that, even within my own institution. If we
>>>>>> could get people dedicated to it, though, it could become
>>>>>> indispensable
>>>>>> enough to attract and co-ordinate more effort. I'm just not sure
>>>>>> how we
>>>>>> get started at this point and my personal options for tackling the
>>>>>> problem seem limited given the overarching funding transition at
>>>>>> Gemini
>>>>>> and the intense focus on projects that are needed to make that
>>>>>> work...
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> James.
>>>>>>
>>>>>>
>>>>>> On 10/06/11 09:48, Perry Greenfield wrote:
>>>>>>>
>>>>>>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>>>>>
>>>>>>>> I just wanted to also add that (in agreement with Marshall)
>>>>>>>> I'm all
>>>>>>>> in favor of many small modules that accomplish a particular task
>>>>>>>> well, rather than packages that aim for a 'do-it-all' approach
>>>>>>>> and
>>>>>>>> fall short. It's always possible to bundle small packages
>>>>>>>> together
>>>>>>>> afterwards, and I don't mean merge development, but instead just
>>>>>>>> bundling the packages for installation (kind of like EPD). I
>>>>>>>> think
>>>>>>>> that is the easiest approach for all of us.
>>>>>>>>
>>>>>>>> Maybe in the long run, a specific set of core packages will
>>>>>>>> emerge
>>>>>>>> as essential and we can then talk about truly merging them
>>>>>>>> into a
>>>>>>>> scipy-like package, but for now, I think the race is still on.
>>>>>>>> And
>>>>>>>> after all, there's nothing to say we *have* to achieve the same
>>>>>>>> setup as in IDL.
>>>>>>>
>>>>>>> It sure seems to me that the time is ripe to start trying to
>>>>>>> coalesce
>>>>>>> some of the ongoing efforts.
>>>>>>>
>>>>>>> Mind you, I don't think it is necessarily good to start with only
>>>>>>> one
>>>>>>> version. Allowing a few different approaches initially has its
>>>>>>> benefits. You get to see more approaches and ideas in play and
>>>>>>> having
>>>>>>> experience with them is very helpful in deciding which one is
>>>>>>> more
>>>>>>> productive. And sometimes there is room for more than one in the
>>>>>>> long
>>>>>>> run. The different approaches may have their own niches. But it
>>>>>>> is
>>>>>>> hard to imagine any long-term need for more than two or three
>>>>>>> different approaches.
>>>>>>>
>>>>>>> Early on there are some pragmatic needs for different approaches.
>>>>>>> For
>>>>>>> example, having a fairly "literal" translation of IDL tools into
>>>>>>> Python has its benefit. It is very useful for those that would
>>>>>>> like to
>>>>>>> migrate IDL code, and given the existing IDL versions, make it
>>>>>>> much
>>>>>>> easier to test their correctness. But I don't see this as a
>>>>>>> substitute
>>>>>>> for a good set of modular tools that have a better object
>>>>>>> design and
>>>>>>> consistent interfaces with other modules. Doing this is more work
>>>>>>> and
>>>>>>> will take more time. So a need for both approaches is likely.
>>>>>>> Some
>>>>>>> could argue the same for replacing IRAF tasks.
>>>>>>>
>>>>>>> All this is much easier said than done of course.
>>>>>>>
>>>>>>> I wish STScI had more resources to devote to this than we've
>>>>>>> actually
>>>>>>> had. We've been planning to do more on this front than we've
>>>>>>> actually
>>>>>>> done. Things come up repeatedly that ruin these plans. But it
>>>>>>> may be
>>>>>>> worth saying where some of our current efforts are going that may
>>>>>>> overlap some of these other efforts.
>>>>>>>
>>>>>>> 1) We've been planning (along with Gemini, and particularly James
>>>>>>> Turner), to try to develop some Sage-like installation package
>>>>>>> that
>>>>>>> would make it easy to install all the basic tools for most
>>>>>>> astronomers. We had hoped to have a beta version out, but one
>>>>>>> of the
>>>>>>> people working on this left at the end of last year, and we've
>>>>>>> not
>>>>>>> been able to replace that person. We are going to continue this
>>>>>>> effort
>>>>>>> with existing staff though. Hopefully in a few months we'll have
>>>>>>> something to try out.
>>>>>>>
>>>>>>> 2) There is a recognition that a more serious effort needs to be
>>>>>>> made
>>>>>>> to replace IRAF functionality. Perhaps one of the benefits of a
>>>>>>> JWST
>>>>>>> delay is that it will allow us to do some of that work more
>>>>>>> explicitly. But we would not do it by replacing IRAF tasks one-
>>>>>>> for-
>>>>>>> one
>>>>>>> but coming up with an entirely different approach which has to
>>>>>>> start
>>>>>>> from the bottom up (the end result could have applications that
>>>>>>> mostly
>>>>>>> emulate IRAF tasks, but also provide much more modular tools).
>>>>>>>
>>>>>>> 3) More specifically, we are currently focussing on how to handle
>>>>>>> WCS
>>>>>>> issues in a more general way than they are handled in FITS. If
>>>>>>> there
>>>>>>> is interest, perhaps we should say more about the intended
>>>>>>> approach.
>>>>>>> This is particularly important for replacing spectrographic
>>>>>>> tools in
>>>>>>> IRAF, and this is where we are starting our effort.
>>>>>>>
>>>>>>> 4) We need a way of saving these WCS models, and FITS is not the
>>>>>>> way.
>>>>>>> We are looking at an alternate data format, not just for WCS
>>>>>>> models,
>>>>>>> but for data in general [gasp!]. Work has begun on this too.
>>>>>>>
>>>>>>> 5) A lot of our recent work has been on pysynphot and ETCs. We
>>>>>>> plan on
>>>>>>> making the computational part of our ETCs a released tool. But
>>>>>>> I'm
>>>>>>> also wondering if we can generalize the pysynphot spectral models
>>>>>>> for
>>>>>>> more general use in spectral tools.
>>>>>>>
>>>>>>> 6) We have been working on a framework for making pipelines
>>>>>>> easier
>>>>>>> to
>>>>>>> build and configure. That won't be ready for at least a few
>>>>>>> months,
>>>>>>> but could well be of general interest and use.
>>>>>>>
>>>>>>> But besides these things, I would like to see if we can't begin
>>>>>>> the
>>>>>>> effort of narrowing some of the underlying libraries being used.
>>>>>>> FITS
>>>>>>> WCS is one obvious area that seems ripe for that.
>>>>>>>
>>>>>>> But the community ought to identify one or two areas that are
>>>>>>> of the
>>>>>>> most interest in consolidating (let's start small). What should
>>>>>>> we
>>>>>>> start with? Focus is important in making any progress in this
>>>>>>> area.
>>>>>>>
>>>>>>> Perry
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> AstroPy mailing list
>>>>>>> AstroPy at scipy.org
>>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>>
>>>>>
>>>>> --
>>>>> James E.H. Turner
>>>>> Gemini Observatory Southern Operations Centre,
>>>>> Casilla 603,          Tel. (+56) 51 205609
>>>>> La Serena, Chile.     Fax. (+56) 51 205650
>>>>> _______________________________________________
>>>>> AstroPy mailing list
>>>>> AstroPy at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>>>
>>>
>>> --
>>> Kelle Cruz, PhD ? http://kellecruz.com/
>>> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


From acasey at mso.anu.edu.au  Mon Jun 13 15:48:28 2011
From: acasey at mso.anu.edu.au (Andy Casey)
Date: Tue, 14 Jun 2011 05:48:28 +1000
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <mailman.1154.1307993130.1086.astropy@scipy.org>
References: <mailman.1154.1307993130.1086.astropy@scipy.org>
Message-ID: <4CC3F8C3-7228-429B-BB8E-8146A97B3761@mso.anu.edu.au>

Hi all,


> One quick question. I have not looked at pyRAF in a while, but it still is only
> a wrapper of the IRAF routines, right? Has anyone considered migrating away
> from the IRAF backend to build pure python or C/Fortran wrapped packages
> of the same routines? To me the installation of of IRAF and then pyRAF on top
> of that has always been a barrier I never have quite made it through. I know
> that one can install the Stsci package, but last time I looked it installs itself
> on the root directory and consists of way to much non-python related stuff.
> I find that hard to deal with as I as a mac-user use Activestate python with
> the framework structure, which to me is very clean and easy to keep organized.

I couldn't agree more. A colleague and I have started this process with one-dimensional and echelle spectra; a module we've named pyspec. It's a python class object which handles spectroscopic data in a native way, allowing you to do all the maths/modifications that you might normally want to do to spectra. 

Although we haven't released it yet because a more definitive suite of tests and utilities are required, we both use it everyday. We plan to introduce this to astrolib once it's documented further and more of the planned utilities are written (fitprofs, fxcor, normalise, etc). Any and all comments are welcome.

https://github.com/wkerzendorf/pyspec

I think a discussion towards a full suite of asto libraries in python is an excellent idea! If there is a splinter meet-up at Scipy/AAS/similar, I think a documented outline covering what happens at the event would be very helpful for those who cannot attend.


Cheers,
Andy



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110614/3fc13077/attachment.html>

From thomas.robitaille at gmail.com  Mon Jun 13 16:41:09 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Mon, 13 Jun 2011 16:41:09 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF64C59.3010501@gemini.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
Message-ID: <E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>

I agree with the idea of a workshop! (but with few talks, and discussing/deciding/planning/coding would be the majority). I think face to face would be much better than teleconference. I also like the idea of a dedicated workshop, not a splinter.

I suggest that those of us interested in a small workshop see what would be possible in our own institutions. I think in the end we should keep this focused on the development (rather than a general python in astronomy conference). Keeping it small will also make it easier to organize and manage.

Cheers,
Tom


On Monday, June 13, 2011 at 1:43 PM, James Turner wrote:
> Maybe we should hold an AstroPy conference, where we can discuss
> co-ordination, get to know each other better and even sit down and
> work on libraries together (like at SciPy). That might help generate
> a bit of momentum. Some of us have had meetings before that were
> full of ideas that didn't go anywhere, but I don't think it has to
> be that way if active people on the ground are talking to one another
> rather than having institutions present their plans and try to
> negotiate at a high level.
> 
> 
> On 13/06/11 13:25, James Turner wrote:
> > It seems that several of us would really like to improve
> > collaboration on Python libraries but have been struggling to pull
> > it off. I've raised the same issue on this list in past months, but my
> > focus has unavoidably been on other things and since I'm wary of
> > shouting a lot without contributing much, I haven't really been able
> > to keep the discussion alive...
> > 
> > I tend to agree with Mark and Stefan about the question of leadership.
> > Perry & co. at Space Telescope deserve recognition for getting us this
> > far with things like PyFITS and PyRAF. Others have taken the initiative
> > with things like astronomical plotting and documentation sprints. We're
> > still lacking a bit of coherence though, which (as Mark suggests) is
> > likely to involve one or a few dedicated, energetic, knowledgeable,
> > hands-on developer(s) who can glue things together. Those people need
> > to be employed by someone, though, to ensure stability & continuity
> > (fortunately there's already a bit of that going on at STScI, eg. with
> > Mike and Matplotlib). Personally, I have the motivation but have not
> > had the time/independence (and might not be assertive enough).
> > Apparently we do have several energetic authors in the community now
> > (like Thomas & Eli), but each with their own project.
> > 
> > A couple of years ago, a number of us at the observatories submitted a
> > white paper to the Decadal Survey, pointing out the need for more
> > co-ordinated funding so that we can have people who focus on cross-
> > institutional platform development & support. The report from the
> > committee did give a nod to our concerns and their importance, but
> > stopped short of making any recommendation, which basically means "good
> > luck with that". Meanwhile, at Gemini we have had our own problems to
> > deal with, which make it very difficult for me to propose something
> > internally beyond working with STScI on the distribution of
> > dependencies that Perry mentioned. Perhaps someone obtaining a grant
> > for this is not out of the question though.
> > 
> > I would like it if we could get together organically behind Astrolib,
> > but sometimes it's difficult to get people away from their immediate
> > priorities to focus on that, even within my own institution. If we
> > could get people dedicated to it, though, it could become indispensable
> > enough to attract and co-ordinate more effort. I'm just not sure how we
> > get started at this point and my personal options for tackling the
> > problem seem limited given the overarching funding transition at Gemini
> > and the intense focus on projects that are needed to make that work...
> > 
> > Cheers,
> > 
> > James.
> > 
> > 
> > On 10/06/11 09:48, Perry Greenfield wrote:
> > > 
> > > On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
> > > 
> > > > I just wanted to also add that (in agreement with Marshall) I'm all
> > > > in favor of many small modules that accomplish a particular task
> > > > well, rather than packages that aim for a 'do-it-all' approach and
> > > > fall short. It's always possible to bundle small packages together
> > > > afterwards, and I don't mean merge development, but instead just
> > > > bundling the packages for installation (kind of like EPD). I think
> > > > that is the easiest approach for all of us.
> > > > 
> > > > Maybe in the long run, a specific set of core packages will emerge
> > > > as essential and we can then talk about truly merging them into a
> > > > scipy-like package, but for now, I think the race is still on. And
> > > > after all, there's nothing to say we *have* to achieve the same
> > > > setup as in IDL.
> > > 
> > > It sure seems to me that the time is ripe to start trying to coalesce
> > > some of the ongoing efforts.
> > > 
> > > Mind you, I don't think it is necessarily good to start with only one
> > > version. Allowing a few different approaches initially has its
> > > benefits. You get to see more approaches and ideas in play and having
> > > experience with them is very helpful in deciding which one is more
> > > productive. And sometimes there is room for more than one in the long
> > > run. The different approaches may have their own niches. But it is
> > > hard to imagine any long-term need for more than two or three
> > > different approaches.
> > > 
> > > Early on there are some pragmatic needs for different approaches. For
> > > example, having a fairly "literal" translation of IDL tools into
> > > Python has its benefit. It is very useful for those that would like to
> > > migrate IDL code, and given the existing IDL versions, make it much
> > > easier to test their correctness. But I don't see this as a substitute
> > > for a good set of modular tools that have a better object design and
> > > consistent interfaces with other modules. Doing this is more work and
> > > will take more time. So a need for both approaches is likely. Some
> > > could argue the same for replacing IRAF tasks.
> > > 
> > > All this is much easier said than done of course.
> > > 
> > > I wish STScI had more resources to devote to this than we've actually
> > > had. We've been planning to do more on this front than we've actually
> > > done. Things come up repeatedly that ruin these plans. But it may be
> > > worth saying where some of our current efforts are going that may
> > > overlap some of these other efforts.
> > > 
> > > 1) We've been planning (along with Gemini, and particularly James
> > > Turner), to try to develop some Sage-like installation package that
> > > would make it easy to install all the basic tools for most
> > > astronomers. We had hoped to have a beta version out, but one of the
> > > people working on this left at the end of last year, and we've not
> > > been able to replace that person. We are going to continue this effort
> > > with existing staff though. Hopefully in a few months we'll have
> > > something to try out.
> > > 
> > > 2) There is a recognition that a more serious effort needs to be made
> > > to replace IRAF functionality. Perhaps one of the benefits of a JWST
> > > delay is that it will allow us to do some of that work more
> > > explicitly. But we would not do it by replacing IRAF tasks one-for-one
> > > but coming up with an entirely different approach which has to start
> > > from the bottom up (the end result could have applications that mostly
> > > emulate IRAF tasks, but also provide much more modular tools).
> > > 
> > > 3) More specifically, we are currently focussing on how to handle WCS
> > > issues in a more general way than they are handled in FITS. If there
> > > is interest, perhaps we should say more about the intended approach.
> > > This is particularly important for replacing spectrographic tools in
> > > IRAF, and this is where we are starting our effort.
> > > 
> > > 4) We need a way of saving these WCS models, and FITS is not the way.
> > > We are looking at an alternate data format, not just for WCS models,
> > > but for data in general [gasp!]. Work has begun on this too.
> > > 
> > > 5) A lot of our recent work has been on pysynphot and ETCs. We plan on
> > > making the computational part of our ETCs a released tool. But I'm
> > > also wondering if we can generalize the pysynphot spectral models for
> > > more general use in spectral tools.
> > > 
> > > 6) We have been working on a framework for making pipelines easier to
> > > build and configure. That won't be ready for at least a few months,
> > > but could well be of general interest and use.
> > > 
> > > But besides these things, I would like to see if we can't begin the
> > > effort of narrowing some of the underlying libraries being used. FITS
> > > WCS is one obvious area that seems ripe for that.
> > > 
> > > But the community ought to identify one or two areas that are of the
> > > most interest in consolidating (let's start small). What should we
> > > start with? Focus is important in making any progress in this area.
> > > 
> > > Perry
> > > 
> > > 
> > > 
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> -- 
> James E.H. Turner
> Gemini Observatory Southern Operations Centre,
> Casilla 603, Tel. (+56) 51 205609
> La Serena, Chile. Fax. (+56) 51 205650
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> http://mail.scipy.org/mailman/listinfo/astropy




From Christoph.Deil at mpi-hd.mpg.de  Mon Jun 13 16:52:42 2011
From: Christoph.Deil at mpi-hd.mpg.de (Christoph Deil)
Date: Mon, 13 Jun 2011 22:52:42 +0200
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <2433AD3B-5711-49A7-B895-1B7240F1FF52@mac.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<F56E9A1D-5954-4698-B699-C1C74C482E10@stsci.edu>
	<BANLkTiknJqJC8jcPs0JX+no2TA8BXmp76w@mail.gmail.com>
	<BANLkTikG35T=o=YsuK9UzsiME9Bx4+_=Yw@mail.gmail.com>
	<D3C94402-208C-4FEF-B158-0EE12D495E5F@stsci.edu>
	<C6C240BE-2AF3-4D8A-976E-0EB5C5937809@mac.com>
	<A7D51446-0B19-43FF-A0B4-48259F7DAFA7@stsci.edu>
	<2433AD3B-5711-49A7-B895-1B7240F1FF52@mac.com>
Message-ID: <5E5A61A9-A76A-4E07-8195-FDB03FAE3835@mpi-hd.mpg.de>


On Jun 13, 2011, at 9:35 PM, Tommy Grav wrote:

> That is great to hear! One of the things I miss is the ability to use python to do simple profile plots of
> stars and asteroids in CCD images (i can't recall the IRAF command at the moment, as it has been a
> while since I used it). Maybe there are some existing package that someone can point me too? I guess one
> way to contribute to the early effort is to review what exists and suggest what packages to assimilate and
> which one needs changes/improvements. Is there a good list of the different astronomy packages out there?
> 
> Cheers
>   Tommy

For a list of python astronomy packages, have a look at
http://www.astropython.org/resources

For profiling images have a look at the functionality in CIAO / Sherpa:
http://cxc.harvard.edu/ciao/
http://cxc.harvard.edu/sherpa/

E.g. here is a tutorial on how to make and fit a radial profile:
http://cxc.harvard.edu/ciao/threads/radial_profile/

Christoph

From jturner at gemini.edu  Mon Jun 13 16:53:19 2011
From: jturner at gemini.edu (James Turner)
Date: Mon, 13 Jun 2011 16:53:19 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
Message-ID: <4DF678BF.3070505@gemini.edu>

> I agree with the idea of a workshop! (but with few talks, and
> discussing/deciding/planning/coding would be the majority). I think
> face to face would be much better than teleconference. I also like
> the idea of a dedicated workshop, not a splinter.

We could do both (a face-to-face workshop with video connections or
whatever to whomever else wants to dial in; I wouldn't want to shut
out students or people in far-flung continents).

I agree that we'd keep the talks brief, just for context (1 day?)
and focus on things like how to contribute, actually doing so and
organization.

> I suggest that those of us interested in a small workshop see what
> would be possible in our own institutions. I think in the end we
> should keep this focused on the development (rather than a general
> python in astronomy conference). Keeping it small will also make it
> easier to organize and manage.

Chile would be nice but is probably a bit too far for people(?). As
someone pointed out, Baltimore and Harvard seem relatively accessible
for people from the US and Europe.

Cheers,

James.


From laidler at stsci.edu  Mon Jun 13 16:51:07 2011
From: laidler at stsci.edu (Victoria G. Laidler)
Date: Mon, 13 Jun 2011 16:51:07 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>
	<4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
Message-ID: <4DF6783B.9090804@stsci.edu>

Thomas Robitaille wrote:
> I agree with the idea of a workshop! (but with few talks, and discussing/deciding/planning/coding would be the majority). 
As far as talks go, I can imagine the following two sessions of 
Lightning Talks (informal, 5 min talks) would be very useful:
- Session 1: How I used community code to do my project: how I found it, 
things that worked well, things that were hard
- Session 2: Why I decided to roll my own instead of using code I knew 
was out there: what I tried, what didn't work, what might have convinced 
me to use the existing code

cheers,
Vicki

> I think face to face would be much better than teleconference. I also like the idea of a dedicated workshop, not a splinter.
>
> I suggest that those of us interested in a small workshop see what would be possible in our own institutions. I think in the end we should keep this focused on the development (rather than a general python in astronomy conference). Keeping it small will also make it easier to organize and manage.
>
> Cheers,
> Tom
>
>
> On Monday, June 13, 2011 at 1:43 PM, James Turner wrote:
>   
>> Maybe we should hold an AstroPy conference, where we can discuss
>> co-ordination, get to know each other better and even sit down and
>> work on libraries together (like at SciPy). That might help generate
>> a bit of momentum. Some of us have had meetings before that were
>> full of ideas that didn't go anywhere, but I don't think it has to
>> be that way if active people on the ground are talking to one another
>> rather than having institutions present their plans and try to
>> negotiate at a high level.
>>
>>
>> On 13/06/11 13:25, James Turner wrote:
>>     
>>> It seems that several of us would really like to improve
>>> collaboration on Python libraries but have been struggling to pull
>>> it off. I've raised the same issue on this list in past months, but my
>>> focus has unavoidably been on other things and since I'm wary of
>>> shouting a lot without contributing much, I haven't really been able
>>> to keep the discussion alive...
>>>
>>> I tend to agree with Mark and Stefan about the question of leadership.
>>> Perry & co. at Space Telescope deserve recognition for getting us this
>>> far with things like PyFITS and PyRAF. Others have taken the initiative
>>> with things like astronomical plotting and documentation sprints. We're
>>> still lacking a bit of coherence though, which (as Mark suggests) is
>>> likely to involve one or a few dedicated, energetic, knowledgeable,
>>> hands-on developer(s) who can glue things together. Those people need
>>> to be employed by someone, though, to ensure stability & continuity
>>> (fortunately there's already a bit of that going on at STScI, eg. with
>>> Mike and Matplotlib). Personally, I have the motivation but have not
>>> had the time/independence (and might not be assertive enough).
>>> Apparently we do have several energetic authors in the community now
>>> (like Thomas & Eli), but each with their own project.
>>>
>>> A couple of years ago, a number of us at the observatories submitted a
>>> white paper to the Decadal Survey, pointing out the need for more
>>> co-ordinated funding so that we can have people who focus on cross-
>>> institutional platform development & support. The report from the
>>> committee did give a nod to our concerns and their importance, but
>>> stopped short of making any recommendation, which basically means "good
>>> luck with that". Meanwhile, at Gemini we have had our own problems to
>>> deal with, which make it very difficult for me to propose something
>>> internally beyond working with STScI on the distribution of
>>> dependencies that Perry mentioned. Perhaps someone obtaining a grant
>>> for this is not out of the question though.
>>>
>>> I would like it if we could get together organically behind Astrolib,
>>> but sometimes it's difficult to get people away from their immediate
>>> priorities to focus on that, even within my own institution. If we
>>> could get people dedicated to it, though, it could become indispensable
>>> enough to attract and co-ordinate more effort. I'm just not sure how we
>>> get started at this point and my personal options for tackling the
>>> problem seem limited given the overarching funding transition at Gemini
>>> and the intense focus on projects that are needed to make that work...
>>>
>>> Cheers,
>>>
>>> James.
>>>
>>>
>>> On 10/06/11 09:48, Perry Greenfield wrote:
>>>       
>>>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>>
>>>>         
>>>>> I just wanted to also add that (in agreement with Marshall) I'm all
>>>>> in favor of many small modules that accomplish a particular task
>>>>> well, rather than packages that aim for a 'do-it-all' approach and
>>>>> fall short. It's always possible to bundle small packages together
>>>>> afterwards, and I don't mean merge development, but instead just
>>>>> bundling the packages for installation (kind of like EPD). I think
>>>>> that is the easiest approach for all of us.
>>>>>
>>>>> Maybe in the long run, a specific set of core packages will emerge
>>>>> as essential and we can then talk about truly merging them into a
>>>>> scipy-like package, but for now, I think the race is still on. And
>>>>> after all, there's nothing to say we *have* to achieve the same
>>>>> setup as in IDL.
>>>>>           
>>>> It sure seems to me that the time is ripe to start trying to coalesce
>>>> some of the ongoing efforts.
>>>>
>>>> Mind you, I don't think it is necessarily good to start with only one
>>>> version. Allowing a few different approaches initially has its
>>>> benefits. You get to see more approaches and ideas in play and having
>>>> experience with them is very helpful in deciding which one is more
>>>> productive. And sometimes there is room for more than one in the long
>>>> run. The different approaches may have their own niches. But it is
>>>> hard to imagine any long-term need for more than two or three
>>>> different approaches.
>>>>
>>>> Early on there are some pragmatic needs for different approaches. For
>>>> example, having a fairly "literal" translation of IDL tools into
>>>> Python has its benefit. It is very useful for those that would like to
>>>> migrate IDL code, and given the existing IDL versions, make it much
>>>> easier to test their correctness. But I don't see this as a substitute
>>>> for a good set of modular tools that have a better object design and
>>>> consistent interfaces with other modules. Doing this is more work and
>>>> will take more time. So a need for both approaches is likely. Some
>>>> could argue the same for replacing IRAF tasks.
>>>>
>>>> All this is much easier said than done of course.
>>>>
>>>> I wish STScI had more resources to devote to this than we've actually
>>>> had. We've been planning to do more on this front than we've actually
>>>> done. Things come up repeatedly that ruin these plans. But it may be
>>>> worth saying where some of our current efforts are going that may
>>>> overlap some of these other efforts.
>>>>
>>>> 1) We've been planning (along with Gemini, and particularly James
>>>> Turner), to try to develop some Sage-like installation package that
>>>> would make it easy to install all the basic tools for most
>>>> astronomers. We had hoped to have a beta version out, but one of the
>>>> people working on this left at the end of last year, and we've not
>>>> been able to replace that person. We are going to continue this effort
>>>> with existing staff though. Hopefully in a few months we'll have
>>>> something to try out.
>>>>
>>>> 2) There is a recognition that a more serious effort needs to be made
>>>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
>>>> delay is that it will allow us to do some of that work more
>>>> explicitly. But we would not do it by replacing IRAF tasks one-for-one
>>>> but coming up with an entirely different approach which has to start
>>>> from the bottom up (the end result could have applications that mostly
>>>> emulate IRAF tasks, but also provide much more modular tools).
>>>>
>>>> 3) More specifically, we are currently focussing on how to handle WCS
>>>> issues in a more general way than they are handled in FITS. If there
>>>> is interest, perhaps we should say more about the intended approach.
>>>> This is particularly important for replacing spectrographic tools in
>>>> IRAF, and this is where we are starting our effort.
>>>>
>>>> 4) We need a way of saving these WCS models, and FITS is not the way.
>>>> We are looking at an alternate data format, not just for WCS models,
>>>> but for data in general [gasp!]. Work has begun on this too.
>>>>
>>>> 5) A lot of our recent work has been on pysynphot and ETCs. We plan on
>>>> making the computational part of our ETCs a released tool. But I'm
>>>> also wondering if we can generalize the pysynphot spectral models for
>>>> more general use in spectral tools.
>>>>
>>>> 6) We have been working on a framework for making pipelines easier to
>>>> build and configure. That won't be ready for at least a few months,
>>>> but could well be of general interest and use.
>>>>
>>>> But besides these things, I would like to see if we can't begin the
>>>> effort of narrowing some of the underlying libraries being used. FITS
>>>> WCS is one obvious area that seems ripe for that.
>>>>
>>>> But the community ought to identify one or two areas that are of the
>>>> most interest in consolidating (let's start small). What should we
>>>> start with? Focus is important in making any progress in this area.
>>>>
>>>> Perry
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>         
>> -- 
>> James E.H. Turner
>> Gemini Observatory Southern Operations Centre,
>> Casilla 603, Tel. (+56) 51 205609
>> La Serena, Chile. Fax. (+56) 51 205650
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>> http://mail.scipy.org/mailman/listinfo/astropy
>>     
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>   



From thomas.robitaille at gmail.com  Mon Jun 13 17:14:48 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Mon, 13 Jun 2011 17:14:48 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF678BF.3070505@gemini.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
Message-ID: <2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>

Just to put the suggestion out there, Tom A. and I would be happy to try and host the meeting at CfA if there was interest for this, preferably in the fall. We have webcast capabilities for the talks (I like the idea of 'lightning' talks), though it would be harder to have remote participation for the hands-on part.

Of course, we're also open to other locations, so if anyone else interested in hosting this, please feel free to suggest!

Cheers,
Tom

On Monday, June 13, 2011 at 4:53 PM, James Turner wrote:

> > I agree with the idea of a workshop! (but with few talks, and
> > discussing/deciding/planning/coding would be the majority). I think
> > face to face would be much better than teleconference. I also like
> > the idea of a dedicated workshop, not a splinter.
> 
> We could do both (a face-to-face workshop with video connections or
> whatever to whomever else wants to dial in; I wouldn't want to shut
> out students or people in far-flung continents).
> 
> I agree that we'd keep the talks brief, just for context (1 day?)
> and focus on things like how to contribute, actually doing so and
> organization.
> 
> > I suggest that those of us interested in a small workshop see what
> > would be possible in our own institutions. I think in the end we
> > should keep this focused on the development (rather than a general
> > python in astronomy conference). Keeping it small will also make it
> > easier to organize and manage.
> 
> Chile would be nice but is probably a bit too far for people(?). As
> someone pointed out, Baltimore and Harvard seem relatively accessible
> for people from the US and Europe.
> 
> Cheers,
> 
> James.




From perry at stsci.edu  Mon Jun 13 17:25:32 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Mon, 13 Jun 2011 17:25:32 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
Message-ID: <3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>

I think we could probably host as well (I have to check the  
logistics). Fall is probably good for us as well but we could probably  
do it earlier if there was interest in doing it sooner than fall.

Perry

On Jun 13, 2011, at 5:14 PM, Thomas Robitaille wrote:

> Just to put the suggestion out there, Tom A. and I would be happy to  
> try and host the meeting at CfA if there was interest for this,  
> preferably in the fall. We have webcast capabilities for the talks  
> (I like the idea of 'lightning' talks), though it would be harder to  
> have remote participation for the hands-on part.
>
> Of course, we're also open to other locations, so if anyone else  
> interested in hosting this, please feel free to suggest!
>
> Cheers,
> Tom
>
> On Monday, June 13, 2011 at 4:53 PM, James Turner wrote:
>
>>> I agree with the idea of a workshop! (but with few talks, and
>>> discussing/deciding/planning/coding would be the majority). I think
>>> face to face would be much better than teleconference. I also like
>>> the idea of a dedicated workshop, not a splinter.
>>
>> We could do both (a face-to-face workshop with video connections or
>> whatever to whomever else wants to dial in; I wouldn't want to shut
>> out students or people in far-flung continents).
>>
>> I agree that we'd keep the talks brief, just for context (1 day?)
>> and focus on things like how to contribute, actually doing so and
>> organization.
>>
>>> I suggest that those of us interested in a small workshop see what
>>> would be possible in our own institutions. I think in the end we
>>> should keep this focused on the development (rather than a general
>>> python in astronomy conference). Keeping it small will also make it
>>> easier to organize and manage.
>>
>> Chile would be nice but is probably a bit too far for people(?). As
>> someone pointed out, Baltimore and Harvard seem relatively accessible
>> for people from the US and Europe.
>>
>> Cheers,
>>
>> James.
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From ericography at gmail.com  Mon Jun 13 17:44:50 2011
From: ericography at gmail.com (Eric G. Barron)
Date: Mon, 13 Jun 2011 15:44:50 -0600
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
Message-ID: <BANLkTikPYT7zZvB9vVwz3pjkgZyXgtU84w@mail.gmail.com>

I don't think it would be much harder to have remote participation.  Plenty
of software projects have online-only or joint face-to-face and online
developer meetings and sprints using tools like IRC, wikis, ticket
trackers...  It would certainly help people like me, who would like to
participate but who have no funding to physically travel to a meeting.

Eric


On Mon, Jun 13, 2011 at 15:14, Thomas Robitaille <
thomas.robitaille at gmail.com> wrote:

> Just to put the suggestion out there, Tom A. and I would be happy to try
> and host the meeting at CfA if there was interest for this, preferably in
> the fall. We have webcast capabilities for the talks (I like the idea of
> 'lightning' talks), though it would be harder to have remote participation
> for the hands-on part.
>
> Of course, we're also open to other locations, so if anyone else interested
> in hosting this, please feel free to suggest!
>
> Cheers,
> Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110613/45d0ffdf/attachment.html>

From thomas.robitaille at gmail.com  Mon Jun 13 19:26:22 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Mon, 13 Jun 2011 19:26:22 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTikPYT7zZvB9vVwz3pjkgZyXgtU84w@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
	<BANLkTikPYT7zZvB9vVwz3pjkgZyXgtU84w@mail.gmail.com>
Message-ID: <2D017079118B4CF094C59CCADADCDFFD@gmail.com>

I'd like to suggest that we start figuring out who is interested in contributing to a common astronomy package, and what you are most interested in. To get things moving, I created a simple wiki page here:

http://astropy.wikispaces.com/developers

Please add your name to register interest, and what areas you are interested in (feel free to add areas). This will make it easier to judge interest for face-to-face meetings and for coordinating working groups to focus on various areas. I picked the wiki host at random, but of course we can move anytime if you have a better solution (for example if we use GitHub for development, we can use the project wiki on there).

Note, should we create a new mailing list, say astropy-dev for people interested in actual development, as opposed to people who just want to use stuff and have questions?

Cheers,
Tom


On Monday, June 13, 2011 at 5:44 PM, Eric G. Barron wrote:

> I don't think it would be much harder to have remote participation. Plenty of software projects have online-only or joint face-to-face and online developer meetings and sprints using tools like IRC, wikis, ticket trackers... It would certainly help people like me, who would like to participate but who have no funding to physically travel to a meeting.
> 
> Eric
> 
> 
> On Mon, Jun 13, 2011 at 15:14, Thomas Robitaille <thomas.robitaille at gmail.com (mailto:thomas.robitaille at gmail.com)> wrote:
> > Just to put the suggestion out there, Tom A. and I would be happy to try and host the meeting at CfA if there was interest for this, preferably in the fall. We have webcast capabilities for the talks (I like the idea of 'lightning' talks), though it would be harder to have remote participation for the hands-on part.
> > 
> >  Of course, we're also open to other locations, so if anyone else interested in hosting this, please feel free to suggest!
> > 
> >  Cheers,
> >  Tom
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> http://mail.scipy.org/mailman/listinfo/astropy




From ebressert at cfa.harvard.edu  Mon Jun 13 20:20:52 2011
From: ebressert at cfa.harvard.edu (Eli Bressert)
Date: Mon, 13 Jun 2011 18:20:52 -0600
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <mailman.1152.1307992991.1086.astropy@scipy.org>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
Message-ID: <5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>

Organizing a splinter meeting at the winter AAS would be a great idea (as Kelle suggested) to get the AstroPy movement going. I'll be at the meeting this year and would be happy to help out on this front.

It seems to me that the strength of Python and modules is that everyone has access to the language (unlike IDL) and multiple modules are developed all the time. If we look around the astronomy community there is already a large number of Python modules for astronomy (e.g. PyFITS, astropysics, asciitables, and many more - see http://www.astropython.org/resources). One of the main difficulties that we are having is integration between the modules. 

Most of the astronomy modules for Python use Numpy and Scipy (frequently Matplotlib too), but things diverge fairly quickly from that point on. Should we agree on a list of the core astronomy modules? Is this possible and is it a good idea? All of the things I have mentioned has shown up on this mailing list before, but it would be useful to move this forward. 

One thought that came up (this is *only* a suggestion) is should we have a basic python package called astropy (when we have agreed on the basic packages for it)? If we do this we could use a similar scheme to Scipy's Scikits (call it astrokits?). This would ensure that the modules would use core astronomy modules. Again, this is just a exploratory idea and I would be happy to hear what others have to say. 

Cheers,
Eli





On Monday, June 13, 2011 at 1:23 PM, astropy-request at scipy.org wrote:

> Send AstroPy mailing list submissions to
> astropy at scipy.org (mailto:astropy at scipy.org)
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mail.scipy.org/mailman/listinfo/astropy
> or, via email, send a message with subject or body 'help' to
> astropy-request at scipy.org (mailto:astropy-request at scipy.org)
> 
> You can reach the person managing the list at
> astropy-owner at scipy.org (mailto:astropy-owner at scipy.org)
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of AstroPy digest..."
> 
> 
> Today's Topics:
> 
>  1. Re: Proliferating py-astro-libs (Matthew Turk)
>  2. Re: Proliferating py-astro-libs (Perry Greenfield)
>  3. Re: Proliferating py-astro-libs (Kelle Cruz)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Mon, 13 Jun 2011 12:11:13 -0700
> From: Matthew Turk <matthewturk at gmail.com (mailto:matthewturk at gmail.com)>
> Subject: Re: [AstroPy] Proliferating py-astro-libs
> To: Tom Aldcroft <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>
> Cc: astropy <astropy at scipy.org (mailto:astropy at scipy.org)>
> Message-ID: <BANLkTinvOyspdCdzB_2+=tjv1XNcPSkSUA at mail.gmail.com (mailto:tjv1XNcPSkSUA at mail.gmail.com)>
> Content-Type: text/plain; charset=windows-1252
> 
> Hi all,
> 
> I'd be very interested in participating in a discussion of these
> issues. I am not an observer, but I am a member of the astrophysical
> simulations community concerned with issues of interop, open science,
> and developing community-focused packages for scientific analysis, and
> I believe that increasing cross-talk between the observation
> community and the simulations community would be of great benefit to
> all concerned.
> 
> I won't be at SciPy (haven't been able to make it since 2009,
> unfortunately!) but I think this kind of discussion might warrant its
> own mini-workshop.
> 
> -Matt
> 
> On Mon, Jun 13, 2011 at 12:07 PM, Tom Aldcroft
> <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)> wrote:
> > What about a splinter meeting at SciPy2011? ?I guess the question is
> > how many interested parties will NOT be there this year.
> > 
> > - Tom
> > 
> > On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com (mailto:kellecruz at gmail.com)> wrote:
> > > I think a sit-down is desperately needed to resolve these issues, figure out
> > > the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
> > > progress to occur.
> > > I'd be happy to participate as a non-python/programming expert and maybe
> > > provide the voice of the "users".
> > > I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
> > > deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
> > > Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that seems
> > > to be where most of the movers and shakers in this game are located.
> > > kelle
> > > 
> > > On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu (mailto:perry at stsci.edu)> wrote:
> > > > 
> > > > That's a good idea.
> > > > 
> > > > Perry
> > > > 
> > > > On Jun 13, 2011, at 1:43 PM, James Turner wrote:
> > > > 
> > > > > Maybe we should hold an AstroPy conference, where we can discuss
> > > > > co-ordination, get to know each other better and even sit down and
> > > > > work on libraries together (like at SciPy). That might help generate
> > > > > a bit of momentum. Some of us have had meetings before that were
> > > > > full of ideas that didn't go anywhere, but I don't think it has to
> > > > > be that way if active people on the ground are talking to one another
> > > > > rather than having institutions present their plans and try to
> > > > > negotiate at a high level.
> > > > > 
> > > > > 
> > > > > On 13/06/11 13:25, James Turner wrote:
> > > > > > It seems that several of us would really like to improve
> > > > > > collaboration on Python libraries but have been struggling to pull
> > > > > > it off. I've raised the same issue on this list in past months, but
> > > > > > my
> > > > > > focus has unavoidably been on other things and since I'm wary of
> > > > > > shouting a lot without contributing much, I haven't really been able
> > > > > > to keep the discussion alive...
> > > > > > 
> > > > > > I tend to agree with Mark and Stefan about the question of
> > > > > > leadership.
> > > > > > Perry & co. at Space Telescope deserve recognition for getting us
> > > > > > this
> > > > > > far with things like PyFITS and PyRAF. Others have taken the
> > > > > > initiative
> > > > > > with things like astronomical plotting and documentation sprints.
> > > > > > We're
> > > > > > still lacking a bit of coherence though, which (as Mark suggests) is
> > > > > > likely to involve one or a few dedicated, energetic, knowledgeable,
> > > > > > hands-on developer(s) who can glue things together. Those people need
> > > > > > to be employed by someone, though, to ensure stability & continuity
> > > > > > (fortunately there's already a bit of that going on at STScI, eg.
> > > > > > with
> > > > > > Mike and Matplotlib). Personally, I have the motivation but have not
> > > > > > had the time/independence (and might not be assertive enough).
> > > > > > Apparently we do have several energetic authors in the community now
> > > > > > (like Thomas & Eli), but each with their own project.
> > > > > > 
> > > > > > A couple of years ago, a number of us at the observatories
> > > > > > submitted a
> > > > > > white paper to the Decadal Survey, pointing out the need for more
> > > > > > co-ordinated funding so that we can have people who focus on cross-
> > > > > > institutional platform development & support. The report from the
> > > > > > committee did give a nod to our concerns and their importance, but
> > > > > > stopped short of making any recommendation, which basically means
> > > > > > "good
> > > > > > luck with that". Meanwhile, at Gemini we have had our own problems to
> > > > > > deal with, which make it very difficult for me to propose something
> > > > > > internally beyond working with STScI on the distribution of
> > > > > > dependencies that Perry mentioned. Perhaps someone obtaining a grant
> > > > > > for this is not out of the question though.
> > > > > > 
> > > > > > I would like it if we could get together organically behind Astrolib,
> > > > > > but sometimes it's difficult to get people away from their immediate
> > > > > > priorities to focus on that, even within my own institution. If we
> > > > > > could get people dedicated to it, though, it could become
> > > > > > indispensable
> > > > > > enough to attract and co-ordinate more effort. I'm just not sure
> > > > > > how we
> > > > > > get started at this point and my personal options for tackling the
> > > > > > problem seem limited given the overarching funding transition at
> > > > > > Gemini
> > > > > > and the intense focus on projects that are needed to make that
> > > > > > work...
> > > > > > 
> > > > > > Cheers,
> > > > > > 
> > > > > > James.
> > > > > > 
> > > > > > 
> > > > > > On 10/06/11 09:48, Perry Greenfield wrote:
> > > > > > > 
> > > > > > > On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
> > > > > > > 
> > > > > > > > I just wanted to also add that (in agreement with Marshall) I'm all
> > > > > > > > in favor of many small modules that accomplish a particular task
> > > > > > > > well, rather than packages that aim for a 'do-it-all' approach and
> > > > > > > > fall short. It's always possible to bundle small packages together
> > > > > > > > afterwards, and I don't mean merge development, but instead just
> > > > > > > > bundling the packages for installation (kind of like EPD). I think
> > > > > > > > that is the easiest approach for all of us.
> > > > > > > > 
> > > > > > > > Maybe in the long run, a specific set of core packages will emerge
> > > > > > > > as essential and we can then talk about truly merging them into a
> > > > > > > > scipy-like package, but for now, I think the race is still on. And
> > > > > > > > after all, there's nothing to say we *have* to achieve the same
> > > > > > > > setup as in IDL.
> > > > > > > 
> > > > > > > It sure seems to me that the time is ripe to start trying to
> > > > > > > coalesce
> > > > > > > some of the ongoing efforts.
> > > > > > > 
> > > > > > > Mind you, I don't think it is necessarily good to start with only
> > > > > > > one
> > > > > > > version. Allowing a few different approaches initially has its
> > > > > > > benefits. You get to see more approaches and ideas in play and
> > > > > > > having
> > > > > > > experience with them is very helpful in deciding which one is more
> > > > > > > productive. And sometimes there is room for more than one in the
> > > > > > > long
> > > > > > > run. The different approaches may have their own niches. But it is
> > > > > > > hard to imagine any long-term need for more than two or three
> > > > > > > different approaches.
> > > > > > > 
> > > > > > > Early on there are some pragmatic needs for different approaches.
> > > > > > > For
> > > > > > > example, having a fairly "literal" translation of IDL tools into
> > > > > > > Python has its benefit. It is very useful for those that would
> > > > > > > like to
> > > > > > > migrate IDL code, and given the existing IDL versions, make it much
> > > > > > > easier to test their correctness. But I don't see this as a
> > > > > > > substitute
> > > > > > > for a good set of modular tools that have a better object design and
> > > > > > > consistent interfaces with other modules. Doing this is more work
> > > > > > > and
> > > > > > > will take more time. So a need for both approaches is likely. Some
> > > > > > > could argue the same for replacing IRAF tasks.
> > > > > > > 
> > > > > > > All this is much easier said than done of course.
> > > > > > > 
> > > > > > > I wish STScI had more resources to devote to this than we've
> > > > > > > actually
> > > > > > > had. We've been planning to do more on this front than we've
> > > > > > > actually
> > > > > > > done. Things come up repeatedly that ruin these plans. But it may be
> > > > > > > worth saying where some of our current efforts are going that may
> > > > > > > overlap some of these other efforts.
> > > > > > > 
> > > > > > > 1) We've been planning (along with Gemini, and particularly James
> > > > > > > Turner), to try to develop some Sage-like installation package that
> > > > > > > would make it easy to install all the basic tools for most
> > > > > > > astronomers. We had hoped to have a beta version out, but one of the
> > > > > > > people working on this left at the end of last year, and we've not
> > > > > > > been able to replace that person. We are going to continue this
> > > > > > > effort
> > > > > > > with existing staff though. Hopefully in a few months we'll have
> > > > > > > something to try out.
> > > > > > > 
> > > > > > > 2) There is a recognition that a more serious effort needs to be
> > > > > > > made
> > > > > > > to replace IRAF functionality. Perhaps one of the benefits of a JWST
> > > > > > > delay is that it will allow us to do some of that work more
> > > > > > > explicitly. But we would not do it by replacing IRAF tasks one-for-
> > > > > > > one
> > > > > > > but coming up with an entirely different approach which has to start
> > > > > > > from the bottom up (the end result could have applications that
> > > > > > > mostly
> > > > > > > emulate IRAF tasks, but also provide much more modular tools).
> > > > > > > 
> > > > > > > 3) More specifically, we are currently focussing on how to handle
> > > > > > > WCS
> > > > > > > issues in a more general way than they are handled in FITS. If there
> > > > > > > is interest, perhaps we should say more about the intended approach.
> > > > > > > This is particularly important for replacing spectrographic tools in
> > > > > > > IRAF, and this is where we are starting our effort.
> > > > > > > 
> > > > > > > 4) We need a way of saving these WCS models, and FITS is not the
> > > > > > > way.
> > > > > > > We are looking at an alternate data format, not just for WCS models,
> > > > > > > but for data in general [gasp!]. Work has begun on this too.
> > > > > > > 
> > > > > > > 5) A lot of our recent work has been on pysynphot and ETCs. We
> > > > > > > plan on
> > > > > > > making the computational part of our ETCs a released tool. But I'm
> > > > > > > also wondering if we can generalize the pysynphot spectral models
> > > > > > > for
> > > > > > > more general use in spectral tools.
> > > > > > > 
> > > > > > > 6) We have been working on a framework for making pipelines easier
> > > > > > > to
> > > > > > > build and configure. That won't be ready for at least a few months,
> > > > > > > but could well be of general interest and use.
> > > > > > > 
> > > > > > > But besides these things, I would like to see if we can't begin the
> > > > > > > effort of narrowing some of the underlying libraries being used.
> > > > > > > FITS
> > > > > > > WCS is one obvious area that seems ripe for that.
> > > > > > > 
> > > > > > > But the community ought to identify one or two areas that are of the
> > > > > > > most interest in consolidating (let's start small). What should we
> > > > > > > start with? Focus is important in making any progress in this area.
> > > > > > > 
> > > > > > > Perry
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > _______________________________________________
> > > > > > > AstroPy mailing list
> > > > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > > 
> > > > > 
> > > > > --
> > > > > James E.H. Turner
> > > > > Gemini Observatory Southern Operations Centre,
> > > > > Casilla 603, ? ? ? ? ?Tel. (+56) 51 205609
> > > > > La Serena, Chile. ? ? Fax. (+56) 51 205650
> > > > > _______________________________________________
> > > > > AstroPy mailing list
> > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > 
> > > > _______________________________________________
> > > > AstroPy mailing list
> > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > 
> > > 
> > > 
> > > --
> > > Kelle Cruz, PhD ??http://kellecruz.com/
> > > 917.725.1334 ??Hunter ext: 16486 ? AMNH ext: 3404
> > > 
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > http://mail.scipy.org/mailman/listinfo/astropy
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Mon, 13 Jun 2011 15:15:28 -0400
> From: Perry Greenfield <perry at stsci.edu (mailto:perry at stsci.edu)>
> Subject: Re: [AstroPy] Proliferating py-astro-libs
> To: Tom Aldcroft <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>
> Cc: astropy <astropy at scipy.org (mailto:astropy at scipy.org)>
> Message-ID: <D3C94402-208C-4FEF-B158-0EE12D495E5F at stsci.edu (mailto:D3C94402-208C-4FEF-B158-0EE12D495E5F at stsci.edu)>
> Content-Type: text/plain; charset=WINDOWS-1252; format=flowed;
>  delsp=yes
> 
> It may be too late. Those attending may have already made travel plans 
> that preclude an extra day or two (I think it deserves at least a 
> day). And it may be too late for those that weren't planning to go.
> 
> Perry
> 
> On Jun 13, 2011, at 3:07 PM, Tom Aldcroft wrote:
> 
> > What about a splinter meeting at SciPy2011? I guess the question is
> > how many interested parties will NOT be there this year.
> > 
> > - Tom
> > 
> > On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com (mailto:kellecruz at gmail.com)> 
> > wrote:
> > > I think a sit-down is desperately needed to resolve these issues, 
> > > figure out
> > > the mgmt structure (aka, pecking order), for the BDFL to emerge, 
> > > and for
> > > progress to occur.
> > > I'd be happy to participate as a non-python/programming expert and 
> > > maybe
> > > provide the voice of the "users".
> > > I propose there be a Splinter Meeting at AAS in Austin. (Splinter 
> > > Meeting
> > > deadline is Dec 1.) Or else someone will have to organize at CfA 
> > > (Tom A?
> > > Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, 
> > > that seems
> > > to be where most of the movers and shakers in this game are located.
> > > kelle
> > > 
> > > On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu (mailto:perry at stsci.edu)> 
> > > wrote:
> > > > 
> > > > That's a good idea.
> > > > 
> > > > Perry
> > > > 
> > > > On Jun 13, 2011, at 1:43 PM, James Turner wrote:
> > > > 
> > > > > Maybe we should hold an AstroPy conference, where we can discuss
> > > > > co-ordination, get to know each other better and even sit down and
> > > > > work on libraries together (like at SciPy). That might help 
> > > > > generate
> > > > > a bit of momentum. Some of us have had meetings before that were
> > > > > full of ideas that didn't go anywhere, but I don't think it has to
> > > > > be that way if active people on the ground are talking to one 
> > > > > another
> > > > > rather than having institutions present their plans and try to
> > > > > negotiate at a high level.
> > > > > 
> > > > > 
> > > > > On 13/06/11 13:25, James Turner wrote:
> > > > > > It seems that several of us would really like to improve
> > > > > > collaboration on Python libraries but have been struggling to pull
> > > > > > it off. I've raised the same issue on this list in past months, 
> > > > > > but
> > > > > > my
> > > > > > focus has unavoidably been on other things and since I'm wary of
> > > > > > shouting a lot without contributing much, I haven't really been 
> > > > > > able
> > > > > > to keep the discussion alive...
> > > > > > 
> > > > > > I tend to agree with Mark and Stefan about the question of
> > > > > > leadership.
> > > > > > Perry & co. at Space Telescope deserve recognition for getting us
> > > > > > this
> > > > > > far with things like PyFITS and PyRAF. Others have taken the
> > > > > > initiative
> > > > > > with things like astronomical plotting and documentation sprints.
> > > > > > We're
> > > > > > still lacking a bit of coherence though, which (as Mark 
> > > > > > suggests) is
> > > > > > likely to involve one or a few dedicated, energetic, 
> > > > > > knowledgeable,
> > > > > > hands-on developer(s) who can glue things together. Those people 
> > > > > > need
> > > > > > to be employed by someone, though, to ensure stability & 
> > > > > > continuity
> > > > > > (fortunately there's already a bit of that going on at STScI, eg.
> > > > > > with
> > > > > > Mike and Matplotlib). Personally, I have the motivation but have 
> > > > > > not
> > > > > > had the time/independence (and might not be assertive enough).
> > > > > > Apparently we do have several energetic authors in the community 
> > > > > > now
> > > > > > (like Thomas & Eli), but each with their own project.
> > > > > > 
> > > > > > A couple of years ago, a number of us at the observatories
> > > > > > submitted a
> > > > > > white paper to the Decadal Survey, pointing out the need for more
> > > > > > co-ordinated funding so that we can have people who focus on 
> > > > > > cross-
> > > > > > institutional platform development & support. The report from the
> > > > > > committee did give a nod to our concerns and their importance, but
> > > > > > stopped short of making any recommendation, which basically means
> > > > > > "good
> > > > > > luck with that". Meanwhile, at Gemini we have had our own 
> > > > > > problems to
> > > > > > deal with, which make it very difficult for me to propose 
> > > > > > something
> > > > > > internally beyond working with STScI on the distribution of
> > > > > > dependencies that Perry mentioned. Perhaps someone obtaining a 
> > > > > > grant
> > > > > > for this is not out of the question though.
> > > > > > 
> > > > > > I would like it if we could get together organically behind 
> > > > > > Astrolib,
> > > > > > but sometimes it's difficult to get people away from their 
> > > > > > immediate
> > > > > > priorities to focus on that, even within my own institution. If we
> > > > > > could get people dedicated to it, though, it could become
> > > > > > indispensable
> > > > > > enough to attract and co-ordinate more effort. I'm just not sure
> > > > > > how we
> > > > > > get started at this point and my personal options for tackling the
> > > > > > problem seem limited given the overarching funding transition at
> > > > > > Gemini
> > > > > > and the intense focus on projects that are needed to make that
> > > > > > work...
> > > > > > 
> > > > > > Cheers,
> > > > > > 
> > > > > > James.
> > > > > > 
> > > > > > 
> > > > > > On 10/06/11 09:48, Perry Greenfield wrote:
> > > > > > > 
> > > > > > > On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
> > > > > > > 
> > > > > > > > I just wanted to also add that (in agreement with Marshall) 
> > > > > > > > I'm all
> > > > > > > > in favor of many small modules that accomplish a particular task
> > > > > > > > well, rather than packages that aim for a 'do-it-all' approach 
> > > > > > > > and
> > > > > > > > fall short. It's always possible to bundle small packages 
> > > > > > > > together
> > > > > > > > afterwards, and I don't mean merge development, but instead just
> > > > > > > > bundling the packages for installation (kind of like EPD). I 
> > > > > > > > think
> > > > > > > > that is the easiest approach for all of us.
> > > > > > > > 
> > > > > > > > Maybe in the long run, a specific set of core packages will 
> > > > > > > > emerge
> > > > > > > > as essential and we can then talk about truly merging them 
> > > > > > > > into a
> > > > > > > > scipy-like package, but for now, I think the race is still on. 
> > > > > > > > And
> > > > > > > > after all, there's nothing to say we *have* to achieve the same
> > > > > > > > setup as in IDL.
> > > > > > > 
> > > > > > > It sure seems to me that the time is ripe to start trying to
> > > > > > > coalesce
> > > > > > > some of the ongoing efforts.
> > > > > > > 
> > > > > > > Mind you, I don't think it is necessarily good to start with only
> > > > > > > one
> > > > > > > version. Allowing a few different approaches initially has its
> > > > > > > benefits. You get to see more approaches and ideas in play and
> > > > > > > having
> > > > > > > experience with them is very helpful in deciding which one is 
> > > > > > > more
> > > > > > > productive. And sometimes there is room for more than one in the
> > > > > > > long
> > > > > > > run. The different approaches may have their own niches. But it 
> > > > > > > is
> > > > > > > hard to imagine any long-term need for more than two or three
> > > > > > > different approaches.
> > > > > > > 
> > > > > > > Early on there are some pragmatic needs for different approaches.
> > > > > > > For
> > > > > > > example, having a fairly "literal" translation of IDL tools into
> > > > > > > Python has its benefit. It is very useful for those that would
> > > > > > > like to
> > > > > > > migrate IDL code, and given the existing IDL versions, make it 
> > > > > > > much
> > > > > > > easier to test their correctness. But I don't see this as a
> > > > > > > substitute
> > > > > > > for a good set of modular tools that have a better object 
> > > > > > > design and
> > > > > > > consistent interfaces with other modules. Doing this is more work
> > > > > > > and
> > > > > > > will take more time. So a need for both approaches is likely. 
> > > > > > > Some
> > > > > > > could argue the same for replacing IRAF tasks.
> > > > > > > 
> > > > > > > All this is much easier said than done of course.
> > > > > > > 
> > > > > > > I wish STScI had more resources to devote to this than we've
> > > > > > > actually
> > > > > > > had. We've been planning to do more on this front than we've
> > > > > > > actually
> > > > > > > done. Things come up repeatedly that ruin these plans. But it 
> > > > > > > may be
> > > > > > > worth saying where some of our current efforts are going that may
> > > > > > > overlap some of these other efforts.
> > > > > > > 
> > > > > > > 1) We've been planning (along with Gemini, and particularly James
> > > > > > > Turner), to try to develop some Sage-like installation package 
> > > > > > > that
> > > > > > > would make it easy to install all the basic tools for most
> > > > > > > astronomers. We had hoped to have a beta version out, but one 
> > > > > > > of the
> > > > > > > people working on this left at the end of last year, and we've 
> > > > > > > not
> > > > > > > been able to replace that person. We are going to continue this
> > > > > > > effort
> > > > > > > with existing staff though. Hopefully in a few months we'll have
> > > > > > > something to try out.
> > > > > > > 
> > > > > > > 2) There is a recognition that a more serious effort needs to be
> > > > > > > made
> > > > > > > to replace IRAF functionality. Perhaps one of the benefits of a 
> > > > > > > JWST
> > > > > > > delay is that it will allow us to do some of that work more
> > > > > > > explicitly. But we would not do it by replacing IRAF tasks one- 
> > > > > > > for-
> > > > > > > one
> > > > > > > but coming up with an entirely different approach which has to 
> > > > > > > start
> > > > > > > from the bottom up (the end result could have applications that
> > > > > > > mostly
> > > > > > > emulate IRAF tasks, but also provide much more modular tools).
> > > > > > > 
> > > > > > > 3) More specifically, we are currently focussing on how to handle
> > > > > > > WCS
> > > > > > > issues in a more general way than they are handled in FITS. If 
> > > > > > > there
> > > > > > > is interest, perhaps we should say more about the intended 
> > > > > > > approach.
> > > > > > > This is particularly important for replacing spectrographic 
> > > > > > > tools in
> > > > > > > IRAF, and this is where we are starting our effort.
> > > > > > > 
> > > > > > > 4) We need a way of saving these WCS models, and FITS is not the
> > > > > > > way.
> > > > > > > We are looking at an alternate data format, not just for WCS 
> > > > > > > models,
> > > > > > > but for data in general [gasp!]. Work has begun on this too.
> > > > > > > 
> > > > > > > 5) A lot of our recent work has been on pysynphot and ETCs. We
> > > > > > > plan on
> > > > > > > making the computational part of our ETCs a released tool. But 
> > > > > > > I'm
> > > > > > > also wondering if we can generalize the pysynphot spectral models
> > > > > > > for
> > > > > > > more general use in spectral tools.
> > > > > > > 
> > > > > > > 6) We have been working on a framework for making pipelines 
> > > > > > > easier
> > > > > > > to
> > > > > > > build and configure. That won't be ready for at least a few 
> > > > > > > months,
> > > > > > > but could well be of general interest and use.
> > > > > > > 
> > > > > > > But besides these things, I would like to see if we can't begin 
> > > > > > > the
> > > > > > > effort of narrowing some of the underlying libraries being used.
> > > > > > > FITS
> > > > > > > WCS is one obvious area that seems ripe for that.
> > > > > > > 
> > > > > > > But the community ought to identify one or two areas that are 
> > > > > > > of the
> > > > > > > most interest in consolidating (let's start small). What should 
> > > > > > > we
> > > > > > > start with? Focus is important in making any progress in this 
> > > > > > > area.
> > > > > > > 
> > > > > > > Perry
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > _______________________________________________
> > > > > > > AstroPy mailing list
> > > > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > > 
> > > > > 
> > > > > --
> > > > > James E.H. Turner
> > > > > Gemini Observatory Southern Operations Centre,
> > > > > Casilla 603, Tel. (+56) 51 205609
> > > > > La Serena, Chile. Fax. (+56) 51 205650
> > > > > _______________________________________________
> > > > > AstroPy mailing list
> > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > 
> > > > _______________________________________________
> > > > AstroPy mailing list
> > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > 
> > > 
> > > 
> > > --
> > > Kelle Cruz, PhD ? http://kellecruz.com/
> > > 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
> > > 
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > http://mail.scipy.org/mailman/listinfo/astropy
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Mon, 13 Jun 2011 15:23:05 -0400
> From: Kelle Cruz <kellecruz at gmail.com (mailto:kellecruz at gmail.com)>
> Subject: Re: [AstroPy] Proliferating py-astro-libs
> To: Tom Aldcroft <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>
> Cc: astropy <astropy at scipy.org (mailto:astropy at scipy.org)>
> Message-ID: <BANLkTinLndKjGCt7OWWXrSuZ+X51t2LiZg at mail.gmail.com (mailto:BANLkTinLndKjGCt7OWWXrSuZ+X51t2LiZg at mail.gmail.com)>
> Content-Type: text/plain; charset="utf-8"
> 
> For those not in the know, like I was, SciPy 2011 is July 11-16 in Austin.
> Registration is $200 for Academics and $150 for students.
> 
> The one thing I can say about this is that it is CRAZY hot in the Texas Hill
> Country right now and it's only going to get worse. (My parents live halfway
> between San Antonio and Austin.) If you decided to go with this option, I
> would likely not attend.
> 
> That said, if there are > 3 ppl on this list already planning on going, then
> you guys should definitely get together to have something a bit more formal
> than just a chat. You could plan the next meeting, evaluate the existing
> packages, start to sketch out a roadmap, maybe even assign some tasks, or
> create some leadership roles (e.g., Lead of Phot packages, Lead of Spectra
> packages, Lead of Xray), etc.
> 
> kelle
> 
> On Mon, Jun 13, 2011 at 3:07 PM, Tom Aldcroft <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)
> > wrote:
> 
> > What about a splinter meeting at SciPy2011? I guess the question is
> > how many interested parties will NOT be there this year.
> > 
> > - Tom
> > 
> > On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz <kellecruz at gmail.com (mailto:kellecruz at gmail.com)> wrote:
> > > I think a sit-down is desperately needed to resolve these issues, figure
> > out
> > > the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
> > > progress to occur.
> > > I'd be happy to participate as a non-python/programming expert and maybe
> > > provide the voice of the "users".
> > > I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
> > > deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
> > > Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that
> > seems
> > > to be where most of the movers and shakers in this game are located.
> > > kelle
> > > 
> > > On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield <perry at stsci.edu (mailto:perry at stsci.edu)>
> > wrote:
> > > > 
> > > > That's a good idea.
> > > > 
> > > > Perry
> > > > 
> > > > On Jun 13, 2011, at 1:43 PM, James Turner wrote:
> > > > 
> > > > > Maybe we should hold an AstroPy conference, where we can discuss
> > > > > co-ordination, get to know each other better and even sit down and
> > > > > work on libraries together (like at SciPy). That might help generate
> > > > > a bit of momentum. Some of us have had meetings before that were
> > > > > full of ideas that didn't go anywhere, but I don't think it has to
> > > > > be that way if active people on the ground are talking to one another
> > > > > rather than having institutions present their plans and try to
> > > > > negotiate at a high level.
> > > > > 
> > > > > 
> > > > > On 13/06/11 13:25, James Turner wrote:
> > > > > > It seems that several of us would really like to improve
> > > > > > collaboration on Python libraries but have been struggling to pull
> > > > > > it off. I've raised the same issue on this list in past months, but
> > > > > > my
> > > > > > focus has unavoidably been on other things and since I'm wary of
> > > > > > shouting a lot without contributing much, I haven't really been able
> > > > > > to keep the discussion alive...
> > > > > > 
> > > > > > I tend to agree with Mark and Stefan about the question of
> > > > > > leadership.
> > > > > > Perry & co. at Space Telescope deserve recognition for getting us
> > > > > > this
> > > > > > far with things like PyFITS and PyRAF. Others have taken the
> > > > > > initiative
> > > > > > with things like astronomical plotting and documentation sprints.
> > > > > > We're
> > > > > > still lacking a bit of coherence though, which (as Mark suggests) is
> > > > > > likely to involve one or a few dedicated, energetic, knowledgeable,
> > > > > > hands-on developer(s) who can glue things together. Those people need
> > > > > > to be employed by someone, though, to ensure stability & continuity
> > > > > > (fortunately there's already a bit of that going on at STScI, eg.
> > > > > > with
> > > > > > Mike and Matplotlib). Personally, I have the motivation but have not
> > > > > > had the time/independence (and might not be assertive enough).
> > > > > > Apparently we do have several energetic authors in the community now
> > > > > > (like Thomas & Eli), but each with their own project.
> > > > > > 
> > > > > > A couple of years ago, a number of us at the observatories
> > > > > > submitted a
> > > > > > white paper to the Decadal Survey, pointing out the need for more
> > > > > > co-ordinated funding so that we can have people who focus on cross-
> > > > > > institutional platform development & support. The report from the
> > > > > > committee did give a nod to our concerns and their importance, but
> > > > > > stopped short of making any recommendation, which basically means
> > > > > > "good
> > > > > > luck with that". Meanwhile, at Gemini we have had our own problems to
> > > > > > deal with, which make it very difficult for me to propose something
> > > > > > internally beyond working with STScI on the distribution of
> > > > > > dependencies that Perry mentioned. Perhaps someone obtaining a grant
> > > > > > for this is not out of the question though.
> > > > > > 
> > > > > > I would like it if we could get together organically behind Astrolib,
> > > > > > but sometimes it's difficult to get people away from their immediate
> > > > > > priorities to focus on that, even within my own institution. If we
> > > > > > could get people dedicated to it, though, it could become
> > > > > > indispensable
> > > > > > enough to attract and co-ordinate more effort. I'm just not sure
> > > > > > how we
> > > > > > get started at this point and my personal options for tackling the
> > > > > > problem seem limited given the overarching funding transition at
> > > > > > Gemini
> > > > > > and the intense focus on projects that are needed to make that
> > > > > > work...
> > > > > > 
> > > > > > Cheers,
> > > > > > 
> > > > > > James.
> > > > > > 
> > > > > > 
> > > > > > On 10/06/11 09:48, Perry Greenfield wrote:
> > > > > > > 
> > > > > > > On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
> > > > > > > 
> > > > > > > > I just wanted to also add that (in agreement with Marshall) I'm all
> > > > > > > > in favor of many small modules that accomplish a particular task
> > > > > > > > well, rather than packages that aim for a 'do-it-all' approach and
> > > > > > > > fall short. It's always possible to bundle small packages together
> > > > > > > > afterwards, and I don't mean merge development, but instead just
> > > > > > > > bundling the packages for installation (kind of like EPD). I think
> > > > > > > > that is the easiest approach for all of us.
> > > > > > > > 
> > > > > > > > Maybe in the long run, a specific set of core packages will emerge
> > > > > > > > as essential and we can then talk about truly merging them into a
> > > > > > > > scipy-like package, but for now, I think the race is still on. And
> > > > > > > > after all, there's nothing to say we *have* to achieve the same
> > > > > > > > setup as in IDL.
> > > > > > > 
> > > > > > > It sure seems to me that the time is ripe to start trying to
> > > > > > > coalesce
> > > > > > > some of the ongoing efforts.
> > > > > > > 
> > > > > > > Mind you, I don't think it is necessarily good to start with only
> > > > > > > one
> > > > > > > version. Allowing a few different approaches initially has its
> > > > > > > benefits. You get to see more approaches and ideas in play and
> > > > > > > having
> > > > > > > experience with them is very helpful in deciding which one is more
> > > > > > > productive. And sometimes there is room for more than one in the
> > > > > > > long
> > > > > > > run. The different approaches may have their own niches. But it is
> > > > > > > hard to imagine any long-term need for more than two or three
> > > > > > > different approaches.
> > > > > > > 
> > > > > > > Early on there are some pragmatic needs for different approaches.
> > > > > > > For
> > > > > > > example, having a fairly "literal" translation of IDL tools into
> > > > > > > Python has its benefit. It is very useful for those that would
> > > > > > > like to
> > > > > > > migrate IDL code, and given the existing IDL versions, make it much
> > > > > > > easier to test their correctness. But I don't see this as a
> > > > > > > substitute
> > > > > > > for a good set of modular tools that have a better object design and
> > > > > > > consistent interfaces with other modules. Doing this is more work
> > > > > > > and
> > > > > > > will take more time. So a need for both approaches is likely. Some
> > > > > > > could argue the same for replacing IRAF tasks.
> > > > > > > 
> > > > > > > All this is much easier said than done of course.
> > > > > > > 
> > > > > > > I wish STScI had more resources to devote to this than we've
> > > > > > > actually
> > > > > > > had. We've been planning to do more on this front than we've
> > > > > > > actually
> > > > > > > done. Things come up repeatedly that ruin these plans. But it may be
> > > > > > > worth saying where some of our current efforts are going that may
> > > > > > > overlap some of these other efforts.
> > > > > > > 
> > > > > > > 1) We've been planning (along with Gemini, and particularly James
> > > > > > > Turner), to try to develop some Sage-like installation package that
> > > > > > > would make it easy to install all the basic tools for most
> > > > > > > astronomers. We had hoped to have a beta version out, but one of the
> > > > > > > people working on this left at the end of last year, and we've not
> > > > > > > been able to replace that person. We are going to continue this
> > > > > > > effort
> > > > > > > with existing staff though. Hopefully in a few months we'll have
> > > > > > > something to try out.
> > > > > > > 
> > > > > > > 2) There is a recognition that a more serious effort needs to be
> > > > > > > made
> > > > > > > to replace IRAF functionality. Perhaps one of the benefits of a JWST
> > > > > > > delay is that it will allow us to do some of that work more
> > > > > > > explicitly. But we would not do it by replacing IRAF tasks one-for-
> > > > > > > one
> > > > > > > but coming up with an entirely different approach which has to start
> > > > > > > from the bottom up (the end result could have applications that
> > > > > > > mostly
> > > > > > > emulate IRAF tasks, but also provide much more modular tools).
> > > > > > > 
> > > > > > > 3) More specifically, we are currently focussing on how to handle
> > > > > > > WCS
> > > > > > > issues in a more general way than they are handled in FITS. If there
> > > > > > > is interest, perhaps we should say more about the intended approach.
> > > > > > > This is particularly important for replacing spectrographic tools in
> > > > > > > IRAF, and this is where we are starting our effort.
> > > > > > > 
> > > > > > > 4) We need a way of saving these WCS models, and FITS is not the
> > > > > > > way.
> > > > > > > We are looking at an alternate data format, not just for WCS models,
> > > > > > > but for data in general [gasp!]. Work has begun on this too.
> > > > > > > 
> > > > > > > 5) A lot of our recent work has been on pysynphot and ETCs. We
> > > > > > > plan on
> > > > > > > making the computational part of our ETCs a released tool. But I'm
> > > > > > > also wondering if we can generalize the pysynphot spectral models
> > > > > > > for
> > > > > > > more general use in spectral tools.
> > > > > > > 
> > > > > > > 6) We have been working on a framework for making pipelines easier
> > > > > > > to
> > > > > > > build and configure. That won't be ready for at least a few months,
> > > > > > > but could well be of general interest and use.
> > > > > > > 
> > > > > > > But besides these things, I would like to see if we can't begin the
> > > > > > > effort of narrowing some of the underlying libraries being used.
> > > > > > > FITS
> > > > > > > WCS is one obvious area that seems ripe for that.
> > > > > > > 
> > > > > > > But the community ought to identify one or two areas that are of the
> > > > > > > most interest in consolidating (let's start small). What should we
> > > > > > > start with? Focus is important in making any progress in this area.
> > > > > > > 
> > > > > > > Perry
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > _______________________________________________
> > > > > > > AstroPy mailing list
> > > > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > > 
> > > > > 
> > > > > --
> > > > > James E.H. Turner
> > > > > Gemini Observatory Southern Operations Centre,
> > > > > Casilla 603, Tel. (+56) 51 205609
> > > > > La Serena, Chile. Fax. (+56) 51 205650
> > > > > _______________________________________________
> > > > > AstroPy mailing list
> > > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > > 
> > > > _______________________________________________
> > > > AstroPy mailing list
> > > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > > http://mail.scipy.org/mailman/listinfo/astropy
> > > 
> > > 
> > > 
> > > --
> > > Kelle Cruz, PhD ? http://kellecruz.com/
> > > 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
> > > 
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > > http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> 
> -- 
> Kelle Cruz, PhD ? http://kellecruz.com/
> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.scipy.org/pipermail/astropy/attachments/20110613/ad224ff8/attachment.html 
> 
> ------------------------------
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> End of AstroPy Digest, Vol 58, Issue 16
> ***************************************




From johann.cohentanugi at gmail.com  Tue Jun 14 03:20:08 2011
From: johann.cohentanugi at gmail.com (Johann Cohen-Tanugi)
Date: Tue, 14 Jun 2011 09:20:08 +0200
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <2D017079118B4CF094C59CCADADCDFFD@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>
	<4DF64C59.3010501@gemini.edu>	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>	<4DF678BF.3070505@gemini.edu>	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>	<BANLkTikPYT7zZvB9vVwz3pjkgZyXgtU84w@mail.gmail.com>
	<2D017079118B4CF094C59CCADADCDFFD@gmail.com>
Message-ID: <4DF70BA8.9060505@gmail.com>

hi

On 06/14/2011 01:26 AM, Thomas Robitaille wrote:
> I'd like to suggest that we start figuring out who is interested in contributing to a common astronomy package, and what you are most interested in. To get things moving, I created a simple wiki page here:
>
> http://astropy.wikispaces.com/developers
thanks for doing this, I just updated it.
> Please add your name to register interest, and what areas you are interested in (feel free to add areas). This will make it easier to judge interest for face-to-face meetings and for coordinating working groups to focus on various areas. I picked the wiki host at random, but of course we can move anytime if you have a better solution (for example if we use GitHub for development, we can use the project wiki on there).
I do believe we should agree on such a platform fairly quickly, and I 
also feel that all indicators from other communities that we tend to 
also belong to (numpy, scipy, ipython, matplotlib?) are pointing toward 
github as being a de facto natural choice.
> Note, should we create a new mailing list, say astropy-dev for people interested in actual development, as opposed to people who just want to use stuff and have questions?
I think it is too early for that. We still want to reach potentially 
everyone....

best,
Johann
> Cheers,
> Tom
>
>
> On Monday, June 13, 2011 at 5:44 PM, Eric G. Barron wrote:
>
>> I don't think it would be much harder to have remote participation. Plenty of software projects have online-only or joint face-to-face and online developer meetings and sprints using tools like IRC, wikis, ticket trackers... It would certainly help people like me, who would like to participate but who have no funding to physically travel to a meeting.
>>
>> Eric
>>
>>
>> On Mon, Jun 13, 2011 at 15:14, Thomas Robitaille<thomas.robitaille at gmail.com (mailto:thomas.robitaille at gmail.com)>  wrote:
>>> Just to put the suggestion out there, Tom A. and I would be happy to try and host the meeting at CfA if there was interest for this, preferably in the fall. We have webcast capabilities for the talks (I like the idea of 'lightning' talks), though it would be harder to have remote participation for the hands-on part.
>>>
>>>   Of course, we're also open to other locations, so if anyone else interested in hosting this, please feel free to suggest!
>>>
>>>   Cheers,
>>>   Tom
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>> http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From johann.cohentanugi at gmail.com  Tue Jun 14 03:47:43 2011
From: johann.cohentanugi at gmail.com (Johann Cohen-Tanugi)
Date: Tue, 14 Jun 2011 09:47:43 +0200
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
Message-ID: <4DF7121F.2020508@gmail.com>



On 06/14/2011 02:20 AM, Eli Bressert wrote:
> Organizing a splinter meeting at the winter AAS would be a great idea (as Kelle suggested) to get the AstroPy movement going. I'll be at the meeting this year and would be happy to help out on this front.
I would still like a lot the idea of ensuring remote participation. 
Let's keep that in mind, if possible.
> It seems to me that the strength of Python and modules is that everyone has access to the language (unlike IDL) and multiple modules are developed all the time. If we look around the astronomy community there is already a large number of Python modules for astronomy (e.g. PyFITS, astropysics, asciitables, and many more - see http://www.astropython.org/resources). One of the main difficulties that we are having is integration between the modules.
>
well, isolated development, and lack of standards in terms of core 
libraries, is the main difficulties I believe. As it has been mentioned 
on this thread, having concurrent ideas sprouting is not necessariily 
evil. What we truly need is a common platform for code development, and 
at the same time having a team (and a BDFL) ramping up in evaluating the 
core libs and making decision for the longer term. A single place for 
development/distribution would help people realize that other tools are 
already available, and the core team evaluate what is available out there.
> Most of the astronomy modules for Python use Numpy and Scipy (frequently Matplotlib too), but things diverge fairly quickly from that point on. Should we agree on a list of the core astronomy modules? Is this possible and is it a good idea? All of the things I have mentioned has shown up on this mailing list before, but it would be useful to move this forward.
I would +1 the starting assumption that numpy+pyfits is uncontroversial. 
Then 2D plotting should probably be matplotlib based, but there are 2 
efforts there : APLpy and Jae-Joon wcsgrid2, which are complementary in 
terms of focus, but potentially concurrent in the longer terms (wcsgrid2 
is a direct use of matplotlib C++ libs to plot astronomical image, while 
APLpy adds an independent C++ layer to matplotlib [correct me if I am 
wrong]).

We should be careful with 3D plotting : first because matplotlib has a 
basic 3D rendering, but mayavi does much better while carrying a heavier 
set of dependencies (vtk, traits? again correct me if I am wrong), and 
second because with 3D we entering the realm of high level dedicated 
scientific needs (think structure formation and n-body simulations for 
cosmologists, single photon space/energy or space/time investigations 
for g-ray astrophysicists, etc....); so we should probably go slowly 
with 3D, and it is not an urgent matter, IMHO.

As far as WCS is concerned, I believe there is Calabretta's wcslib and 
WCSTools out there. Then Kapteyn and pyWCS link to WCSLIB, while 
WCSTools has its own python wrapping IIRC. So some decisions will be 
needed here. I would contend that this is one of the urgent matters to 
settle. Note that wcsgrid2 can be built against either pywcs and 
Kapteyn, so we may not have to choose against either one.

I hope that I am not the only one here :), but I would like to emphasize 
that a gamma-ray physicist, who may be using fairly different high-level 
tools for analysis of data from Fermi LAT for instance, is *very* 
interested in this thread :) Let's keep in mind :
- CCD based image reduction
- single photon astrophysics
- cosmology (I believe we are going to have to look into healpix  as a 
core lib for instance)
- exoplanets
- amateur astronomy
- many other specialized area of astronomy/astrophysics....
> One thought that came up (this is *only* a suggestion) is should we have a basic python package called astropy (when we have agreed on the basic packages for it)? If we do this we could use a similar scheme to Scipy's Scikits (call it astrokits?). This would ensure that the modules would use core astronomy modules. Again, this is just a exploratory idea and I would be happy to hear what others have to say.
yes, this seems like the natural path to try to follow : create a 
toolkit astropy, with core libs and standard core integrated after 
review, and a Scikits-like central for independent code hacking. It 
would be worth to cross-check with scipy-dev what kind of issues were 
raised with scikits use..... I seem to remember threads about related 
problems.

sorry for this long email, these are just a few thoughts that came to my 
mind.
best,
Johann
> Cheers,
> Eli
>
>
>
>
>
> On Monday, June 13, 2011 at 1:23 PM, astropy-request at scipy.org wrote:
>
>> Send AstroPy mailing list submissions to
>> astropy at scipy.org (mailto:astropy at scipy.org)
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://mail.scipy.org/mailman/listinfo/astropy
>> or, via email, send a message with subject or body 'help' to
>> astropy-request at scipy.org (mailto:astropy-request at scipy.org)
>>
>> You can reach the person managing the list at
>> astropy-owner at scipy.org (mailto:astropy-owner at scipy.org)
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of AstroPy digest..."
>>
>>
>> Today's Topics:
>>
>>   1. Re: Proliferating py-astro-libs (Matthew Turk)
>>   2. Re: Proliferating py-astro-libs (Perry Greenfield)
>>   3. Re: Proliferating py-astro-libs (Kelle Cruz)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Mon, 13 Jun 2011 12:11:13 -0700
>> From: Matthew Turk<matthewturk at gmail.com (mailto:matthewturk at gmail.com)>
>> Subject: Re: [AstroPy] Proliferating py-astro-libs
>> To: Tom Aldcroft<aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>
>> Cc: astropy<astropy at scipy.org (mailto:astropy at scipy.org)>
>> Message-ID:<BANLkTinvOyspdCdzB_2+=tjv1XNcPSkSUA at mail.gmail.com (mailto:tjv1XNcPSkSUA at mail.gmail.com)>
>> Content-Type: text/plain; charset=windows-1252
>>
>> Hi all,
>>
>> I'd be very interested in participating in a discussion of these
>> issues. I am not an observer, but I am a member of the astrophysical
>> simulations community concerned with issues of interop, open science,
>> and developing community-focused packages for scientific analysis, and
>> I believe that increasing cross-talk between the observation
>> community and the simulations community would be of great benefit to
>> all concerned.
>>
>> I won't be at SciPy (haven't been able to make it since 2009,
>> unfortunately!) but I think this kind of discussion might warrant its
>> own mini-workshop.
>>
>> -Matt
>>
>> On Mon, Jun 13, 2011 at 12:07 PM, Tom Aldcroft
>> <aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>  wrote:
>>> What about a splinter meeting at SciPy2011? ?I guess the question is
>>> how many interested parties will NOT be there this year.
>>>
>>> - Tom
>>>
>>> On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz<kellecruz at gmail.com (mailto:kellecruz at gmail.com)>  wrote:
>>>> I think a sit-down is desperately needed to resolve these issues, figure out
>>>> the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
>>>> progress to occur.
>>>> I'd be happy to participate as a non-python/programming expert and maybe
>>>> provide the voice of the "users".
>>>> I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
>>>> deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
>>>> Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that seems
>>>> to be where most of the movers and shakers in this game are located.
>>>> kelle
>>>>
>>>> On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield<perry at stsci.edu (mailto:perry at stsci.edu)>  wrote:
>>>>> That's a good idea.
>>>>>
>>>>> Perry
>>>>>
>>>>> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>>>>>
>>>>>> Maybe we should hold an AstroPy conference, where we can discuss
>>>>>> co-ordination, get to know each other better and even sit down and
>>>>>> work on libraries together (like at SciPy). That might help generate
>>>>>> a bit of momentum. Some of us have had meetings before that were
>>>>>> full of ideas that didn't go anywhere, but I don't think it has to
>>>>>> be that way if active people on the ground are talking to one another
>>>>>> rather than having institutions present their plans and try to
>>>>>> negotiate at a high level.
>>>>>>
>>>>>>
>>>>>> On 13/06/11 13:25, James Turner wrote:
>>>>>>> It seems that several of us would really like to improve
>>>>>>> collaboration on Python libraries but have been struggling to pull
>>>>>>> it off. I've raised the same issue on this list in past months, but
>>>>>>> my
>>>>>>> focus has unavoidably been on other things and since I'm wary of
>>>>>>> shouting a lot without contributing much, I haven't really been able
>>>>>>> to keep the discussion alive...
>>>>>>>
>>>>>>> I tend to agree with Mark and Stefan about the question of
>>>>>>> leadership.
>>>>>>> Perry&  co. at Space Telescope deserve recognition for getting us
>>>>>>> this
>>>>>>> far with things like PyFITS and PyRAF. Others have taken the
>>>>>>> initiative
>>>>>>> with things like astronomical plotting and documentation sprints.
>>>>>>> We're
>>>>>>> still lacking a bit of coherence though, which (as Mark suggests) is
>>>>>>> likely to involve one or a few dedicated, energetic, knowledgeable,
>>>>>>> hands-on developer(s) who can glue things together. Those people need
>>>>>>> to be employed by someone, though, to ensure stability&  continuity
>>>>>>> (fortunately there's already a bit of that going on at STScI, eg.
>>>>>>> with
>>>>>>> Mike and Matplotlib). Personally, I have the motivation but have not
>>>>>>> had the time/independence (and might not be assertive enough).
>>>>>>> Apparently we do have several energetic authors in the community now
>>>>>>> (like Thomas&  Eli), but each with their own project.
>>>>>>>
>>>>>>> A couple of years ago, a number of us at the observatories
>>>>>>> submitted a
>>>>>>> white paper to the Decadal Survey, pointing out the need for more
>>>>>>> co-ordinated funding so that we can have people who focus on cross-
>>>>>>> institutional platform development&  support. The report from the
>>>>>>> committee did give a nod to our concerns and their importance, but
>>>>>>> stopped short of making any recommendation, which basically means
>>>>>>> "good
>>>>>>> luck with that". Meanwhile, at Gemini we have had our own problems to
>>>>>>> deal with, which make it very difficult for me to propose something
>>>>>>> internally beyond working with STScI on the distribution of
>>>>>>> dependencies that Perry mentioned. Perhaps someone obtaining a grant
>>>>>>> for this is not out of the question though.
>>>>>>>
>>>>>>> I would like it if we could get together organically behind Astrolib,
>>>>>>> but sometimes it's difficult to get people away from their immediate
>>>>>>> priorities to focus on that, even within my own institution. If we
>>>>>>> could get people dedicated to it, though, it could become
>>>>>>> indispensable
>>>>>>> enough to attract and co-ordinate more effort. I'm just not sure
>>>>>>> how we
>>>>>>> get started at this point and my personal options for tackling the
>>>>>>> problem seem limited given the overarching funding transition at
>>>>>>> Gemini
>>>>>>> and the intense focus on projects that are needed to make that
>>>>>>> work...
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> James.
>>>>>>>
>>>>>>>
>>>>>>> On 10/06/11 09:48, Perry Greenfield wrote:
>>>>>>>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>>>>>>
>>>>>>>>> I just wanted to also add that (in agreement with Marshall) I'm all
>>>>>>>>> in favor of many small modules that accomplish a particular task
>>>>>>>>> well, rather than packages that aim for a 'do-it-all' approach and
>>>>>>>>> fall short. It's always possible to bundle small packages together
>>>>>>>>> afterwards, and I don't mean merge development, but instead just
>>>>>>>>> bundling the packages for installation (kind of like EPD). I think
>>>>>>>>> that is the easiest approach for all of us.
>>>>>>>>>
>>>>>>>>> Maybe in the long run, a specific set of core packages will emerge
>>>>>>>>> as essential and we can then talk about truly merging them into a
>>>>>>>>> scipy-like package, but for now, I think the race is still on. And
>>>>>>>>> after all, there's nothing to say we *have* to achieve the same
>>>>>>>>> setup as in IDL.
>>>>>>>> It sure seems to me that the time is ripe to start trying to
>>>>>>>> coalesce
>>>>>>>> some of the ongoing efforts.
>>>>>>>>
>>>>>>>> Mind you, I don't think it is necessarily good to start with only
>>>>>>>> one
>>>>>>>> version. Allowing a few different approaches initially has its
>>>>>>>> benefits. You get to see more approaches and ideas in play and
>>>>>>>> having
>>>>>>>> experience with them is very helpful in deciding which one is more
>>>>>>>> productive. And sometimes there is room for more than one in the
>>>>>>>> long
>>>>>>>> run. The different approaches may have their own niches. But it is
>>>>>>>> hard to imagine any long-term need for more than two or three
>>>>>>>> different approaches.
>>>>>>>>
>>>>>>>> Early on there are some pragmatic needs for different approaches.
>>>>>>>> For
>>>>>>>> example, having a fairly "literal" translation of IDL tools into
>>>>>>>> Python has its benefit. It is very useful for those that would
>>>>>>>> like to
>>>>>>>> migrate IDL code, and given the existing IDL versions, make it much
>>>>>>>> easier to test their correctness. But I don't see this as a
>>>>>>>> substitute
>>>>>>>> for a good set of modular tools that have a better object design and
>>>>>>>> consistent interfaces with other modules. Doing this is more work
>>>>>>>> and
>>>>>>>> will take more time. So a need for both approaches is likely. Some
>>>>>>>> could argue the same for replacing IRAF tasks.
>>>>>>>>
>>>>>>>> All this is much easier said than done of course.
>>>>>>>>
>>>>>>>> I wish STScI had more resources to devote to this than we've
>>>>>>>> actually
>>>>>>>> had. We've been planning to do more on this front than we've
>>>>>>>> actually
>>>>>>>> done. Things come up repeatedly that ruin these plans. But it may be
>>>>>>>> worth saying where some of our current efforts are going that may
>>>>>>>> overlap some of these other efforts.
>>>>>>>>
>>>>>>>> 1) We've been planning (along with Gemini, and particularly James
>>>>>>>> Turner), to try to develop some Sage-like installation package that
>>>>>>>> would make it easy to install all the basic tools for most
>>>>>>>> astronomers. We had hoped to have a beta version out, but one of the
>>>>>>>> people working on this left at the end of last year, and we've not
>>>>>>>> been able to replace that person. We are going to continue this
>>>>>>>> effort
>>>>>>>> with existing staff though. Hopefully in a few months we'll have
>>>>>>>> something to try out.
>>>>>>>>
>>>>>>>> 2) There is a recognition that a more serious effort needs to be
>>>>>>>> made
>>>>>>>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
>>>>>>>> delay is that it will allow us to do some of that work more
>>>>>>>> explicitly. But we would not do it by replacing IRAF tasks one-for-
>>>>>>>> one
>>>>>>>> but coming up with an entirely different approach which has to start
>>>>>>>> from the bottom up (the end result could have applications that
>>>>>>>> mostly
>>>>>>>> emulate IRAF tasks, but also provide much more modular tools).
>>>>>>>>
>>>>>>>> 3) More specifically, we are currently focussing on how to handle
>>>>>>>> WCS
>>>>>>>> issues in a more general way than they are handled in FITS. If there
>>>>>>>> is interest, perhaps we should say more about the intended approach.
>>>>>>>> This is particularly important for replacing spectrographic tools in
>>>>>>>> IRAF, and this is where we are starting our effort.
>>>>>>>>
>>>>>>>> 4) We need a way of saving these WCS models, and FITS is not the
>>>>>>>> way.
>>>>>>>> We are looking at an alternate data format, not just for WCS models,
>>>>>>>> but for data in general [gasp!]. Work has begun on this too.
>>>>>>>>
>>>>>>>> 5) A lot of our recent work has been on pysynphot and ETCs. We
>>>>>>>> plan on
>>>>>>>> making the computational part of our ETCs a released tool. But I'm
>>>>>>>> also wondering if we can generalize the pysynphot spectral models
>>>>>>>> for
>>>>>>>> more general use in spectral tools.
>>>>>>>>
>>>>>>>> 6) We have been working on a framework for making pipelines easier
>>>>>>>> to
>>>>>>>> build and configure. That won't be ready for at least a few months,
>>>>>>>> but could well be of general interest and use.
>>>>>>>>
>>>>>>>> But besides these things, I would like to see if we can't begin the
>>>>>>>> effort of narrowing some of the underlying libraries being used.
>>>>>>>> FITS
>>>>>>>> WCS is one obvious area that seems ripe for that.
>>>>>>>>
>>>>>>>> But the community ought to identify one or two areas that are of the
>>>>>>>> most interest in consolidating (let's start small). What should we
>>>>>>>> start with? Focus is important in making any progress in this area.
>>>>>>>>
>>>>>>>> Perry
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> AstroPy mailing list
>>>>>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>>>
>>>>>> --
>>>>>> James E.H. Turner
>>>>>> Gemini Observatory Southern Operations Centre,
>>>>>> Casilla 603, ? ? ? ? ?Tel. (+56) 51 205609
>>>>>> La Serena, Chile. ? ? Fax. (+56) 51 205650
>>>>>> _______________________________________________
>>>>>> AstroPy mailing list
>>>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>> _______________________________________________
>>>>> AstroPy mailing list
>>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>>
>>>> --
>>>> Kelle Cruz, PhD ??http://kellecruz.com/
>>>> 917.725.1334 ??Hunter ext: 16486 ? AMNH ext: 3404
>>>>
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Mon, 13 Jun 2011 15:15:28 -0400
>> From: Perry Greenfield<perry at stsci.edu (mailto:perry at stsci.edu)>
>> Subject: Re: [AstroPy] Proliferating py-astro-libs
>> To: Tom Aldcroft<aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>
>> Cc: astropy<astropy at scipy.org (mailto:astropy at scipy.org)>
>> Message-ID:<D3C94402-208C-4FEF-B158-0EE12D495E5F at stsci.edu (mailto:D3C94402-208C-4FEF-B158-0EE12D495E5F at stsci.edu)>
>> Content-Type: text/plain; charset=WINDOWS-1252; format=flowed;
>>   delsp=yes
>>
>> It may be too late. Those attending may have already made travel plans
>> that preclude an extra day or two (I think it deserves at least a
>> day). And it may be too late for those that weren't planning to go.
>>
>> Perry
>>
>> On Jun 13, 2011, at 3:07 PM, Tom Aldcroft wrote:
>>
>>> What about a splinter meeting at SciPy2011? I guess the question is
>>> how many interested parties will NOT be there this year.
>>>
>>> - Tom
>>>
>>> On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz<kellecruz at gmail.com (mailto:kellecruz at gmail.com)>
>>> wrote:
>>>> I think a sit-down is desperately needed to resolve these issues,
>>>> figure out
>>>> the mgmt structure (aka, pecking order), for the BDFL to emerge,
>>>> and for
>>>> progress to occur.
>>>> I'd be happy to participate as a non-python/programming expert and
>>>> maybe
>>>> provide the voice of the "users".
>>>> I propose there be a Splinter Meeting at AAS in Austin. (Splinter
>>>> Meeting
>>>> deadline is Dec 1.) Or else someone will have to organize at CfA
>>>> (Tom A?
>>>> Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell,
>>>> that seems
>>>> to be where most of the movers and shakers in this game are located.
>>>> kelle
>>>>
>>>> On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield<perry at stsci.edu (mailto:perry at stsci.edu)>
>>>> wrote:
>>>>> That's a good idea.
>>>>>
>>>>> Perry
>>>>>
>>>>> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>>>>>
>>>>>> Maybe we should hold an AstroPy conference, where we can discuss
>>>>>> co-ordination, get to know each other better and even sit down and
>>>>>> work on libraries together (like at SciPy). That might help
>>>>>> generate
>>>>>> a bit of momentum. Some of us have had meetings before that were
>>>>>> full of ideas that didn't go anywhere, but I don't think it has to
>>>>>> be that way if active people on the ground are talking to one
>>>>>> another
>>>>>> rather than having institutions present their plans and try to
>>>>>> negotiate at a high level.
>>>>>>
>>>>>>
>>>>>> On 13/06/11 13:25, James Turner wrote:
>>>>>>> It seems that several of us would really like to improve
>>>>>>> collaboration on Python libraries but have been struggling to pull
>>>>>>> it off. I've raised the same issue on this list in past months,
>>>>>>> but
>>>>>>> my
>>>>>>> focus has unavoidably been on other things and since I'm wary of
>>>>>>> shouting a lot without contributing much, I haven't really been
>>>>>>> able
>>>>>>> to keep the discussion alive...
>>>>>>>
>>>>>>> I tend to agree with Mark and Stefan about the question of
>>>>>>> leadership.
>>>>>>> Perry&  co. at Space Telescope deserve recognition for getting us
>>>>>>> this
>>>>>>> far with things like PyFITS and PyRAF. Others have taken the
>>>>>>> initiative
>>>>>>> with things like astronomical plotting and documentation sprints.
>>>>>>> We're
>>>>>>> still lacking a bit of coherence though, which (as Mark
>>>>>>> suggests) is
>>>>>>> likely to involve one or a few dedicated, energetic,
>>>>>>> knowledgeable,
>>>>>>> hands-on developer(s) who can glue things together. Those people
>>>>>>> need
>>>>>>> to be employed by someone, though, to ensure stability&
>>>>>>> continuity
>>>>>>> (fortunately there's already a bit of that going on at STScI, eg.
>>>>>>> with
>>>>>>> Mike and Matplotlib). Personally, I have the motivation but have
>>>>>>> not
>>>>>>> had the time/independence (and might not be assertive enough).
>>>>>>> Apparently we do have several energetic authors in the community
>>>>>>> now
>>>>>>> (like Thomas&  Eli), but each with their own project.
>>>>>>>
>>>>>>> A couple of years ago, a number of us at the observatories
>>>>>>> submitted a
>>>>>>> white paper to the Decadal Survey, pointing out the need for more
>>>>>>> co-ordinated funding so that we can have people who focus on
>>>>>>> cross-
>>>>>>> institutional platform development&  support. The report from the
>>>>>>> committee did give a nod to our concerns and their importance, but
>>>>>>> stopped short of making any recommendation, which basically means
>>>>>>> "good
>>>>>>> luck with that". Meanwhile, at Gemini we have had our own
>>>>>>> problems to
>>>>>>> deal with, which make it very difficult for me to propose
>>>>>>> something
>>>>>>> internally beyond working with STScI on the distribution of
>>>>>>> dependencies that Perry mentioned. Perhaps someone obtaining a
>>>>>>> grant
>>>>>>> for this is not out of the question though.
>>>>>>>
>>>>>>> I would like it if we could get together organically behind
>>>>>>> Astrolib,
>>>>>>> but sometimes it's difficult to get people away from their
>>>>>>> immediate
>>>>>>> priorities to focus on that, even within my own institution. If we
>>>>>>> could get people dedicated to it, though, it could become
>>>>>>> indispensable
>>>>>>> enough to attract and co-ordinate more effort. I'm just not sure
>>>>>>> how we
>>>>>>> get started at this point and my personal options for tackling the
>>>>>>> problem seem limited given the overarching funding transition at
>>>>>>> Gemini
>>>>>>> and the intense focus on projects that are needed to make that
>>>>>>> work...
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> James.
>>>>>>>
>>>>>>>
>>>>>>> On 10/06/11 09:48, Perry Greenfield wrote:
>>>>>>>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>>>>>>
>>>>>>>>> I just wanted to also add that (in agreement with Marshall)
>>>>>>>>> I'm all
>>>>>>>>> in favor of many small modules that accomplish a particular task
>>>>>>>>> well, rather than packages that aim for a 'do-it-all' approach
>>>>>>>>> and
>>>>>>>>> fall short. It's always possible to bundle small packages
>>>>>>>>> together
>>>>>>>>> afterwards, and I don't mean merge development, but instead just
>>>>>>>>> bundling the packages for installation (kind of like EPD). I
>>>>>>>>> think
>>>>>>>>> that is the easiest approach for all of us.
>>>>>>>>>
>>>>>>>>> Maybe in the long run, a specific set of core packages will
>>>>>>>>> emerge
>>>>>>>>> as essential and we can then talk about truly merging them
>>>>>>>>> into a
>>>>>>>>> scipy-like package, but for now, I think the race is still on.
>>>>>>>>> And
>>>>>>>>> after all, there's nothing to say we *have* to achieve the same
>>>>>>>>> setup as in IDL.
>>>>>>>> It sure seems to me that the time is ripe to start trying to
>>>>>>>> coalesce
>>>>>>>> some of the ongoing efforts.
>>>>>>>>
>>>>>>>> Mind you, I don't think it is necessarily good to start with only
>>>>>>>> one
>>>>>>>> version. Allowing a few different approaches initially has its
>>>>>>>> benefits. You get to see more approaches and ideas in play and
>>>>>>>> having
>>>>>>>> experience with them is very helpful in deciding which one is
>>>>>>>> more
>>>>>>>> productive. And sometimes there is room for more than one in the
>>>>>>>> long
>>>>>>>> run. The different approaches may have their own niches. But it
>>>>>>>> is
>>>>>>>> hard to imagine any long-term need for more than two or three
>>>>>>>> different approaches.
>>>>>>>>
>>>>>>>> Early on there are some pragmatic needs for different approaches.
>>>>>>>> For
>>>>>>>> example, having a fairly "literal" translation of IDL tools into
>>>>>>>> Python has its benefit. It is very useful for those that would
>>>>>>>> like to
>>>>>>>> migrate IDL code, and given the existing IDL versions, make it
>>>>>>>> much
>>>>>>>> easier to test their correctness. But I don't see this as a
>>>>>>>> substitute
>>>>>>>> for a good set of modular tools that have a better object
>>>>>>>> design and
>>>>>>>> consistent interfaces with other modules. Doing this is more work
>>>>>>>> and
>>>>>>>> will take more time. So a need for both approaches is likely.
>>>>>>>> Some
>>>>>>>> could argue the same for replacing IRAF tasks.
>>>>>>>>
>>>>>>>> All this is much easier said than done of course.
>>>>>>>>
>>>>>>>> I wish STScI had more resources to devote to this than we've
>>>>>>>> actually
>>>>>>>> had. We've been planning to do more on this front than we've
>>>>>>>> actually
>>>>>>>> done. Things come up repeatedly that ruin these plans. But it
>>>>>>>> may be
>>>>>>>> worth saying where some of our current efforts are going that may
>>>>>>>> overlap some of these other efforts.
>>>>>>>>
>>>>>>>> 1) We've been planning (along with Gemini, and particularly James
>>>>>>>> Turner), to try to develop some Sage-like installation package
>>>>>>>> that
>>>>>>>> would make it easy to install all the basic tools for most
>>>>>>>> astronomers. We had hoped to have a beta version out, but one
>>>>>>>> of the
>>>>>>>> people working on this left at the end of last year, and we've
>>>>>>>> not
>>>>>>>> been able to replace that person. We are going to continue this
>>>>>>>> effort
>>>>>>>> with existing staff though. Hopefully in a few months we'll have
>>>>>>>> something to try out.
>>>>>>>>
>>>>>>>> 2) There is a recognition that a more serious effort needs to be
>>>>>>>> made
>>>>>>>> to replace IRAF functionality. Perhaps one of the benefits of a
>>>>>>>> JWST
>>>>>>>> delay is that it will allow us to do some of that work more
>>>>>>>> explicitly. But we would not do it by replacing IRAF tasks one-
>>>>>>>> for-
>>>>>>>> one
>>>>>>>> but coming up with an entirely different approach which has to
>>>>>>>> start
>>>>>>>> from the bottom up (the end result could have applications that
>>>>>>>> mostly
>>>>>>>> emulate IRAF tasks, but also provide much more modular tools).
>>>>>>>>
>>>>>>>> 3) More specifically, we are currently focussing on how to handle
>>>>>>>> WCS
>>>>>>>> issues in a more general way than they are handled in FITS. If
>>>>>>>> there
>>>>>>>> is interest, perhaps we should say more about the intended
>>>>>>>> approach.
>>>>>>>> This is particularly important for replacing spectrographic
>>>>>>>> tools in
>>>>>>>> IRAF, and this is where we are starting our effort.
>>>>>>>>
>>>>>>>> 4) We need a way of saving these WCS models, and FITS is not the
>>>>>>>> way.
>>>>>>>> We are looking at an alternate data format, not just for WCS
>>>>>>>> models,
>>>>>>>> but for data in general [gasp!]. Work has begun on this too.
>>>>>>>>
>>>>>>>> 5) A lot of our recent work has been on pysynphot and ETCs. We
>>>>>>>> plan on
>>>>>>>> making the computational part of our ETCs a released tool. But
>>>>>>>> I'm
>>>>>>>> also wondering if we can generalize the pysynphot spectral models
>>>>>>>> for
>>>>>>>> more general use in spectral tools.
>>>>>>>>
>>>>>>>> 6) We have been working on a framework for making pipelines
>>>>>>>> easier
>>>>>>>> to
>>>>>>>> build and configure. That won't be ready for at least a few
>>>>>>>> months,
>>>>>>>> but could well be of general interest and use.
>>>>>>>>
>>>>>>>> But besides these things, I would like to see if we can't begin
>>>>>>>> the
>>>>>>>> effort of narrowing some of the underlying libraries being used.
>>>>>>>> FITS
>>>>>>>> WCS is one obvious area that seems ripe for that.
>>>>>>>>
>>>>>>>> But the community ought to identify one or two areas that are
>>>>>>>> of the
>>>>>>>> most interest in consolidating (let's start small). What should
>>>>>>>> we
>>>>>>>> start with? Focus is important in making any progress in this
>>>>>>>> area.
>>>>>>>>
>>>>>>>> Perry
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> AstroPy mailing list
>>>>>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>>>
>>>>>> --
>>>>>> James E.H. Turner
>>>>>> Gemini Observatory Southern Operations Centre,
>>>>>> Casilla 603, Tel. (+56) 51 205609
>>>>>> La Serena, Chile. Fax. (+56) 51 205650
>>>>>> _______________________________________________
>>>>>> AstroPy mailing list
>>>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>> _______________________________________________
>>>>> AstroPy mailing list
>>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>>
>>>> --
>>>> Kelle Cruz, PhD ? http://kellecruz.com/
>>>> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
>>>>
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Mon, 13 Jun 2011 15:23:05 -0400
>> From: Kelle Cruz<kellecruz at gmail.com (mailto:kellecruz at gmail.com)>
>> Subject: Re: [AstroPy] Proliferating py-astro-libs
>> To: Tom Aldcroft<aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)>
>> Cc: astropy<astropy at scipy.org (mailto:astropy at scipy.org)>
>> Message-ID:<BANLkTinLndKjGCt7OWWXrSuZ+X51t2LiZg at mail.gmail.com (mailto:BANLkTinLndKjGCt7OWWXrSuZ+X51t2LiZg at mail.gmail.com)>
>> Content-Type: text/plain; charset="utf-8"
>>
>> For those not in the know, like I was, SciPy 2011 is July 11-16 in Austin.
>> Registration is $200 for Academics and $150 for students.
>>
>> The one thing I can say about this is that it is CRAZY hot in the Texas Hill
>> Country right now and it's only going to get worse. (My parents live halfway
>> between San Antonio and Austin.) If you decided to go with this option, I
>> would likely not attend.
>>
>> That said, if there are>  3 ppl on this list already planning on going, then
>> you guys should definitely get together to have something a bit more formal
>> than just a chat. You could plan the next meeting, evaluate the existing
>> packages, start to sketch out a roadmap, maybe even assign some tasks, or
>> create some leadership roles (e.g., Lead of Phot packages, Lead of Spectra
>> packages, Lead of Xray), etc.
>>
>> kelle
>>
>> On Mon, Jun 13, 2011 at 3:07 PM, Tom Aldcroft<aldcroft at head.cfa.harvard.edu (mailto:aldcroft at head.cfa.harvard.edu)
>>> wrote:
>>> What about a splinter meeting at SciPy2011? I guess the question is
>>> how many interested parties will NOT be there this year.
>>>
>>> - Tom
>>>
>>> On Mon, Jun 13, 2011 at 2:59 PM, Kelle Cruz<kellecruz at gmail.com (mailto:kellecruz at gmail.com)>  wrote:
>>>> I think a sit-down is desperately needed to resolve these issues, figure
>>> out
>>>> the mgmt structure (aka, pecking order), for the BDFL to emerge, and for
>>>> progress to occur.
>>>> I'd be happy to participate as a non-python/programming expert and maybe
>>>> provide the voice of the "users".
>>>> I propose there be a Splinter Meeting at AAS in Austin. (Splinter Meeting
>>>> deadline is Dec 1.) Or else someone will have to organize at CfA (Tom A?
>>>> Thom R?) or STScI (Marshall? Perry?) since, as far as I can tell, that
>>> seems
>>>> to be where most of the movers and shakers in this game are located.
>>>> kelle
>>>>
>>>> On Mon, Jun 13, 2011 at 2:00 PM, Perry Greenfield<perry at stsci.edu (mailto:perry at stsci.edu)>
>>> wrote:
>>>>> That's a good idea.
>>>>>
>>>>> Perry
>>>>>
>>>>> On Jun 13, 2011, at 1:43 PM, James Turner wrote:
>>>>>
>>>>>> Maybe we should hold an AstroPy conference, where we can discuss
>>>>>> co-ordination, get to know each other better and even sit down and
>>>>>> work on libraries together (like at SciPy). That might help generate
>>>>>> a bit of momentum. Some of us have had meetings before that were
>>>>>> full of ideas that didn't go anywhere, but I don't think it has to
>>>>>> be that way if active people on the ground are talking to one another
>>>>>> rather than having institutions present their plans and try to
>>>>>> negotiate at a high level.
>>>>>>
>>>>>>
>>>>>> On 13/06/11 13:25, James Turner wrote:
>>>>>>> It seems that several of us would really like to improve
>>>>>>> collaboration on Python libraries but have been struggling to pull
>>>>>>> it off. I've raised the same issue on this list in past months, but
>>>>>>> my
>>>>>>> focus has unavoidably been on other things and since I'm wary of
>>>>>>> shouting a lot without contributing much, I haven't really been able
>>>>>>> to keep the discussion alive...
>>>>>>>
>>>>>>> I tend to agree with Mark and Stefan about the question of
>>>>>>> leadership.
>>>>>>> Perry&  co. at Space Telescope deserve recognition for getting us
>>>>>>> this
>>>>>>> far with things like PyFITS and PyRAF. Others have taken the
>>>>>>> initiative
>>>>>>> with things like astronomical plotting and documentation sprints.
>>>>>>> We're
>>>>>>> still lacking a bit of coherence though, which (as Mark suggests) is
>>>>>>> likely to involve one or a few dedicated, energetic, knowledgeable,
>>>>>>> hands-on developer(s) who can glue things together. Those people need
>>>>>>> to be employed by someone, though, to ensure stability&  continuity
>>>>>>> (fortunately there's already a bit of that going on at STScI, eg.
>>>>>>> with
>>>>>>> Mike and Matplotlib). Personally, I have the motivation but have not
>>>>>>> had the time/independence (and might not be assertive enough).
>>>>>>> Apparently we do have several energetic authors in the community now
>>>>>>> (like Thomas&  Eli), but each with their own project.
>>>>>>>
>>>>>>> A couple of years ago, a number of us at the observatories
>>>>>>> submitted a
>>>>>>> white paper to the Decadal Survey, pointing out the need for more
>>>>>>> co-ordinated funding so that we can have people who focus on cross-
>>>>>>> institutional platform development&  support. The report from the
>>>>>>> committee did give a nod to our concerns and their importance, but
>>>>>>> stopped short of making any recommendation, which basically means
>>>>>>> "good
>>>>>>> luck with that". Meanwhile, at Gemini we have had our own problems to
>>>>>>> deal with, which make it very difficult for me to propose something
>>>>>>> internally beyond working with STScI on the distribution of
>>>>>>> dependencies that Perry mentioned. Perhaps someone obtaining a grant
>>>>>>> for this is not out of the question though.
>>>>>>>
>>>>>>> I would like it if we could get together organically behind Astrolib,
>>>>>>> but sometimes it's difficult to get people away from their immediate
>>>>>>> priorities to focus on that, even within my own institution. If we
>>>>>>> could get people dedicated to it, though, it could become
>>>>>>> indispensable
>>>>>>> enough to attract and co-ordinate more effort. I'm just not sure
>>>>>>> how we
>>>>>>> get started at this point and my personal options for tackling the
>>>>>>> problem seem limited given the overarching funding transition at
>>>>>>> Gemini
>>>>>>> and the intense focus on projects that are needed to make that
>>>>>>> work...
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> James.
>>>>>>>
>>>>>>>
>>>>>>> On 10/06/11 09:48, Perry Greenfield wrote:
>>>>>>>> On Jun 9, 2011, at 11:12 PM, Thomas Robitaille wrote:
>>>>>>>>
>>>>>>>>> I just wanted to also add that (in agreement with Marshall) I'm all
>>>>>>>>> in favor of many small modules that accomplish a particular task
>>>>>>>>> well, rather than packages that aim for a 'do-it-all' approach and
>>>>>>>>> fall short. It's always possible to bundle small packages together
>>>>>>>>> afterwards, and I don't mean merge development, but instead just
>>>>>>>>> bundling the packages for installation (kind of like EPD). I think
>>>>>>>>> that is the easiest approach for all of us.
>>>>>>>>>
>>>>>>>>> Maybe in the long run, a specific set of core packages will emerge
>>>>>>>>> as essential and we can then talk about truly merging them into a
>>>>>>>>> scipy-like package, but for now, I think the race is still on. And
>>>>>>>>> after all, there's nothing to say we *have* to achieve the same
>>>>>>>>> setup as in IDL.
>>>>>>>> It sure seems to me that the time is ripe to start trying to
>>>>>>>> coalesce
>>>>>>>> some of the ongoing efforts.
>>>>>>>>
>>>>>>>> Mind you, I don't think it is necessarily good to start with only
>>>>>>>> one
>>>>>>>> version. Allowing a few different approaches initially has its
>>>>>>>> benefits. You get to see more approaches and ideas in play and
>>>>>>>> having
>>>>>>>> experience with them is very helpful in deciding which one is more
>>>>>>>> productive. And sometimes there is room for more than one in the
>>>>>>>> long
>>>>>>>> run. The different approaches may have their own niches. But it is
>>>>>>>> hard to imagine any long-term need for more than two or three
>>>>>>>> different approaches.
>>>>>>>>
>>>>>>>> Early on there are some pragmatic needs for different approaches.
>>>>>>>> For
>>>>>>>> example, having a fairly "literal" translation of IDL tools into
>>>>>>>> Python has its benefit. It is very useful for those that would
>>>>>>>> like to
>>>>>>>> migrate IDL code, and given the existing IDL versions, make it much
>>>>>>>> easier to test their correctness. But I don't see this as a
>>>>>>>> substitute
>>>>>>>> for a good set of modular tools that have a better object design and
>>>>>>>> consistent interfaces with other modules. Doing this is more work
>>>>>>>> and
>>>>>>>> will take more time. So a need for both approaches is likely. Some
>>>>>>>> could argue the same for replacing IRAF tasks.
>>>>>>>>
>>>>>>>> All this is much easier said than done of course.
>>>>>>>>
>>>>>>>> I wish STScI had more resources to devote to this than we've
>>>>>>>> actually
>>>>>>>> had. We've been planning to do more on this front than we've
>>>>>>>> actually
>>>>>>>> done. Things come up repeatedly that ruin these plans. But it may be
>>>>>>>> worth saying where some of our current efforts are going that may
>>>>>>>> overlap some of these other efforts.
>>>>>>>>
>>>>>>>> 1) We've been planning (along with Gemini, and particularly James
>>>>>>>> Turner), to try to develop some Sage-like installation package that
>>>>>>>> would make it easy to install all the basic tools for most
>>>>>>>> astronomers. We had hoped to have a beta version out, but one of the
>>>>>>>> people working on this left at the end of last year, and we've not
>>>>>>>> been able to replace that person. We are going to continue this
>>>>>>>> effort
>>>>>>>> with existing staff though. Hopefully in a few months we'll have
>>>>>>>> something to try out.
>>>>>>>>
>>>>>>>> 2) There is a recognition that a more serious effort needs to be
>>>>>>>> made
>>>>>>>> to replace IRAF functionality. Perhaps one of the benefits of a JWST
>>>>>>>> delay is that it will allow us to do some of that work more
>>>>>>>> explicitly. But we would not do it by replacing IRAF tasks one-for-
>>>>>>>> one
>>>>>>>> but coming up with an entirely different approach which has to start
>>>>>>>> from the bottom up (the end result could have applications that
>>>>>>>> mostly
>>>>>>>> emulate IRAF tasks, but also provide much more modular tools).
>>>>>>>>
>>>>>>>> 3) More specifically, we are currently focussing on how to handle
>>>>>>>> WCS
>>>>>>>> issues in a more general way than they are handled in FITS. If there
>>>>>>>> is interest, perhaps we should say more about the intended approach.
>>>>>>>> This is particularly important for replacing spectrographic tools in
>>>>>>>> IRAF, and this is where we are starting our effort.
>>>>>>>>
>>>>>>>> 4) We need a way of saving these WCS models, and FITS is not the
>>>>>>>> way.
>>>>>>>> We are looking at an alternate data format, not just for WCS models,
>>>>>>>> but for data in general [gasp!]. Work has begun on this too.
>>>>>>>>
>>>>>>>> 5) A lot of our recent work has been on pysynphot and ETCs. We
>>>>>>>> plan on
>>>>>>>> making the computational part of our ETCs a released tool. But I'm
>>>>>>>> also wondering if we can generalize the pysynphot spectral models
>>>>>>>> for
>>>>>>>> more general use in spectral tools.
>>>>>>>>
>>>>>>>> 6) We have been working on a framework for making pipelines easier
>>>>>>>> to
>>>>>>>> build and configure. That won't be ready for at least a few months,
>>>>>>>> but could well be of general interest and use.
>>>>>>>>
>>>>>>>> But besides these things, I would like to see if we can't begin the
>>>>>>>> effort of narrowing some of the underlying libraries being used.
>>>>>>>> FITS
>>>>>>>> WCS is one obvious area that seems ripe for that.
>>>>>>>>
>>>>>>>> But the community ought to identify one or two areas that are of the
>>>>>>>> most interest in consolidating (let's start small). What should we
>>>>>>>> start with? Focus is important in making any progress in this area.
>>>>>>>>
>>>>>>>> Perry
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> AstroPy mailing list
>>>>>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>>>
>>>>>> --
>>>>>> James E.H. Turner
>>>>>> Gemini Observatory Southern Operations Centre,
>>>>>> Casilla 603, Tel. (+56) 51 205609
>>>>>> La Serena, Chile. Fax. (+56) 51 205650
>>>>>> _______________________________________________
>>>>>> AstroPy mailing list
>>>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>> _______________________________________________
>>>>> AstroPy mailing list
>>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>>>
>>>> --
>>>> Kelle Cruz, PhD ? http://kellecruz.com/
>>>> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
>>>>
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>> -- 
>> Kelle Cruz, PhD ? http://kellecruz.com/
>> 917.725.1334 ? Hunter ext: 16486 ? AMNH ext: 3404
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: http://mail.scipy.org/pipermail/astropy/attachments/20110613/ad224ff8/attachment.html
>>
>> ------------------------------
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>> End of AstroPy Digest, Vol 58, Issue 16
>> ***************************************
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From martin.raue at desy.de  Tue Jun 14 06:45:41 2011
From: martin.raue at desy.de (Martin Raue)
Date: Tue, 14 Jun 2011 12:45:41 +0200
Subject: [AstroPy] pyfits - memory leak in new_table / 'hello, i am new here'
Message-ID: <6DDFB054-38E3-4C22-A76C-BEF594FEFE6E@desy.de>

Hello all,

I am new here. I work in very high energy gamma-ray astronomy with HESS and CTA and currently developing some high level analysis tools for CTA using python (the classical numpy, scipy, matplotlib, pyfits).

I stumbled upon a problem with memory leakage in pyfits when using new_table.

Something similar has been reported here
http://physicsnlinux.wordpress.com/2011/03/28/pyfits-memory-leak-in-new_table/#more-103
and should be fixed in
http://trac6.assembla.com/pyfits/changeset/844

But I still have problems in 844 and 851. Unfortunately, I have problems installing the latest trunk on my system (some output attached).

Any suggestion and help would be very welcome!

Best wishes,
Martin

==============
Example script

import numpy as np
import pyfits
import gc

for i in range(20) :

    # create dummy data
    a = np.linspace(-1., 1., 1000000)
    b = a ** 3.

    a_col = pyfits.Column(name='ACOL', format='1E', array=a)
    b_col = pyfits.Column(name='BCOL', format='1E', array=b)

    coldefs_new = pyfits.ColDefs([a_col, b_col])

    newtable = pyfits.new_table(coldefs_new)

    # try to clean memory
    del newtable
    del coldefs_new
    gc.collect()

gc.collect()

=======================================================
setup.py output when trying to install the pyfits trunk

Something wrong in the installer modules ...

[uh2macastro04:~/tmp/trunk] $ python setup.py install
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.15.tar.gz
Extracting in /var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/tmp4nv73s
Now working in /var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/tmp4nv73s/distribute-0.6.15
Building a Distribute egg in /Users/mraue/tmp/trunk
/Users/mraue/tmp/trunk/distribute-0.6.15-py2.6.egg
zip_safe flag not set; analyzing archive contents...

Installed /private/var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/easy_install-NTCEXF/stsci.distutils-0.2dev-r13055/d2to1-0.1.5-py2.6.egg
zip_safe flag not set; analyzing archive contents...
stsci.__init__: module references __path__

Installed /Users/mraue/tmp/trunk/stsci.distutils-0.2dev_r13055-py2.6.egg
Searching for d2to1>=0.1.5
Reading http://pypi.python.org/simple/d2to1/
Best match: d2to1 0.1.5
Downloading http://pypi.python.org/packages/source/d/d2to1/d2to1-0.1.5.tar.gz#md5=34129f809b503449b283771b5c3f038b
Processing d2to1-0.1.5.tar.gz
Running d2to1-0.1.5/setup.py -q bdist_egg --dist-dir /var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/easy_install-ABs3GN/d2to1-0.1.5/egg-dist-tmp-oSIXxB
Traceback (most recent call last):
  File "setup.py", line 33, in <module>
    use_2to3=True
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 113, in setup
    _setup_distribution = dist = klass(attrs)
  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 221, in __init__
  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 245, in fetch_build_eggs
  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 544, in resolve
  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 786, in best_match
  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 798, in obtain
  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 293, in fetch_build_egg
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 584, in easy_install
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 614, in install_item
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 804, in install_eggs
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 1081, in build_and_install
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 1070, in run_setup
  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 29, in run_setup
  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 70, in run
  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 31, in <lambda>
  File "setup.py", line 21, in <module>
    except ImportError:
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "/private/var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/easy_install-NTCEXF/stsci.distutils-0.2dev-r13055/d2to1-0.1.5-py2.6.egg/d2to1/core.py", line 111, in run_command
AttributeError: 'NoneType' object has no attribute 'info'

[uh2macastro04:~/tmp/trunk] $ python
Python 2.6.5 (r265:79359, Mar 24 2010, 01:32:55) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin

-----------------------------------

Martin Raue

University of Hamburg
Institute for Experimental Physics
Luruper Chaussee 149
D-22761 Hamburg
Germany

Phone  : +49-40-8998-5216
Fax    : +49-40-8998-2170
E-Mail : martin.raue at desy.de





From moritz.guenther at gmx.de  Tue Jun 14 08:43:01 2011
From: moritz.guenther at gmx.de (Hans Moritz Guenther)
Date: Tue, 14 Jun 2011 08:43:01 -0400
Subject: [AstroPy] Format of postings
Message-ID: <4DF75755.9090201@gmx.de>

Hi,

I have a suggestion for the format of postings:
When you reply to a message on this list, please do not quote everything 
that has ever been said on that thread. It makes it hard to read in the 
"Digest" mode.

The last digest for this list had > 1000 lines for only 2 messages and 
reached indentation level 9 of quoted emails. It took me some time to 
find out where message number 2 started.

If people join the list an do not know what happened before, all the 
older postings are available in the archive:

http://mail.scipy.org/pipermail/astropy/

Hans Moritz G?nther



From perry at stsci.edu  Tue Jun 14 08:52:38 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 14 Jun 2011 08:52:38 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF70BA8.9060505@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>
	<4DF64C59.3010501@gemini.edu>	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>	<4DF678BF.3070505@gemini.edu>	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>	<BANLkTikPYT7zZvB9vVwz3pjkgZyXgtU84w@mail.gmail.com>
	<2D017079118B4CF094C59CCADADCDFFD@gmail.com>
	<4DF70BA8.9060505@gmail.com>
Message-ID: <EF109595-B7C4-41B6-9D79-08BB42111DA0@stsci.edu>


On Jun 14, 2011, at 3:20 AM, Johann Cohen-Tanugi wrote:

> hi
>
> On 06/14/2011 01:26 AM, Thomas Robitaille wrote:
>> I'd like to suggest that we start figuring out who is interested in  
>> contributing to a common astronomy package, and what you are most  
>> interested in. To get things moving, I created a simple wiki page  
>> here:
>>
>> http://astropy.wikispaces.com/developers
> thanks for doing this, I just updated it.
>> Please add your name to register interest, and what areas you are  
>> interested in (feel free to add areas). This will make it easier to  
>> judge interest for face-to-face meetings and for coordinating  
>> working groups to focus on various areas. I picked the wiki host at  
>> random, but of course we can move anytime if you have a better  
>> solution (for example if we use GitHub for development, we can use  
>> the project wiki on there).
> I do believe we should agree on such a platform fairly quickly, and I
> also feel that all indicators from other communities that we tend to
> also belong to (numpy, scipy, ipython, matplotlib?) are pointing  
> toward
> github as being a de facto natural choice.
>> Note, should we create a new mailing list, say astropy-dev for  
>> people interested in actual development, as opposed to people who  
>> just want to use stuff and have questions?
> I think it is too early for that. We still want to reach potentially
> everyone....
>
I agree on this last point. Often there ends up being a lot of cross  
posting anyway. I don't think the traffic has gotten to the level that  
this needs to be contemplated. When the traffic is a problem, then we  
can always do it then.

Perry



From crawford at saao.ac.za  Tue Jun 14 08:55:45 2011
From: crawford at saao.ac.za (Steve Crawford)
Date: Tue, 14 Jun 2011 14:55:45 +0200 (SAST)
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4CC3F8C3-7228-429B-BB8E-8146A97B3761@mso.anu.edu.au>
Message-ID: <27b7a71e-9ea6-4465-9fd2-fa834c2d3116@mailcape>



Hi everyone,

I would be very keen to participate in the discussion and I've added my information to the contact page.   I haven't been very involved with the community so far, but I would like to get more involved and stop working away at the fringes.  I'd be very keen on participating in this discussion and if I'm able to, I'd hope to attend (either in person or via videocon) and the northern Fall would be perfect. 

Just to give anyone interested a little background information about myself and what I'm working on, I'm the lead developer for PySALT ( http://pysalt.salt.ac.za/) which is a python/pyraf package for data reduction of SALT data.  The next version will be released shortly, although it has gotten held up by actually getting the telescope on sky and working.   The next version will include packages for basic data reduction, long slit spectroscopy, Fabry-Perot, and high-speed photometry.  Although PySALT uses some IRAF tasks, I've been slowly replacing the functionality for some of the basic reduction routines as much as I can with pure python.  Unfortunately, our resources are very limited so I'm very supportive of this idea of not replicating the work!

One other thing I've developed as part of this is the PySpectrograph package (http://code.google.com/p/pyspectrograph/), which can provide a general model for a spectrograph with a single dispersive element and provide functionality for general analysis of spectroscopic data.

Even though we are small, the South African community is very active in python development with major functions of both SALT and MeerKAT having python as the back end and with python being taught to the students coming through our main programs.   

Cheers,
Steve Crawford


From rose.a.finn at gmail.com  Tue Jun 14 10:00:48 2011
From: rose.a.finn at gmail.com (Rose Finn)
Date: Tue, 14 Jun 2011 10:00:48 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
Message-ID: <BANLkTinPktfpj4eQ70H8B+VyQMWnjLRO2A@mail.gmail.com>

Hi everyone,

I use python for almost all of my programming and analysis, but I am far
from an expert programmer.  Like Kelle, I would be happy to provide a user's
perspective, and my vote if for a fall meeting at CfA - purely selfish
reasons as CfA is w/in driving distance for me!

As a user, I can adopt packages more easily when I have access to a few
examples of how to use it.  Perhaps users can help contribute by providing
these examples for packages that they already use.

Take care,
Rose

On Mon, Jun 13, 2011 at 5:14 PM, Thomas Robitaille <
thomas.robitaille at gmail.com> wrote:

> Just to put the suggestion out there, Tom A. and I would be happy to try
> and host the meeting at CfA if there was interest for this, preferably in
> the fall. We have webcast capabilities for the talks (I like the idea of
> 'lightning' talks), though it would be harder to have remote participation
> for the hands-on part.
>
> Of course, we're also open to other locations, so if anyone else interested
> in hosting this, please feel free to suggest!
>
> Cheers,
> Tom
>
> On Monday, June 13, 2011 at 4:53 PM, James Turner wrote:
>
> > > I agree with the idea of a workshop! (but with few talks, and
> > > discussing/deciding/planning/coding would be the majority). I think
> > > face to face would be much better than teleconference. I also like
> > > the idea of a dedicated workshop, not a splinter.
> >
> > We could do both (a face-to-face workshop with video connections or
> > whatever to whomever else wants to dial in; I wouldn't want to shut
> > out students or people in far-flung continents).
> >
> > I agree that we'd keep the talks brief, just for context (1 day?)
> > and focus on things like how to contribute, actually doing so and
> > organization.
> >
> > > I suggest that those of us interested in a small workshop see what
> > > would be possible in our own institutions. I think in the end we
> > > should keep this focused on the development (rather than a general
> > > python in astronomy conference). Keeping it small will also make it
> > > easier to organize and manage.
> >
> > Chile would be nice but is probably a bit too far for people(?). As
> > someone pointed out, Baltimore and Harvard seem relatively accessible
> > for people from the US and Europe.
> >
> > Cheers,
> >
> > James.
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 
Rose A. Finn, PhD
Department of Physics & Astronomy
Siena College
Loudonville, NY
(518) 782-6764
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110614/a0812789/attachment.html>

From erik.tollerud at gmail.com  Tue Jun 14 11:20:26 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Tue, 14 Jun 2011 11:20:26 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
Message-ID: <BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>

I completely agree that an in-person meeting is probably the only way
to really push this along, and webcasting/conferencing would help even
more.

So it sounds like the following options are plausible:

*Splinter meeting at AAS in Austin this January
*Fall Workshop at Space Telescope
*Fall Workshop at CfA/Harvard
*Scipy session next month

Personally, I'm rather strongly inclined against SciPy given that I
(and I suspect others who might attend) wouldn't be able to re-arrange
travel this quickly. I would think AAS might be a bit better given
that many people might be there anyway, although it might be easier to
run if it's a separate workshop... But the best thing to do is to
simply get a vote by interested parties as to which they might be able
to attend.

Are there any options other than the 4 I've listed above?  If not,
then Thomas, perhaps you can pot a poll on the wikispaces site for
which is the preferred option (wikispaces has a surveymonkey widget
you can embed into the page)?


On Mon, Jun 13, 2011 at 5:25 PM, Perry Greenfield <perry at stsci.edu> wrote:
> I think we could probably host as well (I have to check the
> logistics). Fall is probably good for us as well but we could probably
> do it earlier if there was interest in doing it sooner than fall.
>
> Perry
>
> On Jun 13, 2011, at 5:14 PM, Thomas Robitaille wrote:
>
>> Just to put the suggestion out there, Tom A. and I would be happy to
>> try and host the meeting at CfA if there was interest for this,
>> preferably in the fall. We have webcast capabilities for the talks
>> (I like the idea of 'lightning' talks), though it would be harder to
>> have remote participation for the hands-on part.
>>
>> Of course, we're also open to other locations, so if anyone else
>> interested in hosting this, please feel free to suggest!
>>
>> Cheers,
>> Tom
>>
>> On Monday, June 13, 2011 at 4:53 PM, James Turner wrote:
>>
>>>> I agree with the idea of a workshop! (but with few talks, and
>>>> discussing/deciding/planning/coding would be the majority). I think
>>>> face to face would be much better than teleconference. I also like
>>>> the idea of a dedicated workshop, not a splinter.
>>>
>>> We could do both (a face-to-face workshop with video connections or
>>> whatever to whomever else wants to dial in; I wouldn't want to shut
>>> out students or people in far-flung continents).
>>>
>>> I agree that we'd keep the talks brief, just for context (1 day?)
>>> and focus on things like how to contribute, actually doing so and
>>> organization.
>>>
>>>> I suggest that those of us interested in a small workshop see what
>>>> would be possible in our own institutions. I think in the end we
>>>> should keep this focused on the development (rather than a general
>>>> python in astronomy conference). Keeping it small will also make it
>>>> easier to organize and manage.
>>>
>>> Chile would be nice but is probably a bit too far for people(?). As
>>> someone pointed out, Baltimore and Harvard seem relatively accessible
>>> for people from the US and Europe.
>>>
>>> Cheers,
>>>
>>> James.
>>
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 
Erik Tollerud


From mhaas at stsci.edu  Tue Jun 14 11:30:02 2011
From: mhaas at stsci.edu (Marcel Haas)
Date: Tue, 14 Jun 2011 15:30:02 +0000
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
Message-ID: <CA1CF5DF.16E1%mhaas@stsci.edu>

Hi all,



With so many emails going around anyway, I thought one more wouldn't hurt.
I'm a postdoc at STScI and use Python for ~90% of my work. I am not
developing any major useful package at the moment, but I am very willing
to step in to any of them if help is desired.

I will indicate stuff I'm interested in on the wiki. As for the meeting in
person, for me 
>*Splinter meeting at AAS in Austin this January
>*Fall Workshop at Space Telescope

would be best. I agree that a meeting in person is better than just only
email (with teleconferences as a nice in-between).



Cheers, Marcel



-- 
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Marcel R. Haas
Postdoc, Astrophysics
Space Telescope Science Institute, Baltimore, MD

http://www.marcelhaas.com <http://www.marcelhaas.com/>
mhaas at stsci.edu

Address: STScI
         3700 San Martin Drive
         Baltimore, MD, 21218
         Office G17E, tel +1 (410) 338 5012
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++






From mperrin at stsci.edu  Tue Jun 14 12:33:48 2011
From: mperrin at stsci.edu (Marshall Perrin)
Date: Tue, 14 Jun 2011 16:33:48 +0000
Subject: [AstroPy] organizing meetings to organize astropy
In-Reply-To: <BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
Message-ID: <BF25BF12-D7FD-41CA-AAC5-A38E0DA07C5C@stsci.edu>

On Jun 14, 2011, at 11:20 AM, Erik Tollerud wrote:
> I completely agree that an in-person meeting is probably the only way
> to really push this along, and webcasting/conferencing would help even
> more.
> 
> So it sounds like the following options are plausible:
> 
> *Splinter meeting at AAS in Austin this January
> *Fall Workshop at Space Telescope
> *Fall Workshop at CfA/Harvard
> *Scipy session next month

I would suggest more than one of the above be pursued. Given that this organizational debate has come up repeatedly for some years now, there's no way that one workshop over a couple days will possibly suffice for all the worthy topics and issues to sort out. 

The various options above have different strengths, too. A splinter before or after the AAS might be the easiest way to get many people in the same room at once, but a workshop at the CfA or STScI will be easier to webcast for remote attendees. (My experience running splinter workshops at the AAS in the past has uniformly been that the bandwidth is abysmal and in no way sufficient for a webcast).

 - Marshall

From jturner at gemini.edu  Tue Jun 14 13:55:03 2011
From: jturner at gemini.edu (James Turner)
Date: Tue, 14 Jun 2011 13:55:03 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
Message-ID: <4DF7A077.7050408@gemini.edu>

> One thought that came up (this is *only* a suggestion) is should we have
> a basic python package called astropy (when we have agreed on the basic
> packages for it)? If we do this we could use a similar scheme to Scipy's
> Scikits (call it astrokits?). This would ensure that the modules would
> use core astronomy modules. Again, this is just a exploratory idea and
> I would be happy to hear what others have to say.

I believe that was basically the idea of Astrolib -- a place to
collect key modules that others in the community would use. Do you
prefer the AstroPy name, or a different organization (or management)
from Astrolib? Or maybe you're suggesting something new to avoid
endorsing one existing effort over another (since there are other
libraries doing similar things now)? I'm just wondering what needs
doing differently. I know documentation was an issue last year --
STScI did set up a Sphinx server, but its presentation is currently
a bit rough around the edges. I certainly don't want to stall any
discussion or ideas by pushing for one particular thing -- this is
just a genuine question regarding what is still needed and why.

Making it a Scikit might not be a bad idea. Presumably that would
mean adopting SciPy coding standards etc.? We'd then need people
(and/or a BDFL) to buy into those.

Cheers,

James.


From thomas.robitaille at gmail.com  Tue Jun 14 14:20:26 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 14 Jun 2011 14:20:26 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <4DF7A077.7050408@gemini.edu>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
Message-ID: <BA702C32DF5249DA835206A0EE4AEE68@gmail.com>

> I believe that was basically the idea of Astrolib -- a place to
> collect key modules that others in the community would use. Do you
> prefer the AstroPy name, or a different organization (or management)
> from Astrolib? Or maybe you're suggesting something new to avoid
> endorsing one existing effort over another (since there are other
> libraries doing similar things now)? I'm just wondering what needs
> doing differently. I know documentation was an issue last year --
> STScI did set up a Sphinx server, but its presentation is currently
> a bit rough around the edges. I certainly don't want to stall any
> discussion or ideas by pushing for one particular thing -- this is
> just a genuine question regarding what is still needed and why.
Maybe this is controversial, but I think that no single institution should host or control the package (otherwise it would be as if we are working for free for that institution, who would ultimately get the credit). We should just be a collection of individuals, some working in our spare time, some funded to develop software, working towards a common open source project. By using a distributed versioning system (e.g. git or hg) and a free hosting solution (neutral territory, e.g. github or mercurial) we can ensure that the project will live beyond the funding for any given institution, and the distributed aspect means that if the hosting solution goes out of business, moving to another will be easy. Of course it would still be possible for institutions to support development of that package.

This is akin to the relationship between Enthought and Numpy/Scipy - they help fund the development, but even though the primary Numpy developer is now the Enthought president, the development is not hosted or controlled by Enthought. So I would vote for an institution-independent package on GitHub. But of course this will only work if it is possible e.g. for STScI or Gemini developers to move the packages to a repository not at their institutions, so there are licensing/copyright/funding/political issues to consider (although of course with git, one can have a full local repository). Maybe the STScI and Gemini people on here could weigh in on this? For example, would there be barriers to moving the astrolib packages into a new repository on e.g. GitHub?

The name astropy has a good ring to it, is in line with numpy and scipy, and would make it clear this does not endorse specifically one of the existing packages. I've reserved the astropy organization on GitHub in case we decide to go down that road.

Cheers,
Tom




From tgrav at mac.com  Tue Jun 14 14:27:43 2011
From: tgrav at mac.com (Tommy Grav)
Date: Tue, 14 Jun 2011 14:27:43 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
Message-ID: <EDE9C53F-D3FC-414D-BADF-7C408990CCD5@mac.com>


On Jun 14, 2011, at 2:20 PM, Thomas Robitaille wrote:

>> I believe that was basically the idea of Astrolib -- a place to
>> collect key modules that others in the community would use. Do you
>> prefer the AstroPy name, or a different organization (or management)
>> from Astrolib? Or maybe you're suggesting something new to avoid
>> endorsing one existing effort over another (since there are other
>> libraries doing similar things now)? I'm just wondering what needs
>> doing differently. I know documentation was an issue last year --
>> STScI did set up a Sphinx server, but its presentation is currently
>> a bit rough around the edges. I certainly don't want to stall any
>> discussion or ideas by pushing for one particular thing -- this is
>> just a genuine question regarding what is still needed and why.
> Maybe this is controversial, but I think that no single institution should host or control the package (otherwise it would be as if we are working for free for that institution, who would ultimately get the credit). We should just be a collection of individuals, some working in our spare time, some funded to develop software, working towards a common open source project. By using a distributed versioning system (e.g. git or hg) and a free hosting solution (neutral territory, e.g. github or mercurial) we can ensure that the project will live beyond the funding for any given institution, and the distributed aspect means that if the hosting solution goes out of business, moving to another will be easy. Of course it would still be possible for institutions to support development of that package.

I also think there needs to be a certain control by certain key contributing 
people. One problem seems to be that when people just develop a package
and contribute it to something like Astrolib there is no continuity between
the package and the use of them. I think the strength of numpy is that 
is sprung from the work of a few people and any new developer has to 
adopt a certain style of coding that maintains the package overall consistency.
Scipy seems to be a little looser, but still has some central control by a
small group of developers. I think astropy/astrolib needs something similar,
making it a package in itself, not just a collection of packages gathered from
a wide variety of sources. There has to be a certain focus to make it successful
I think. 

Cheers
 Tommy



From matthewturk at gmail.com  Tue Jun 14 14:34:30 2011
From: matthewturk at gmail.com (Matthew Turk)
Date: Tue, 14 Jun 2011 11:34:30 -0700
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
Message-ID: <BANLkTin_xyLvD=FUXDEV2TLZSAnaAr8Kig@mail.gmail.com>

Hi Tom,

On Tue, Jun 14, 2011 at 11:20 AM, Thomas Robitaille
<thomas.robitaille at gmail.com> wrote:
>> I believe that was basically the idea of Astrolib -- a place to
>> collect key modules that others in the community would use. Do you
>> prefer the AstroPy name, or a different organization (or management)
>> from Astrolib? Or maybe you're suggesting something new to avoid
>> endorsing one existing effort over another (since there are other
>> libraries doing similar things now)? I'm just wondering what needs
>> doing differently. I know documentation was an issue last year --
>> STScI did set up a Sphinx server, but its presentation is currently
>> a bit rough around the edges. I certainly don't want to stall any
>> discussion or ideas by pushing for one particular thing -- this is
>> just a genuine question regarding what is still needed and why.
> Maybe this is controversial, but I think that no single institution should host or control the package (otherwise it would be as if we are working for free for that institution, who would ultimately get the credit). We should just be a collection of individuals, some working in our spare time, some funded to develop software, working towards a common open source project. By using a distributed versioning system (e.g. git or hg) and a free hosting solution (neutral territory, e.g. github or mercurial) we can ensure that the project will live beyond the funding for any given institution, and the distributed aspect means that if the hosting solution goes out of business, moving to another will be easy. Of course it would still be possible for institutions to support development of that package.

With the Enzo (astrophysical simulation) community we found this 100%
to be the case.  We moved from SVN to mercurial hosted on Google Code
(which does allow for forking!) mirrored on BitBucket, in a specific
attempt to move it off the site of a given institution.  Moving to a
neutral location and toward a community-driven development model has
resulted in a huge number of improvements and a great expansion of the
developer and user bases.

For what it's worth, in our community we have a large buy-in for
mercurial (with a number of extensions and value-adds that directly
call the API).  I recognize that the observer and simulation
communities do not necessarily need as much direct cross-talk as
intra-community collaborations, but I felt I should put this out
there.

-Matt

>
> This is akin to the relationship between Enthought and Numpy/Scipy - they help fund the development, but even though the primary Numpy developer is now the Enthought president, the development is not hosted or controlled by Enthought. So I would vote for an institution-independent package on GitHub. But of course this will only work if it is possible e.g. for STScI or Gemini developers to move the packages to a repository not at their institutions, so there are licensing/copyright/funding/political issues to consider (although of course with git, one can have a full local repository). Maybe the STScI and Gemini people on here could weigh in on this? For example, would there be barriers to moving the astrolib packages into a new repository on e.g. GitHub?

Whether or not Enthought controls the development, they do drive a
substantial amount of it -- such as the datetime dtype, the Cython
refactoring, and the .NET wrapping.  One should expect such de facto
relationships to grow during the development of an astrophysical
python library, and it's not such a bad thing.

-Matt

>
> The name astropy has a good ring to it, is in line with numpy and scipy, and would make it clear this does not endorse specifically one of the existing packages. I've reserved the astropy organization on GitHub in case we decide to go down that road.
>
> Cheers,
> Tom
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From jturner at gemini.edu  Tue Jun 14 14:42:49 2011
From: jturner at gemini.edu (James Turner)
Date: Tue, 14 Jun 2011 14:42:49 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
Message-ID: <4DF7ABA9.5010905@gemini.edu>

Thanks for the candid remarks. It seems like a good time for Perry
to comment :-). I do think the solution needs community buy-in, so
this kind of discussion is important.

I believe the AstroPy list belongs to Paul Barrett and I'd be
interested to know what he thinks about naming stuff after it.

James.


PS. Regarding the Astrolib repository, I know there have been some
practical reasons for moving it a couple of times and now it is
hosted on a paid Sourceforge-like site (not sure it does git). I
don't know whether that counts as a neutral location.


On 14/06/11 14:20, Thomas Robitaille wrote:
>> I believe that was basically the idea of Astrolib -- a place to
>> collect key modules that others in the community would use. Do you
>> prefer the AstroPy name, or a different organization (or management)
>> from Astrolib? Or maybe you're suggesting something new to avoid
>> endorsing one existing effort over another (since there are other
>> libraries doing similar things now)? I'm just wondering what needs
>> doing differently. I know documentation was an issue last year --
>> STScI did set up a Sphinx server, but its presentation is currently
>> a bit rough around the edges. I certainly don't want to stall any
>> discussion or ideas by pushing for one particular thing -- this is
>> just a genuine question regarding what is still needed and why.
> Maybe this is controversial, but I think that no single institution should host or control the package (otherwise it would be as if we are working for free for that institution, who would ultimately get the credit). We should just be a collection of individuals, some working in our spare time, some funded to develop software, working towards a common open source project. By using a distributed versioning system (e.g. git or hg) and a free hosting solution (neutral territory, e.g. github or mercurial) we can ensure that the project will live beyond the funding for any given institution, and the distributed aspect means that if the hosting solution goes out of business, moving to another will be easy. Of course it would still be possible for institutions to support development of that package.
>
> This is akin to the relationship between Enthought and Numpy/Scipy - they help fund the development, but even though the primary Numpy developer is now the Enthought president, the development is not hosted or controlled by Enthought. So I would vote for an institution-independent package on GitHub. But of course this will only work if it is possible e.g. for STScI or Gemini developers to move the packages to a repository not at their institutions, so there are licensing/copyright/funding/political issues to consider (although of course with git, one can have a full local repository). Maybe the STScI and Gemini people on here could weigh in on this? For example, would there be barriers to moving the astrolib packages into a new repository on e.g. GitHub?
>
> The name astropy has a good ring to it, is in line with numpy and scipy, and would make it clear this does not endorse specifically one of the existing packages. I've reserved the astropy organization on GitHub in case we decide to go down that road.
>
> Cheers,
> Tom


From thomas.robitaille at gmail.com  Tue Jun 14 15:04:17 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 14 Jun 2011 15:04:17 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <BANLkTin_xyLvD=FUXDEV2TLZSAnaAr8Kig@mail.gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<BANLkTin_xyLvD=FUXDEV2TLZSAnaAr8Kig@mail.gmail.com>
Message-ID: <B2F07E60CA184B5BB2EC09E3AACEE9EB@gmail.com>

> Whether or not Enthought controls the development, they do drive a
> substantial amount of it -- such as the datetime dtype, the Cython
> refactoring, and the .NET wrapping. One should expect such de facto
> relationships to grow during the development of an astrophysical
> python library, and it's not such a bad thing.
Absolutely! In the end, a fair fraction of the development would come from institutions such as STScI, and this is clearly something that would have to be acknowledged alongside the contributions from individual developers, as for Enthought/Scipy.

Cheers,
Tom
> 
> -Matt
> 
> > 
> > The name astropy has a good ring to it, is in line with numpy and scipy, and would make it clear this does not endorse specifically one of the existing packages. I've reserved the astropy organization on GitHub in case we decide to go down that road.
> > 
> > Cheers,
> > Tom
> > 
> > 
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy




From sienkiew at stsci.edu  Tue Jun 14 15:37:24 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Tue, 14 Jun 2011 15:37:24 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>
	<4DF64C59.3010501@gemini.edu>	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>	<4DF678BF.3070505@gemini.edu>	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
Message-ID: <4DF7B874.80103@stsci.edu>

Erik Tollerud wrote:
> I completely agree that an in-person meeting is probably the only way
> to really push this along, and webcasting/conferencing would help even
> more.
>   

You should do it if that's what it takes...

... but I do need to point out that I've seen many projects like this 
organized and conducted entirely by email.  You do not need to wait for 
the meeting to start organizing things.  You can communicate faster by 
speaking in person than by typing email, but you have to include the lag 
time in your calculation -- if it takes you 3 weeks to get it together 
via email, you can have a lot of discussion before you even start a 
meeting that is scheduled for September.

What single person or small group will be the organizer?  Not BDFL, or 
even BDFN (.. for now), but somebody to guide the discussion, take 
notes, document the current state of the decisions so far, etc.

Any volunteers?

If we have more than one, shall we hold an election?


( This is **important** - there is a lot of useful discussion happening 
on the list right now, but it will all be lost if we don't formally make 
decisions and accept those decisions into the project. )




From perry at stsci.edu  Tue Jun 14 15:51:22 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 14 Jun 2011 15:51:22 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
Message-ID: <28E322A1-C259-4F50-B4D3-E928D063AAB7@stsci.edu>

Long afternoon meetings...

Doesn't seem that controversial to me.

On Jun 14, 2011, at 2:20 PM, Thomas Robitaille wrote:
>>
> Maybe this is controversial, but I think that no single institution  
> should host or control the package (otherwise it would be as if we  
> are working for free for that institution, who would ultimately get  
> the credit). We should just be a collection of individuals, some  
> working in our spare time, some funded to develop software, working  
> towards a common open source project. By using a distributed  
> versioning system (e.g. git or hg) and a free hosting solution  
> (neutral territory, e.g. github or mercurial) we can ensure that the  
> project will live beyond the funding for any given institution, and  
> the distributed aspect means that if the hosting solution goes out  
> of business, moving to another will be easy. Of course it would  
> still be possible for institutions to support development of that  
> package.
>
I understand the sentiment but, a few comments.

> This is akin to the relationship between Enthought and Numpy/Scipy -  
> they help fund the development, but even though the primary Numpy  
> developer is now the Enthought president, the development is not  
> hosted or controlled by Enthought. So I would vote for an  
> institution-independent package on GitHub. But of course this will  
> only work if it is possible e.g. for STScI or Gemini developers to  
> move the packages to a repository not at their institutions, so  
> there are licensing/copyright/funding/political issues to consider  
> (although of course with git, one can have a full local repository).  
> Maybe the STScI and Gemini people on here could weigh in on this?  
> For example, would there be barriers to moving the astrolib packages  
> into a new repository on e.g. GitHub?
>
I don't think this is quite true. From what I understand, Enthought  
does host the source code repositories for both numpy and scipy  
(scipy.org servers belong to Enthought). Why, they host this mailing  
list ;-).

It is true that we fund the astrolib repository, and that it currently  
isn't a free hosting, it is hosted off of STScI machines. And it is a  
generic SVN repository which could easily be relocated should that  
commercial hosting service disappear. (We regularly backup the  
repository locally). And astrolib.org is a generic url with no  
intrinsic STScI link. Doing both of these was to try to avoid the  
institutional ownership issue that you raise.

But we will use whatever repository system people will use the most.  
If that would be git, that would be fine with us (or me anyway :-)  
It's not clear yet to me if that is git (votes?).

Perry



From perry at stsci.edu  Tue Jun 14 15:57:31 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 14 Jun 2011 15:57:31 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <EDE9C53F-D3FC-414D-BADF-7C408990CCD5@mac.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<EDE9C53F-D3FC-414D-BADF-7C408990CCD5@mac.com>
Message-ID: <CC90342B-B2C6-4282-9F94-499324AE227C@stsci.edu>


On Jun 14, 2011, at 2:27 PM, Tommy Grav wrote:
>>
>
> I also think there needs to be a certain control by certain key  
> contributing
> people. One problem seems to be that when people just develop a  
> package
> and contribute it to something like Astrolib there is no continuity  
> between
> the package and the use of them. I think the strength of numpy is that
> is sprung from the work of a few people and any new developer has to
> adopt a certain style of coding that maintains the package overall  
> consistency.
> Scipy seems to be a little looser, but still has some central  
> control by a
> small group of developers. I think astropy/astrolib needs something  
> similar,
> making it a package in itself, not just a collection of packages  
> gathered from
> a wide variety of sources. There has to be a certain focus to make  
> it successful
> I think.
>
> Cheers
> Tommy

You must remember that it was easier with these two projects,  
particularly numpy. numpy is essentially a pretty clear integrated  
whole with a series of previous owner from which it was derived. At  
one point it was developed primarily by Jim Hugunin, then we did  
numarray, then Travis essentially merged the two.

Not quite so simple for scipy, but scipy started out mostly as gluing  
together existing libraries. As such, there weren't a lot of  
architectural decisions to make about interfaces (and I think some  
complain about the lack of consistency still as a result). These  
involve many, mostly mature, libraries that had their own primary  
developers (and still do for the most part).

In this case, we are talking about remaking much of the astronomical  
software from scratch with many different kinds of potential users. So  
I think in many respects, this is much more difficult than either  
numpy or scipy had to deal with in organizational terms.

Perry



From perry at stsci.edu  Tue Jun 14 16:02:24 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 14 Jun 2011 16:02:24 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <4DF7ABA9.5010905@gemini.edu>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<4DF7ABA9.5010905@gemini.edu>
Message-ID: <1A4242C8-F0B5-4517-ABDB-5AC4E28630CB@stsci.edu>


On Jun 14, 2011, at 2:42 PM, James Turner wrote:

> Thanks for the candid remarks. It seems like a good time for Perry
> to comment :-). I do think the solution needs community buy-in, so
> this kind of discussion is important.
>
> I believe the AstroPy list belongs to Paul Barrett and I'd be
> interested to know what he thinks about naming stuff after it.
>
> James.
>
>
> PS. Regarding the Astrolib repository, I know there have been some
> practical reasons for moving it a couple of times and now it is
> hosted on a paid Sourceforge-like site (not sure it does git). I
> don't know whether that counts as a neutral location.
>

I don't know if Paul thinks he owns it (I thinks he reads this list so  
he can jump in here). I think it started at Goddard, but was migrated  
to scipy.org years ago when he was here at STScI. STScI has  
essentially been managing the mailing list since then. I don't think  
anyone has really claimed ownership of that term or astrolib (we  
don't). I think either should be controlled by any one institution.

And just looking, it does seem assembla (where we are hosting the svn  
repository) does support git now. One reason we put it there was that  
if you are paying for it, you have some legal recourse with the  
hosting organization, which you don't nearly as much with the free ones.

Perry



From tgrav at mac.com  Tue Jun 14 16:17:11 2011
From: tgrav at mac.com (Tommy Grav)
Date: Tue, 14 Jun 2011 16:17:11 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <CC90342B-B2C6-4282-9F94-499324AE227C@stsci.edu>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<EDE9C53F-D3FC-414D-BADF-7C408990CCD5@mac.com>
	<CC90342B-B2C6-4282-9F94-499324AE227C@stsci.edu>
Message-ID: <593B1B13-6B2A-4EC2-84C4-4337CBEB21BF@mac.com>


On Jun 14, 2011, at 3:57 PM, Perry Greenfield wrote:

> You must remember that it was easier with these two projects, particularly numpy. numpy is essentially a pretty clear integrated whole with a series of previous owner from which it was derived. At one point it was developed primarily by Jim Hugunin, then we did numarray, then Travis essentially merged the two.
> 
> Not quite so simple for scipy, but scipy started out mostly as gluing together existing libraries. As such, there weren't a lot of architectural decisions to make about interfaces (and I think some complain about the lack of consistency still as a result). These involve many, mostly mature, libraries that had their own primary developers (and still do for the most part).
> 
> In this case, we are talking about remaking much of the astronomical software from scratch with many different kinds of potential users. So I think in many respects, this is much more difficult than either numpy or scipy had to deal with in organizational terms.

No question it is more difficult, which I think is one of the major reasons current efforts have
not managed to gain more traction in the community. I think some general guidelines before
packages are accepted into the library is needed, and that takes some leadership by the most
dedicated of the developers (and also the bravery to say no to packages that have a coding
style that clashes to much with the main part of the package). But no question this is going
to be a challenge and lots of work and time to get to the point where numpy and scipy are 
today.

Cheers
  Tommy 


From William.T.Bridgman at nasa.gov  Tue Jun 14 16:20:17 2011
From: William.T.Bridgman at nasa.gov (Bridgman, William T.)
Date: Tue, 14 Jun 2011 16:20:17 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <BANLkTin_xyLvD=FUXDEV2TLZSAnaAr8Kig@mail.gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<BANLkTin_xyLvD=FUXDEV2TLZSAnaAr8Kig@mail.gmail.com>
Message-ID: <54DF7D50-EC7E-4196-B02A-974DD965644B@nasa.gov>

I've been following this while trying to find out how my NASA  
contractor status might impact how I can contribute to this effort.

However, I'd like to make the point that for a lot of modern  
visualization efforts that we do at the SVS for general public  
outreach, observations and simulations can be integrated into a single  
viz more easily if the tools can play nice together.  I have a strong  
interest in this area of the development.

Here's some items where I had to integrate very diverse datasets to  
generate the final product:

http://svs.gsfc.nasa.gov/goto?3316
http://svs.gsfc.nasa.gov/goto?3743
http://svs.gsfc.nasa.gov/search/Series/SolarDynamo.html

I used numpy, scipy, & pyfits for much of it, but also had to convert  
datasets to objects & texture with tools like VTK & PIL.

If there is a way to have the simulation and observation tools playing  
nice together, it would simplify outreach (and consequently funding)  
efforts.

Tom

On Jun 14, 2011, at 2:34 PM, Matthew Turk wrote:

>
> For what it's worth, in our community we have a large buy-in for
> mercurial (with a number of extensions and value-adds that directly
> call the API).  I recognize that the observer and simulation
> communities do not necessarily need as much direct cross-talk as
> intra-community collaborations, but I felt I should put this out
> there.
>
> -Matt

--
Dr. William T."Tom" Bridgman               Scientific Visualization  
Studio
Global Science & Technology, Inc.          NASA/Goddard Space Flight  
Center
Email: William.T.Bridgman at nasa.gov         Code 610.3
Phone: 301-286-1346                        Greenbelt, MD 20771
FAX:   301-286-1634                        http://svs.gsfc.nasa.gov/




-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 16749 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20110614/ae198c53/attachment.bin>

From perry at stsci.edu  Tue Jun 14 16:29:52 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 14 Jun 2011 16:29:52 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <28E322A1-C259-4F50-B4D3-E928D063AAB7@stsci.edu>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<28E322A1-C259-4F50-B4D3-E928D063AAB7@stsci.edu>
Message-ID: <101E487C-444E-4438-B715-B66D920D9831@stsci.edu>

Some corrections pointe out to me

On Jun 14, 2011, at 3:51 PM, Perry Greenfield wrote:
>>
>>
> I don't think this is quite true. From what I understand, Enthought  
> does host the source code repositories for both numpy and scipy  
> (scipy.org servers belong to Enthought). Why, they host this mailing  
> list ;-).
>
Well, it was on Enthought servers for a while, but not now, but it  
still appears that they pay for the hosting service.

And as for the astropy mailing list, it actually started at STScI  
apparently.


From perry at stsci.edu  Tue Jun 14 16:32:48 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 14 Jun 2011 16:32:48 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <593B1B13-6B2A-4EC2-84C4-4337CBEB21BF@mac.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<EDE9C53F-D3FC-414D-BADF-7C408990CCD5@mac.com>
	<CC90342B-B2C6-4282-9F94-499324AE227C@stsci.edu>
	<593B1B13-6B2A-4EC2-84C4-4337CBEB21BF@mac.com>
Message-ID: <602094CA-EB83-4C75-97D5-7107FB479BC1@stsci.edu>

I wonder if the right approach is that for us a BDFL mainly ensures  
that all the various projects have consistency of interfaces, approach  
and such, but isn't worried about designing everything. That would  
devolve to BDFLs at a second level to take responsibility for more  
specific areas.

On Jun 14, 2011, at 4:17 PM, Tommy Grav wrote:

>
> On Jun 14, 2011, at 3:57 PM, Perry Greenfield wrote:
>
>> You must remember that it was easier with these two projects,  
>> particularly numpy. numpy is essentially a pretty clear integrated  
>> whole with a series of previous owner from which it was derived. At  
>> one point it was developed primarily by Jim Hugunin, then we did  
>> numarray, then Travis essentially merged the two.
>>
>> Not quite so simple for scipy, but scipy started out mostly as  
>> gluing together existing libraries. As such, there weren't a lot of  
>> architectural decisions to make about interfaces (and I think some  
>> complain about the lack of consistency still as a result). These  
>> involve many, mostly mature, libraries that had their own primary  
>> developers (and still do for the most part).
>>
>> In this case, we are talking about remaking much of the  
>> astronomical software from scratch with many different kinds of  
>> potential users. So I think in many respects, this is much more  
>> difficult than either numpy or scipy had to deal with in  
>> organizational terms.
>
> No question it is more difficult, which I think is one of the major  
> reasons current efforts have
> not managed to gain more traction in the community. I think some  
> general guidelines before
> packages are accepted into the library is needed, and that takes  
> some leadership by the most
> dedicated of the developers (and also the bravery to say no to  
> packages that have a coding
> style that clashes to much with the main part of the package). But  
> no question this is going
> to be a challenge and lots of work and time to get to the point  
> where numpy and scipy are
> today.
>
> Cheers
>  Tommy
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From neilcrighton at gmail.com  Tue Jun 14 16:34:44 2011
From: neilcrighton at gmail.com (Neil Crighton)
Date: Tue, 14 Jun 2011 22:34:44 +0200
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <CA1CF5DF.16E1%mhaas@stsci.edu>
References: <BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
	<CA1CF5DF.16E1%mhaas@stsci.edu>
Message-ID: <BANLkTikO1C9HnJf=P6y0M=G9uQJ=ywVyoA@mail.gmail.com>

On 14 June 2011 17:30, Marcel Haas <mhaas at stsci.edu> wrote:
> With so many emails going around anyway, I thought one more wouldn't hurt.
> I'm a postdoc at STScI and use Python for ~90% of my work. I am not
> developing any major useful package at the moment, but I am very willing
> to step in to any of them if help is desired.

Hi,

In similar vein, I'm also happy to contribute. I'm strongly in favour
of having small self-contained packages that try to do one thing well
rather than larger packages that try to encompass many areas.

Much of the python code I've written for my research in a repository
here https://bitbucket.org/nhmc/pyserpens, but I pity anyone who tries
to use it :)

Neil


From jturner at gemini.edu  Tue Jun 14 16:36:55 2011
From: jturner at gemini.edu (James Turner)
Date: Tue, 14 Jun 2011 16:36:55 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <602094CA-EB83-4C75-97D5-7107FB479BC1@stsci.edu>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>	<4DF7A077.7050408@gemini.edu>	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>	<EDE9C53F-D3FC-414D-BADF-7C408990CCD5@mac.com>	<CC90342B-B2C6-4282-9F94-499324AE227C@stsci.edu>	<593B1B13-6B2A-4EC2-84C4-4337CBEB21BF@mac.com>
	<602094CA-EB83-4C75-97D5-7107FB479BC1@stsci.edu>
Message-ID: <4DF7C667.6020000@gemini.edu>

Sounds like Torvalds and the Linux kernel subsystem maintainers.
Obviously that has worked for them, but Linus has always had the
ultimate editorial control.


On 14/06/11 16:32, Perry Greenfield wrote:
> I wonder if the right approach is that for us a BDFL mainly ensures
> that all the various projects have consistency of interfaces, approach
> and such, but isn't worried about designing everything. That would
> devolve to BDFLs at a second level to take responsibility for more
> specific areas.
>
> On Jun 14, 2011, at 4:17 PM, Tommy Grav wrote:
>
>>
>> On Jun 14, 2011, at 3:57 PM, Perry Greenfield wrote:
>>
>>> You must remember that it was easier with these two projects,
>>> particularly numpy. numpy is essentially a pretty clear integrated
>>> whole with a series of previous owner from which it was derived. At
>>> one point it was developed primarily by Jim Hugunin, then we did
>>> numarray, then Travis essentially merged the two.
>>>
>>> Not quite so simple for scipy, but scipy started out mostly as
>>> gluing together existing libraries. As such, there weren't a lot of
>>> architectural decisions to make about interfaces (and I think some
>>> complain about the lack of consistency still as a result). These
>>> involve many, mostly mature, libraries that had their own primary
>>> developers (and still do for the most part).
>>>
>>> In this case, we are talking about remaking much of the
>>> astronomical software from scratch with many different kinds of
>>> potential users. So I think in many respects, this is much more
>>> difficult than either numpy or scipy had to deal with in
>>> organizational terms.
>>
>> No question it is more difficult, which I think is one of the major
>> reasons current efforts have
>> not managed to gain more traction in the community. I think some
>> general guidelines before
>> packages are accepted into the library is needed, and that takes
>> some leadership by the most
>> dedicated of the developers (and also the bravery to say no to
>> packages that have a coding
>> style that clashes to much with the main part of the package). But
>> no question this is going
>> to be a challenge and lots of work and time to get to the point
>> where numpy and scipy are
>> today.
>>
>> Cheers
>>   Tommy
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


From thomas.robitaille at gmail.com  Tue Jun 14 16:42:49 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 14 Jun 2011 16:42:49 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF7B874.80103@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
	<4DF7B874.80103@stsci.edu>
Message-ID: <95F4D6CAC0074BAD9DC6DEE80730067A@gmail.com>

> What single person or small group will be the organizer? Not BDFL, or 
> even BDFN (.. for now), but somebody to guide the discussion, take 
> notes, document the current state of the decisions so far, etc.
> 
> Any volunteers?
> 
> If we have more than one, shall we hold an election?
I agree, and am worried that if we don't then the discussion will die out and come back next year (after all, we just need to look at the July 2010 discussions on the mailing list, and other ones before that...). I propose that anyone interested in taking on the responsibility of moderating the conversation and decision making process declare that they are interested in doing so by Thursday evening on the wiki page. I've added a section "Candidates to temporarily moderate the discussions/decisions" at

http://astropy.wikispaces.com/developers

If there is more than one person by Thursday evening, then we can create a poll and have people vote on Friday. If you are interested in the overall planning but not leading, then add yourself to the overall planning/big picture stuff. People in that category could be part of a kind of 'steering' committee.

I would love to see this project actually happen, however long it takes to get there, and I think we should get things started, so I'll probably add myself as a candidate. I'd be happy to try and keep track of decisions that have been made, organize votes, and try and steer things in the direction we want to go in. But if anyone feels that they would like to do this, then please add your name to the list!

Cheers,
Tom

> 
> 
> ( This is **important** - there is a lot of useful discussion happening 
> on the list right now, but it will all be lost if we don't formally make 
> decisions and accept those decisions into the project. )
> 
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> http://mail.scipy.org/mailman/listinfo/astropy




From laidler at stsci.edu  Tue Jun 14 16:43:11 2011
From: laidler at stsci.edu (Victoria G. Laidler)
Date: Tue, 14 Jun 2011 16:43:11 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <602094CA-EB83-4C75-97D5-7107FB479BC1@stsci.edu>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>	<4DF7A077.7050408@gemini.edu>	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>	<EDE9C53F-D3FC-414D-BADF-7C408990CCD5@mac.com>	<CC90342B-B2C6-4282-9F94-499324AE227C@stsci.edu>	<593B1B13-6B2A-4EC2-84C4-4337CBEB21BF@mac.com>
	<602094CA-EB83-4C75-97D5-7107FB479BC1@stsci.edu>
Message-ID: <4DF7C7DF.9070105@stsci.edu>

Perry Greenfield wrote:
> I wonder if the right approach is that for us a BDFL mainly ensures  
> that all the various projects have consistency of interfaces, approach  
> and such, but isn't worried about designing everything. That would  
> devolve to BDFLs at a second level to take responsibility for more  
> specific areas.
>   
That was in the suggestion Mark posted earlier, which I'm quoting here 
because it never made it into this thread:

Mark Sienkiewicz wrote:
> The most effective approach would require:
>
> - one person willing to coordinate the project -- but maybe as many 5 at 
> most.  (Unless you subdivide the project into independent parts, it 
> doesn't work as well with more.)  Basically, we need a Guido -- he acted 
> as BDFL for python at the request of (and with the consent of) the 
> python community.  He provided valuable organization that could not have 
> existed without somebody working at it.
>
> - some small number of people to act as core developers.  These people 
> design the system and coordinate contributions.  This is also a lot of 
> work, and you're looking for people with software design skills.  Maybe 
> 3 to 10 people.
>
> - a bunch of people willing be contributing developers.  These people 
> contribute subsets.  This is less work than being a core developer, but 
> still a bunch more than hacking together a custom solution for your 
> specific needs and putting it up on your web site.
>
> - users who are interested enough to overcome the learning curve.  Why 
> should I use your simple coordinate transformation when I can write my 
> own faster than I can figure out how to use yours?  But, implement and 
> document enough useful capabilities in your package, and it is worth my 
> time to use them instead of writing my own copy of everything.  
> (Remember:  The user of your package only knows what you tell them -- as 
> far as the user knows, anything that isn't documented doesn't exist.)
>
> This rough model is typical of successful free software projects.  Think 
> of examples like python, perl, linux, GCC, gnuplot, freebsd, or gnome.  
> They all work this way.
>   



From sienkiew at stsci.edu  Tue Jun 14 16:47:57 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Tue, 14 Jun 2011 16:47:57 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
Message-ID: <4DF7C8FD.7090705@stsci.edu>

Thomas Robitaille wrote:
> Maybe this is controversial, but I think that no single institution should host or control the package (otherwise it would be as if we are working for free for that institution, who would ultimately get the credit). We should just be a collection of individuals, some working in our spare time, some funded to develop software, working towards a common open source project.


Where we stand with astrolib right now:  STScI is paying the hosting fee 
to have it stored on assembla.com.  I would have been happy to store it 
on one of our subversion servers here, but we are subject to NASA 
computer security policies that make it awkward for us to grant access 
for people who are not STScI employees.  (With Assembla, I only need 
Perry's approval.)  It is subversion because the previous repository was 
subversion, and we are using subversion for our other work at STScI.

If the community feels like this means that STScI "has control" or "gets 
credit", then it could be stored somewhere else.  That may or may not 
fix anything.  If I create a repository on github, then I have every bit 
as much control as I have if I create a repository on Assembla.

So, for example, if you create a github repository for us to move 
astrolib to, you are basically trading a circumstance where _I_ am in 
control of the repository for one where _you_ are in control of the 
repository.  i.e. No significant change, except for the name of the 
person who has the password needed to grant commit access.

Fortunately for the community, _I_ don't care who has control of the 
repository.  I'm watching over it because Perry said "get us a place to 
put astrolib".  If he says "give it to that guy", that's fine with me.


(Actually, there is one difference:  Assembla has a contractual 
obligation to provide service to me; github/sourceforge/whatever have no 
contractual obligation to you.  When we selected a hosting service, we 
thought about whether there might be a different quality of service 
between "paying customer" and "getting service for free".)


>  By using a distributed versioning system (e.g. git or hg) and a free hosting solution (neutral territory, e.g. github or mercurial) we can ensure that the project will live beyond the funding for any given institution, and the distributed aspect means that if the hosting solution goes out of business, moving to another will be easy.


In fact, a DVCS makes little difference to this case.  You can make your 
own private copy of the astrolib subversion repository at any time.  I 
keep daily backups of all my off-site subversion repositories (astrolib 
is not the only thing I have at Assembla) as part of my disaster 
recovery plan.  I know the backups are valid because my CI system checks 
out code from the backups, so the backups are routinely tested.

If Assembla disappears tomorrow, I still have a copy of everything, that 
I can load into any other hosting service.  If STScI disappears 
tomorrow, you could also have a copy that you could recover from -- all 
you have to do is maintain a local copy, which is the same thing you 
would do with the DVCS.


> For example, would there be barriers to moving the astrolib packages 
> into a new repository on e.g. GitHub?

Yes - if the new repository requires a different VCS, then some people 
will have to learn to use the new VCS.  In the case of a DVCS, there is 
a new work flow that you need to explain to potential developers.  For 
example, I could have to support both svn and git for my developers.

On the plus side, numpy is using git.  If you choose git, then I don't 
need _two_ new VCS.

On the minus side, during my last major software build, I did not 
succeed in compiling git on my linux machines.  But then, the last time 
I tried to compile subverion, it didn't work either.  The dependencies 
of both are nasty.

Mark S



From erik.tollerud at gmail.com  Tue Jun 14 16:50:57 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Tue, 14 Jun 2011 16:50:57 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <95F4D6CAC0074BAD9DC6DEE80730067A@gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
	<4DF7B874.80103@stsci.edu>
	<95F4D6CAC0074BAD9DC6DEE80730067A@gmail.com>
Message-ID: <BANLkTimg=PhAyQ+phbNPLo7P+pAMaWi8DQ@mail.gmail.com>

Actually, I think it might be better to have a smallish group do this
(3-5), assuming enough are willing.   But otherwise, I think the
wikispace-based nomination process makes sense.

I think more than one is better mainly because the BDFL/BDFN model may
not be entirely fitting here.  It works well for Python because Guido
wrote the initial code base himself, so it already had a clear vision.
 In this case, we're talking about combining a number of disparate
approaches and ways of working, so a group with somewhat different
perspectives makes sense.  But as long as it's small enough (3 is
probably ideal), it hopefully will still be able to quickly agree on a
unified approach.


>> Any volunteers?
>>
>> If we have more than one, shall we hold an election?
> I agree, and am worried that if we don't then the discussion will die out and come back next year (after all, we just need to look at the July 2010 discussions on the mailing list, and other ones before that...). I propose that anyone interested in taking on the responsibility of moderating the conversation and decision making process declare that they are interested in doing so by Thursday evening on the wiki page. I've added a section "Candidates to temporarily moderate the discussions/decisions" at
>
> http://astropy.wikispaces.com/developers
>
> If there is more than one person by Thursday evening, then we can create a poll and have people vote on Friday. If you are interested in the overall planning but not leading, then add yourself to the overall planning/big picture stuff. People in that category could be part of a kind of 'steering' committee.


-- 
Erik Tollerud


From thomas.robitaille at gmail.com  Tue Jun 14 16:57:45 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 14 Jun 2011 16:57:45 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <4DF7C8FD.7090705@stsci.edu>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<4DF7C8FD.7090705@stsci.edu>
Message-ID: <F9796FAB890A4CBB83D3D03F5A2C2990@gmail.com>

> So, for example, if you create a github repository for us to move 
> astrolib to, you are basically trading a circumstance where _I_ am in 
> control of the repository for one where _you_ are in control of the 
> repository. i.e. No significant change, except for the name of the 
> person who has the password needed to grant commit access.

Thanks for your comments on the whole version control issue! Just to check - on GitHub it is possible to have as many 'owners' as you want, and there is no single more important owner. Check out NumPy for example:

https://github.com/numpy

Any one of the owners can grant commit privileges to anyone else, so there is no single person in control, which may alleviate some of these worries?

In any case, I don't think the repo/VCS issue is as big a deal as I made it out to be. In the end, one can always change hosting, and it's not the most important thing to worry about right now - we can always put it up to a vote when the time comes. If people are willing to participate, I don't think the choice of VCS should matter :-)

Cheers,
Tom

> 
> Fortunately for the community, _I_ don't care who has control of the 
> repository. I'm watching over it because Perry said "get us a place to 
> put astrolib". If he says "give it to that guy", that's fine with me.
> 
> 
> (Actually, there is one difference: Assembla has a contractual 
> obligation to provide service to me; github/sourceforge/whatever have no 
> contractual obligation to you. When we selected a hosting service, we 
> thought about whether there might be a different quality of service 
> between "paying customer" and "getting service for free".)
> 
> 
> >  By using a distributed versioning system (e.g. git or hg) and a free hosting solution (neutral territory, e.g. github or mercurial) we can ensure that the project will live beyond the funding for any given institution, and the distributed aspect means that if the hosting solution goes out of business, moving to another will be easy.
> 
> 
> In fact, a DVCS makes little difference to this case. You can make your 
> own private copy of the astrolib subversion repository at any time. I 
> keep daily backups of all my off-site subversion repositories (astrolib 
> is not the only thing I have at Assembla) as part of my disaster 
> recovery plan. I know the backups are valid because my CI system checks 
> out code from the backups, so the backups are routinely tested.
> 
> If Assembla disappears tomorrow, I still have a copy of everything, that 
> I can load into any other hosting service. If STScI disappears 
> tomorrow, you could also have a copy that you could recover from -- all 
> you have to do is maintain a local copy, which is the same thing you 
> would do with the DVCS.
> 
> 
> > For example, would there be barriers to moving the astrolib packages 
> > into a new repository on e.g. GitHub?
> 
> Yes - if the new repository requires a different VCS, then some people 
> will have to learn to use the new VCS. In the case of a DVCS, there is 
> a new work flow that you need to explain to potential developers. For 
> example, I could have to support both svn and git for my developers.
> 
> On the plus side, numpy is using git. If you choose git, then I don't 
> need _two_ new VCS.
> 
> On the minus side, during my last major software build, I did not 
> succeed in compiling git on my linux machines. But then, the last time 
> I tried to compile subverion, it didn't work either. The dependencies 
> of both are nasty.
> 
> Mark S




From matthewturk at gmail.com  Tue Jun 14 17:00:54 2011
From: matthewturk at gmail.com (Matthew Turk)
Date: Tue, 14 Jun 2011 14:00:54 -0700
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <F9796FAB890A4CBB83D3D03F5A2C2990@gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<4DF7C8FD.7090705@stsci.edu>
	<F9796FAB890A4CBB83D3D03F5A2C2990@gmail.com>
Message-ID: <BANLkTin4G1o0qm3GukW2q4Mm1Cvc3UGvSA@mail.gmail.com>

Hi Tom,

On Tue, Jun 14, 2011 at 1:57 PM, Thomas Robitaille
<thomas.robitaille at gmail.com> wrote:
>> So, for example, if you create a github repository for us to move
>> astrolib to, you are basically trading a circumstance where _I_ am in
>> control of the repository for one where _you_ are in control of the
>> repository. i.e. No significant change, except for the name of the
>> person who has the password needed to grant commit access.
>
> Thanks for your comments on the whole version control issue! Just to check - on GitHub it is possible to have as many 'owners' as you want, and there is no single more important owner. Check out NumPy for example:
>
> https://github.com/numpy
>
> Any one of the owners can grant commit privileges to anyone else, so there is no single person in control, which may alleviate some of these worries?
>
> In any case, I don't think the repo/VCS issue is as big a deal as I made it out to be. In the end, one can always change hosting, and it's not the most important thing to worry about right now - we can always put it up to a vote when the time comes. If people are willing to participate, I don't think the choice of VCS should matter :-)

One thing I would bring up is that there have been any number of
yak-shaving discussions about rebasing and history rewriting and
dropping things from the reflog after the matplotlib, numpy, cython
and scipy projects have moved.  Hopefully AstroPy will be able to
avoid these by allowing or favoring a branchy model of development.
One thing I have noticed is that these discussions seem to happen much
more commonly with git than hg, and I'm not entirely clear on why that
is; I suspect it may just be that hg encourages one not to rewrite
history, and so the hydra-approach is more accepted.  With Enzo
specifically branchy development has been very handy.

-Matt

>
> Cheers,
> Tom
>
>>
>> Fortunately for the community, _I_ don't care who has control of the
>> repository. I'm watching over it because Perry said "get us a place to
>> put astrolib". If he says "give it to that guy", that's fine with me.
>>
>>
>> (Actually, there is one difference: Assembla has a contractual
>> obligation to provide service to me; github/sourceforge/whatever have no
>> contractual obligation to you. When we selected a hosting service, we
>> thought about whether there might be a different quality of service
>> between "paying customer" and "getting service for free".)
>>
>>
>> > ?By using a distributed versioning system (e.g. git or hg) and a free hosting solution (neutral territory, e.g. github or mercurial) we can ensure that the project will live beyond the funding for any given institution, and the distributed aspect means that if the hosting solution goes out of business, moving to another will be easy.
>>
>>
>> In fact, a DVCS makes little difference to this case. You can make your
>> own private copy of the astrolib subversion repository at any time. I
>> keep daily backups of all my off-site subversion repositories (astrolib
>> is not the only thing I have at Assembla) as part of my disaster
>> recovery plan. I know the backups are valid because my CI system checks
>> out code from the backups, so the backups are routinely tested.
>>
>> If Assembla disappears tomorrow, I still have a copy of everything, that
>> I can load into any other hosting service. If STScI disappears
>> tomorrow, you could also have a copy that you could recover from -- all
>> you have to do is maintain a local copy, which is the same thing you
>> would do with the DVCS.
>>
>>
>> > For example, would there be barriers to moving the astrolib packages
>> > into a new repository on e.g. GitHub?
>>
>> Yes - if the new repository requires a different VCS, then some people
>> will have to learn to use the new VCS. In the case of a DVCS, there is
>> a new work flow that you need to explain to potential developers. For
>> example, I could have to support both svn and git for my developers.
>>
>> On the plus side, numpy is using git. If you choose git, then I don't
>> need _two_ new VCS.
>>
>> On the minus side, during my last major software build, I did not
>> succeed in compiling git on my linux machines. But then, the last time
>> I tried to compile subverion, it didn't work either. The dependencies
>> of both are nasty.
>>
>> Mark S
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From matthewturk at gmail.com  Tue Jun 14 17:04:32 2011
From: matthewturk at gmail.com (Matthew Turk)
Date: Tue, 14 Jun 2011 14:04:32 -0700
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <54DF7D50-EC7E-4196-B02A-974DD965644B@nasa.gov>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<BANLkTin_xyLvD=FUXDEV2TLZSAnaAr8Kig@mail.gmail.com>
	<54DF7D50-EC7E-4196-B02A-974DD965644B@nasa.gov>
Message-ID: <BANLkTimYboN7hc4krmofXOy3g_aydFdZZQ@mail.gmail.com>

Hi Tom,

On Tue, Jun 14, 2011 at 1:20 PM, Bridgman, William T.
<William.T.Bridgman at nasa.gov> wrote:
> I've been following this while trying to find out how my NASA contractor
> status might impact how I can contribute to this effort.
>
> However, I'd like to make the point that for a lot of modern visualization
> efforts that we do at the SVS for general public outreach, observations and
> simulations can be integrated into a single viz more easily if the tools can
> play nice together. ?I have a strong interest in this area of the
> development.
>
> Here's some items where I had to integrate very diverse datasets to generate
> the final product:
>
> http://svs.gsfc.nasa.gov/goto?3316
> http://svs.gsfc.nasa.gov/goto?3743
> http://svs.gsfc.nasa.gov/search/Series/SolarDynamo.html
>
> I used numpy, scipy, & pyfits for much of it, but also had to convert
> datasets to objects & texture with tools like VTK & PIL.
>
> If there is a way to have the simulation and observation tools playing nice
> together, it would simplify outreach (and consequently funding) efforts.

These are great visualizations, very communicative and strong.

I have been following this discussion for a while, and tried to
refrain from commenting except where necessary because in some sense I
am an outsider.  The community I come from -- simulations -- has
struggled with data interoperability.  We have developed a project and
de facto community that addresses a number of the issues you have
brought up, which you can find at http://yt.enzotools.org/ ; yt is a
multi-platform, multi-code analysis and visualization package, and we
are increasing participation massively (
https://www.ohloh.net/p/yt_amr ) through hg and vigorous outreach
efforts.

We'd be very interested in starting a conversation about how to better
interoperate between simulation and observations, particularly as they
touch on the visualizations you have presented -- I would encourage
you to email our developers list; I suspect this kind of discussion is
far enough outside the scope of this thread that we should conduct
such a discussion elsewhere.  There are a number of people involved in
simulated observations and simulation/observation interplay that don't
follow this list as closely, who I have added as CC's on this email to
give a heads up to.

-Matt

>
> Tom
>
> On Jun 14, 2011, at 2:34 PM, Matthew Turk wrote:
>
>>
>> For what it's worth, in our community we have a large buy-in for
>> mercurial (with a number of extensions and value-adds that directly
>> call the API). ?I recognize that the observer and simulation
>> communities do not necessarily need as much direct cross-talk as
>> intra-community collaborations, but I felt I should put this out
>> there.
>>
>> -Matt
>
> --
> Dr. William T."Tom" Bridgman ? ? ? ? ? ? ? Scientific Visualization Studio
> Global Science & Technology, Inc. ? ? ? ? ?NASA/Goddard Space Flight Center
> Email: William.T.Bridgman at nasa.gov ? ? ? ? Code 610.3
> Phone: 301-286-1346 ? ? ? ? ? ? ? ? ? ? ? ?Greenbelt, MD 20771
> FAX: ? 301-286-1634 ? ? ? ? ? ? ? ? ? ? ? ?http://svs.gsfc.nasa.gov/
>
>
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>


From thomas.robitaille at gmail.com  Tue Jun 14 17:04:53 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 14 Jun 2011 17:04:53 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTimg=PhAyQ+phbNPLo7P+pAMaWi8DQ@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
	<4DF7B874.80103@stsci.edu> <95F4D6CAC0074BAD9DC6DEE80730067A@gmail.com>
	<BANLkTimg=PhAyQ+phbNPLo7P+pAMaWi8DQ@mail.gmail.com>
Message-ID: <983C853918434F1EBEC3C72AE4EC0063@gmail.com>

I agree that a triumvirate or small committee might be more balanced for decisions. What I'm suggesting at this stage is more just someone to moderate the discussion and get things moving - i.e. organizing votes rather than making decisions. Including decisions about how we should make decisions ;-)

I will create a poll for the VCS, but I think first the most important is to figure out the moderator(s) for the process and decide on the design of what we want. Hosting and coding should be something we worry about once we know what kind of package we want to create and how we are splitting up the work (after all, we might decide an astrokits approach is better, which means the hosting can be scattered, so then it won't matter so much).

Cheers,
Tom


On Tuesday, June 14, 2011 at 4:50 PM, Erik Tollerud wrote:

> Actually, I think it might be better to have a smallish group do this
> (3-5), assuming enough are willing. But otherwise, I think the
> wikispace-based nomination process makes sense.
> 
> I think more than one is better mainly because the BDFL/BDFN model may
> not be entirely fitting here. It works well for Python because Guido
> wrote the initial code base himself, so it already had a clear vision.
>  In this case, we're talking about combining a number of disparate
> approaches and ways of working, so a group with somewhat different
> perspectives makes sense. But as long as it's small enough (3 is
> probably ideal), it hopefully will still be able to quickly agree on a
> unified approach.
> 
> 
> > > Any volunteers?
> > > 
> > > If we have more than one, shall we hold an election?
> > I agree, and am worried that if we don't then the discussion will die out and come back next year (after all, we just need to look at the July 2010 discussions on the mailing list, and other ones before that...). I propose that anyone interested in taking on the responsibility of moderating the conversation and decision making process declare that they are interested in doing so by Thursday evening on the wiki page. I've added a section "Candidates to temporarily moderate the discussions/decisions" at
> > 
> > http://astropy.wikispaces.com/developers
> > 
> > If there is more than one person by Thursday evening, then we can create a poll and have people vote on Friday. If you are interested in the overall planning but not leading, then add yourself to the overall planning/big picture stuff. People in that category could be part of a kind of 'steering' committee.
> 
> 
> -- 
> Erik Tollerud




From perry at stsci.edu  Tue Jun 14 17:03:55 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 14 Jun 2011 17:03:55 -0400
Subject: [AstroPy] meeting issues
Message-ID: <EDE56C22-0A87-4933-9751-1DAFE577F1F1@stsci.edu>

So, perhaps we should try to get an idea of who would go to each of  
various options for face to face meetings and do that off list so  
that. Should that be added to the wiki set up by Thomas?

I do think there is a good reason to meet face to face, especially  
early on in trying get this sort of thing going. There are certain   
kinds of things that just work better that way.

The 3 categories seem to be:

1) meet at the AAS
2) meet at CFA or STScI in the fall
3) meet at scipy

Sooner would be better, but I'm afraid it is too late for scipy.

I'd rather not wait for the next AAS.

The advantage of 2 is that there are no distractions of another  
meeting or conference to deal with, the meeting would be dedicated to  
the effort entirely (and it minimizes the travel costs for one of the  
two larger organizations). Obviously I'm favoring the second option.  
But it would be good to see who could make each of these.

Perry



From jaime at iaa.es  Tue Jun 14 17:37:58 2011
From: jaime at iaa.es (Jaime Perea)
Date: Tue, 14 Jun 2011 23:37:58 +0200
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
Message-ID: <201106142337.58892.jaime@iaa.es>

Hello list

My first post to this list.

I have been working for a while now in a rather complicated data reduction pipeline. I don't considerer myself as an 
expert, no way, but it's a long time ago when I wrote my first python script. I firmly  agree with the idea of an 
astropy meeting, I guess that there is lot of people here in the old Europe that will enjoy  meeting other people and 
share knowledge.

In some sense I think that python is just like a lego, there are basic pieces, bricks that put together allow the 
programmer to write smart code.  With this idea I would like to you to help me to identify such bricks, some of them are 
easy, some maybe not.

- Numpy / SciPy - obvious, 
  
- Matplotlib for the 2D, there exist some other alternatives such as plplot.

- pyfits - This is a master piece of software I still remember something like 
open(unit=lun,file=fits,form='unformatted',recl=2880) and I don't want to see it again.

- *wcs, for this time of big catalogs, they are also mandatory.

- pyraf - this allow to access tons of astronomy sofware already there 

With such building blocks you can do a lot of things.  But I still would like to have an interface to other stuff that 
we use a lot: 

- Statistics - something handful, easy, scipy contains a lot, but there are astronomical tasks that are so repetive that 
perhaps may  deserve their own building block.

- Access to databases and eventually Virtual Observatory. e.g. For calibration we access 2MASS very often, we use 
urllib, but I would like to have something with a more general scope. 

- Also I tend to think at short wavelengths, but I would love to identify to what extent software such as AIPS or GIPSY 
is pythonic enough now, and if we can assume that there exist interfaces to such soft.

- number crunching - with stuff such as numpy anv even going further to mpi4py or pypar or whatever it turns possible  
to write numeric (modeling) code. At least for the analysis stage.

- interactive data analysis, can we turn ipython or whatever "more astronomical" or do we have to do something 
different?

I think we need to identify such pieces and write glue code that allow us to interact with them. So perhaps the first 
task has to be an inventory of sofware and human resources. And if this is done perhaps we may realize the way
we have to manage the Project.
  
I support the idea of a meeting in fall.

Receive all my best

-- 

           Jaime D. Perea Duarte. <jaime at iaa dot es>
             Linux registered user #10472

           Dep. Astrofisica Extragalactica.
           Instituto de Astrofisica de Andalucia (CSIC)
           Apdo. 3004, 18080 Granada, Spain. 


From andrej.prsa at guest.arnes.si  Tue Jun 14 17:53:49 2011
From: andrej.prsa at guest.arnes.si (Andrej Prsa)
Date: Tue, 14 Jun 2011 17:53:49 -0400
Subject: [AstroPy] meeting issues
In-Reply-To: <EDE56C22-0A87-4933-9751-1DAFE577F1F1@stsci.edu>
References: <EDE56C22-0A87-4933-9751-1DAFE577F1F1@stsci.edu>
Message-ID: <20110614175349.31b46571@gemma>

Hi Perry,

> 1) meet at the AAS
> 2) meet at CFA or STScI in the fall
> 3) meet at scipy

I would vote for (2). Perhaps a doodle poll would help here.

If there was any interest to make these workshops a bi-annual thing
(and I would strongly support it), I would be happy to organize and
host a workshop here at Villanova. We have conference/telecon/webcast
infrastructure available at the university.

Cheers,
Andrej


From aldcroft at head.cfa.harvard.edu  Tue Jun 14 20:43:56 2011
From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft)
Date: Tue, 14 Jun 2011 20:43:56 -0400
Subject: [AstroPy] Deployment and packaging
Message-ID: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>

In the extensive recent discussions there has been only slight mention
of the problems in package deployment and the related issue of
multiple Python installations.  It's common practice now for large
tool packages like CIAO, CASA, STSci_Python etc to provide binary
installs which include Python, the relevant tools and other package
dependencies.  These sit next to other installations like the native
Python, EPD, Mac ports or Mac homebrew, SciSoft, etc. This is an
annoyance for developer types but can be a showstopper for astronomers
who wonder why the script they got from their collaborator won't run.
I recently heard someone state that she purposely avoids using 3rd
party astro packages so that her code will run on different
installations.  Given the current situation this is an unfortunate but
understandable strategy.

To that end I've added a new topic "Packaging and Deployment" to the
AstroPy developers wiki.  Like many other topics this is not easy, but
I invite comments now and participation in future discussions.  The
list below starts with basic ideas for yet another Python
distribution, and ends with the two hard parts:

- Base Python binary installation with NumPy, SciPy, matplotlib
- Optional full install that includes "everything", ala EPD
- Support for linux x86, x86_64, MacOS 10.5+ (with Windows at a lower priority)
- Non-root, movable, installation which is contained in a single directory
- Package management tools to upgrade some / all packages to latest on PyPI
- IMPORTANT: buy-in from developers to distribute "pip installable"
packages on PyPI
- IMPORTANT: buy-in from developers of large tool packages to use this
distribution instead of rolling their own.

Apart from the last two (important) points, everything in the list is
already exists in one form or another, e.g. sage, ActivePython, and
probably James Turner's distribution-in-work.  Sage is really bloated
and I much prefer ActivePython (which is of course a commercial
product and therefore not entirely suitable to this purpose).

Getting everything on PyPI should not be too difficult, but getting
large tool packages to use a new distribution will likely be a
difficult and multi-year effort.

- Tom


From perry at stsci.edu  Tue Jun 14 21:12:05 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 14 Jun 2011 21:12:05 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
Message-ID: <C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>

Indeed a very important topic (the most important in my view). I don't  
have time to go into details but a few comments are inserted below.

On Jun 14, 2011, at 8:43 PM, Tom Aldcroft wrote:

> In the extensive recent discussions there has been only slight mention
> of the problems in package deployment and the related issue of
> multiple Python installations.  It's common practice now for large
> tool packages like CIAO, CASA, STSci_Python etc to provide binary
> installs which include Python, the relevant tools and other package
> dependencies.  These sit next to other installations like the native
> Python, EPD, Mac ports or Mac homebrew, SciSoft, etc. This is an
> annoyance for developer types but can be a showstopper for astronomers
> who wonder why the script they got from their collaborator won't run.
> I recently heard someone state that she purposely avoids using 3rd
> party astro packages so that her code will run on different
> installations.  Given the current situation this is an unfortunate but
> understandable strategy.
>
> To that end I've added a new topic "Packaging and Deployment" to the
> AstroPy developers wiki.  Like many other topics this is not easy, but
> I invite comments now and participation in future discussions.  The
> list below starts with basic ideas for yet another Python
> distribution, and ends with the two hard parts:
>
> - Base Python binary installation with NumPy, SciPy, matplotlib

But how to distribute binaries? And what about non-Python tools that  
are useful in the suite of astronomical tools?

> - Optional full install that includes "everything", ala EPD
> - Support for linux x86, x86_64, MacOS 10.5+ (with Windows at a  
> lower priority)
> - Non-root, movable, installation which is contained in a single  
> directory
> - Package management tools to upgrade some / all packages to latest  
> on PyPI
> - IMPORTANT: buy-in from developers to distribute "pip installable"
> packages on PyPI

We are already working on this. But I can't reiterate enough, doing  
this does not solve many of the important installation problems.  
Things can be in PyPI, use one of the standard Python distribution  
tools properly, and still not install reliably.

> - IMPORTANT: buy-in from developers of large tool packages to use this
> distribution instead of rolling their own.
>
But the most important item is missing from the list. Any guesses?

> Apart from the last two (important) points, everything in the list is
> already exists in one form or another, e.g. sage, ActivePython, and
> probably James Turner's distribution-in-work.  Sage is really bloated
> and I much prefer ActivePython (which is of course a commercial
> product and therefore not entirely suitable to this purpose).
>
Sage is unsuitable for our needs as a distribution (but it does have  
some nice tools and approaches for distribution).

Perry


From tgrav at mac.com  Tue Jun 14 21:44:33 2011
From: tgrav at mac.com (Tommy Grav)
Date: Tue, 14 Jun 2011 21:44:33 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
Message-ID: <EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>


On Jun 14, 2011, at 9:12 PM, Perry Greenfield wrote:

>> - Base Python binary installation with NumPy, SciPy, matplotlib
> 
> But how to distribute binaries? And what about non-Python tools that  
> are useful in the suite of astronomical tools?

Maybe I am misunderstanding, but when I am looking for a python
astronomy package I don't want a slew of non-python tools. What
I am looking for is packages that are useful as packages to use
in my own python codes. pyfits and pywcs are great examples of
clean python packages that are fairly easy to install and use
in python code. I am hoping and advocating that astropy/astrolib
becomes something like scipy, rather than just a bag of disparate 
package that have little on common than astronomy. 

Chers
  Tommy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110614/173623e4/attachment.html>

From perry at stsci.edu  Tue Jun 14 22:11:28 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Tue, 14 Jun 2011 22:11:28 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
Message-ID: <3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>


On Jun 14, 2011, at 9:44 PM, Tommy Grav wrote:

>
> On Jun 14, 2011, at 9:12 PM, Perry Greenfield wrote:
>
>>> - Base Python binary installation with NumPy, SciPy, matplotlib
>>
>> But how to distribute binaries? And what about non-Python tools that
>> are useful in the suite of astronomical tools?
>
> Maybe I am misunderstanding, but when I am looking for a python
> astronomy package I don't want a slew of non-python tools. What
> I am looking for is packages that are useful as packages to use
> in my own python codes. pyfits and pywcs are great examples of
> clean python packages that are fairly easy to install and use
> in python code. I am hoping and advocating that astropy/astrolib
> becomes something like scipy, rather than just a bag of disparate
> package that have little on common than astronomy.
>
You may not, but many others do. IRAF is only one example. SExtractor  
is another. And there are others. There isn't yet enough to replace  
these, and won't be for some years (if ever, it is probably silly to  
think everything we need will be in Python, or that there won't be  
some very useful things out there not written in Python).

Note that scipy is not easy to install everywhere.

So, certainly, standard Python installs for separate tools should be  
available. We aren't talking about making that go away.

Perry


From tgrav at mac.com  Tue Jun 14 22:43:44 2011
From: tgrav at mac.com (Tommy Grav)
Date: Tue, 14 Jun 2011 22:43:44 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
Message-ID: <8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>


On Jun 14, 2011, at 10:11 PM, Perry Greenfield wrote:

> 
> On Jun 14, 2011, at 9:44 PM, Tommy Grav wrote:
> 
>> 
>> On Jun 14, 2011, at 9:12 PM, Perry Greenfield wrote:
>> 
>>>> - Base Python binary installation with NumPy, SciPy, matplotlib
>>> 
>>> But how to distribute binaries? And what about non-Python tools that
>>> are useful in the suite of astronomical tools?
>> 
>> Maybe I am misunderstanding, but when I am looking for a python
>> astronomy package I don't want a slew of non-python tools. What
>> I am looking for is packages that are useful as packages to use
>> in my own python codes. pyfits and pywcs are great examples of
>> clean python packages that are fairly easy to install and use
>> in python code. I am hoping and advocating that astropy/astrolib
>> becomes something like scipy, rather than just a bag of disparate
>> package that have little on common than astronomy.
>> 
> You may not, but many others do.

No doubt. I can only voice my opinion :)

> IRAF is only one example. SExtractor is another. And there are others. There isn't yet enough to replace these, and won't be for some years (if ever, it is probably silly to think everything we need will be in Python, or that there won't be some very useful things out there not written in Python).

My opinion is that those tools belong in as separate packages. To me they just don't
belong in a python astronomy. I guess what I want is a astronomy library, rather than
a package collection. I guess that the SExtractor could be included as a C library with
a python front end, but IRAF is just a pain to install and to me constitutes a rather
hard hill to climb. Also my hope is that a proper astronomy library with C extension finally
someday make IRAF absolutely obsolete. 

> Note that scipy is not easy to install everywhere.
> 
> So, certainly, standard Python installs for separate tools should be available. We aren't talking about making that go away.

I understand that. My point, however, was that the cramming of everything astronomy into these
tool-collections is in my opinion what has been the problem for the current libraries/packages 
from becoming the defacto standard as scipy and numpy has. Your milage may vary. 

Cheers
  Tommy 



From mperrin at stsci.edu  Tue Jun 14 22:53:11 2011
From: mperrin at stsci.edu (Marshall Perrin)
Date: Wed, 15 Jun 2011 02:53:11 +0000
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
Message-ID: <91F74EF4-2551-4A4C-89A4-1B4614128D01@stsci.edu>


Packaging and distribution is definitely a hard problem - still unsolved in practice despite vast amounts of labor over decades (see: rpm, apt-get, cpan, gems, pip, iOS App store, and oh so many more).  Given that vastly larger communities than astronomy have struggled with this issue, I think we should be careful to focus our efforts, and make use of existing tools whenever possible. And moreover acknowledge up front that a perfect solution to this problem does not exist.

> list below starts with basic ideas for yet another Python
> distribution, and ends with the two hard parts:

> - Base Python binary installation with NumPy, SciPy, matplotlib


So I'm a little surprised that your very first idea is for "yet another Python distribution". In my view it makes more sense to layer our efforts on top of some already existing effort. EPD is the most obvious - I can easily imagine a script that downloads and installs EPD and then installs some set of modules on top of that EPD, thus avoiding repeating effort that others have already provided and providing an install with wide compatibility with community provided tools from outside of astronomy too.   Likewise, it would be pretty straightforward to have a meta-package on Macports, fink, or homebrew that downloads and installs some large suite of Python code using those existing package managers. In contrast, creating yet another Python installation would only exacerbate the problems of  multiple competing Python installations confusing users and preventing code from running, no?

One reason that I quite like Macports is that it makes it easy to have dependencies on non-Python software. Of course, that's not a cross-platform solution but I can imagine a set of maintained-in-parallel packages on Mac and Linux package repositories.

 - Marshall 


On Jun 14, 2011, at 8:43 PM, Tom Aldcroft wrote:
> 
> - Optional full install that includes "everything", ala EPD
> - Support for linux x86, x86_64, MacOS 10.5+ (with Windows at a lower priority)
> - Non-root, movable, installation which is contained in a single directory
> - Package management tools to upgrade some / all packages to latest on PyPI
> - IMPORTANT: buy-in from developers to distribute "pip installable"
> packages on PyPI
> - IMPORTANT: buy-in from developers of large tool packages to use this
> distribution instead of rolling their own.
> 
> Apart from the last two (important) points, everything in the list is
> already exists in one form or another, e.g. sage, ActivePython, and
> probably James Turner's distribution-in-work.  Sage is really bloated
> and I much prefer ActivePython (which is of course a commercial
> product and therefore not entirely suitable to this purpose).
> 
> Getting everything on PyPI should not be too difficult, but getting
> large tool packages to use a new distribution will likely be a
> difficult and multi-year effort.
> 
> - Tom
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From andrew at physics.uwa.edu.au  Wed Jun 15 00:01:44 2011
From: andrew at physics.uwa.edu.au (Andrew Williams)
Date: Wed, 15 Jun 2011 12:01:44 +0800
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
Message-ID: <4DF82EA8.5010105@physics.uwa.edu.au>

On 15/06/2011 10:43 AM, Tommy Grav wrote:
>
> On Jun 14, 2011, at 10:11 PM, Perry Greenfield wrote:
>> IRAF is only one example. SExtractor is another. And there are
>> others. There isn't yet enough to replace these, and won't be for
>> some years (if ever, it is probably silly to think everything we
>> need will be in Python, or that there won't be some very useful
>> things out there not written in Python).
>
> My opinion is that those tools belong in as separate packages. To me
> they just don't

I strongly agree - people who work with tools like these already will be 
_less_ likely to adopt an 'astropy' package if it also includes an extra 
copy of SExtractor, IRAF, and every other tool they might possibly want, 
generating conflicts with anything that's already installed.

'All in one' setups like this make life easier for people setting up new 
computers for training workshops or for new postgrads, but they make 
life a lot harder for people who already have the tools they need set 
up, and don't want to break them by installing some monolithic 
collection of loosely coupled packages. It's that latter set of expert 
users who are the ones you most want to get involved using and 
contributing to the collection...

For the same reason, I'm also very much against the idea of turning this 
into another complete python distribution. Apart from Windows users, 
most people already have Python installed with the operating system. 
Installing a second copy (and getting all the other libraries you want 
to use installed under _both_ Python versions) is a real pain.

There's nothing to stop people re-packaging 'astropy', or whatever the 
Python-only collection gets called, along with SExtractor, IRAF, numpy, 
and everything else, if there is demand for an 'all in one' collection.

I'm more used to catering for the other end of the user base - the 
people who, at worst, _only_ have access to software that's part of the 
default OS install, or at best, packages maintained in the ubuntu/fedora 
software systems. For example, I wrote a pyfits-like FITS file library 
in pure Python for people who couldn't get pyfits installed on the 
machine they had to use...

Another point - if this 'astropy' collection is maintained as a 
ubuntu/fedora package (plus the OSX equivalent, I forget the name), it 
will expand the available user base hugely - an awful lot of people use 
computers they don't have root access for. It's really, really hard to 
convince most sysadmins to download some random piece of software off 
the internet and install it on a machine unless it's maintained in the 
software packaging system for that OS.

The more things you add, like SExtractor and numpy (which are both 
already distinct ubuntu packages), the harder it will be to package it 
properly that way.

Andrew



From oneaufs at gmail.com  Wed Jun 15 00:10:09 2011
From: oneaufs at gmail.com (Prasanth)
Date: Wed, 15 Jun 2011 09:40:09 +0530
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <91F74EF4-2551-4A4C-89A4-1B4614128D01@stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<91F74EF4-2551-4A4C-89A4-1B4614128D01@stsci.edu>
Message-ID: <BANLkTin6t1H7vNxRwqjqML1kVQK59x8J_g@mail.gmail.com>

Hello,

 EPD is the most obvious - I can easily imagine a script that downloads and
> installs EPD and then installs some set of modules on top of that EPD, thus
> avoiding repeating effort that others have already provided and providing an
> install with wide compatibility with community provided tools from outside
> of astronomy too.


EPD is perhaps the easiest thing to do, but using EPD will mean that the
tools will remain outside the reach of non-academic users/contributors.

Prasanth

>
>
>
> On Jun 14, 2011, at 8:43 PM, Tom Aldcroft wrote:
> >
> > - Optional full install that includes "everything", ala EPD
> > - Support for linux x86, x86_64, MacOS 10.5+ (with Windows at a lower
> priority)
> > - Non-root, movable, installation which is contained in a single
> directory
> > - Package management tools to upgrade some / all packages to latest on
> PyPI
> > - IMPORTANT: buy-in from developers to distribute "pip installable"
> > packages on PyPI
> > - IMPORTANT: buy-in from developers of large tool packages to use this
> > distribution instead of rolling their own.
> >
> > Apart from the last two (important) points, everything in the list is
> > already exists in one form or another, e.g. sage, ActivePython, and
> > probably James Turner's distribution-in-work.  Sage is really bloated
> > and I much prefer ActivePython (which is of course a commercial
> > product and therefore not entirely suitable to this purpose).
> >
> > Getting everything on PyPI should not be too difficult, but getting
> > large tool packages to use a new distribution will likely be a
> > difficult and multi-year effort.
> >
> > - Tom
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org
> > http://mail.scipy.org/mailman/listinfo/astropy
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110615/f83bbbf7/attachment.html>

From erik.tollerud at gmail.com  Wed Jun 15 00:32:40 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Wed, 15 Jun 2011 00:32:40 -0400
Subject: [AstroPy] meeting issues
In-Reply-To: <EDE56C22-0A87-4933-9751-1DAFE577F1F1@stsci.edu>
References: <EDE56C22-0A87-4933-9751-1DAFE577F1F1@stsci.edu>
Message-ID: <BANLkTi=ZHNOstGX61mWBZehcXhbQq3RhVg@mail.gmail.com>

To facilitate this process, I've set up a poll with these options at

http://astropy.wikispaces.com/2011+Coordination+Meeting+Poll

Hopefully all interested parties can make their availability clear
there and we can see which option is best.

Personally, I agree with Thomas that the best solution is probably
*both* 1 and 2 (3 is more challenging given the short timescale for
those who had not previously planned on attending).

On Tue, Jun 14, 2011 at 5:03 PM, Perry Greenfield <perry at stsci.edu> wrote:
> So, perhaps we should try to get an idea of who would go to each of
> various options for face to face meetings and do that off list so
> that. Should that be added to the wiki set up by Thomas?
>
> I do think there is a good reason to meet face to face, especially
> early on in trying get this sort of thing going. There are certain
> kinds of things that just work better that way.
>
> The 3 categories seem to be:
>
> 1) meet at the AAS
> 2) meet at CFA or STScI in the fall
> 3) meet at scipy
>
> Sooner would be better, but I'm afraid it is too late for scipy.
>
> I'd rather not wait for the next AAS.
>
> The advantage of 2 is that there are no distractions of another
> meeting or conference to deal with, the meeting would be dedicated to
> the effort entirely (and it minimizes the travel costs for one of the
> two larger organizations). Obviously I'm favoring the second option.
> But it would be good to see who could make each of these.
>
> Perry
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 
Erik Tollerud


From oneaufs at gmail.com  Wed Jun 15 00:42:54 2011
From: oneaufs at gmail.com (Prasanth)
Date: Wed, 15 Jun 2011 10:12:54 +0530
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <BANLkTin4G1o0qm3GukW2q4Mm1Cvc3UGvSA@mail.gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<4DF7C8FD.7090705@stsci.edu>
	<F9796FAB890A4CBB83D3D03F5A2C2990@gmail.com>
	<BANLkTin4G1o0qm3GukW2q4Mm1Cvc3UGvSA@mail.gmail.com>
Message-ID: <BANLkTiksodZfj3qL9oAF39dbDF4qC7OyMQ@mail.gmail.com>

Hello,


> One thing I would bring up is that there have been any number of
> yak-shaving discussions about rebasing and history rewriting and
> dropping things from the reflog after the matplotlib, numpy, cython
> and scipy projects have moved.  Hopefully AstroPy will be able to
> avoid these by allowing or favoring a branchy model of development.
> One thing I have noticed is that these discussions seem to happen much
> more commonly with git than hg, and I'm not entirely clear on why that
> is; I suspect it may just be that hg encourages one not to rewrite
> history, and so the hydra-approach is more accepted.  With Enzo
> specifically branchy development has been very handy.
>
> The developers at nvie.com have suggested a branching model, available at
http://goo.gl/QI0uv, that seems to have become quite popular. If we like the
idea that there is a core group of 3-4 developers who have write access to
the repository that contains the release software, then this kind of model
will be something to consider.

One benefit of having something like Github, is that each contributor can
work in their on small repository, and send pull requests to the main
repository. The main repository can then cater to the audience that would
like to have a "kitchen-sink included" distribution, while experienced
developers can pick and choose from the smaller ones. I am not saying that a
"kitchen-sink included" approach should be the choice, but whatever the
final main astropy/astrolib distribution is. It will be the job of the core
developers to make sure that the code submitted from individual developers
meet the necessary criteria for inclusion into the main repository.

In fact the individual developers themselves can have two or more branches
where one branch would be in a form that is acceptable to the main
repository, while the other would be liked by people who want finer control.
But this may raise its own problems, which is ok as long as we advertise the
possible issues upfront.

As long as the developer refrains from history rewrites in the branch from
which the main repository is asked to pull from, the conflicts can
be manageable(?).

Depending on the complexity of the individual project, some may need to work
more closely with the main repository than others. But this would still be
very flexible.

It may be too early for this kind of specifics, but I thought it would be
nice to have something concrete in addition to the necessarily general
discussions.

Prasanth

>
> >
> > Cheers,
> > Tom
> >
> >>
> >> Fortunately for the community, _I_ don't care who has control of the
> >> repository. I'm watching over it because Perry said "get us a place to
> >> put astrolib". If he says "give it to that guy", that's fine with me.
> >>
> >>
> >> (Actually, there is one difference: Assembla has a contractual
> >> obligation to provide service to me; github/sourceforge/whatever have no
> >> contractual obligation to you. When we selected a hosting service, we
> >> thought about whether there might be a different quality of service
> >> between "paying customer" and "getting service for free".)
> >>
> >>
> >> >  By using a distributed versioning system (e.g. git or hg) and a free
> hosting solution (neutral territory, e.g. github or mercurial) we can ensure
> that the project will live beyond the funding for any given institution, and
> the distributed aspect means that if the hosting solution goes out of
> business, moving to another will be easy.
> >>
> >>
> >> In fact, a DVCS makes little difference to this case. You can make your
> >> own private copy of the astrolib subversion repository at any time. I
> >> keep daily backups of all my off-site subversion repositories (astrolib
> >> is not the only thing I have at Assembla) as part of my disaster
> >> recovery plan. I know the backups are valid because my CI system checks
> >> out code from the backups, so the backups are routinely tested.
> >>
> >> If Assembla disappears tomorrow, I still have a copy of everything, that
> >> I can load into any other hosting service. If STScI disappears
> >> tomorrow, you could also have a copy that you could recover from -- all
> >> you have to do is maintain a local copy, which is the same thing you
> >> would do with the DVCS.
> >>
> >>
> >> > For example, would there be barriers to moving the astrolib packages
> >> > into a new repository on e.g. GitHub?
> >>
> >> Yes - if the new repository requires a different VCS, then some people
> >> will have to learn to use the new VCS. In the case of a DVCS, there is
> >> a new work flow that you need to explain to potential developers. For
> >> example, I could have to support both svn and git for my developers.
> >>
> >> On the plus side, numpy is using git. If you choose git, then I don't
> >> need _two_ new VCS.
> >>
> >> On the minus side, during my last major software build, I did not
> >> succeed in compiling git on my linux machines. But then, the last time
> >> I tried to compile subverion, it didn't work either. The dependencies
> >> of both are nasty.
> >>
> >> Mark S
> >
> >
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org
> > http://mail.scipy.org/mailman/listinfo/astropy
> >
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110615/8e4eb64e/attachment.html>

From laidler at stsci.edu  Wed Jun 15 00:50:14 2011
From: laidler at stsci.edu (Vicki Laidler)
Date: Wed, 15 Jun 2011 04:50:14 +0000
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <4DF82EA8.5010105@physics.uwa.edu.au>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>,
	<4DF82EA8.5010105@physics.uwa.edu.au>
Message-ID: <173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>

I too think that the goal of this effort should neither start with yet another python distribution, nor produce a mixed general-purpose astronomy library containing IRAF, SExtractor, or whathaveyou. Both those items produce significant barriers at both ends (the developer/deployment end, and the user/installation end). 

Let's at least start out by shooting for a scikit-like python library (C extensions allowed(?)) so that people who really just want a coherent python astronomy library (a la the Astron library in IDL) can have one, and people who want to make a more general astronomy library (like the unified release) can include the resulting product as part of it.

Meanwhile, we can work on collecting some survey data from our potential users, segmented by the various user communities of interest -- I thought Andrew's point that it's the more expert users we want to attract to the project was a very good one; and if need be, revisit the question when we have actual measured data rather than anecdata on which to base the decision.

Vicki

________________________________________
From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Andrew Williams [andrew at physics.uwa.edu.au]
Sent: Wednesday, June 15, 2011 12:01 AM
To: astropy at scipy.org
Subject: Re: [AstroPy] Deployment and packaging

On 15/06/2011 10:43 AM, Tommy Grav wrote:
>
> On Jun 14, 2011, at 10:11 PM, Perry Greenfield wrote:
>> IRAF is only one example. SExtractor is another. And there are
>> others. There isn't yet enough to replace these, and won't be for
>> some years (if ever, it is probably silly to think everything we
>> need will be in Python, or that there won't be some very useful
>> things out there not written in Python).
>
> My opinion is that those tools belong in as separate packages. To me
> they just don't

I strongly agree - people who work with tools like these already will be
_less_ likely to adopt an 'astropy' package if it also includes an extra
copy of SExtractor, IRAF, and every other tool they might possibly want,
generating conflicts with anything that's already installed.

'All in one' setups like this make life easier for people setting up new
computers for training workshops or for new postgrads, but they make
life a lot harder for people who already have the tools they need set
up, and don't want to break them by installing some monolithic
collection of loosely coupled packages. It's that latter set of expert
users who are the ones you most want to get involved using and
contributing to the collection...

For the same reason, I'm also very much against the idea of turning this
into another complete python distribution. Apart from Windows users,
most people already have Python installed with the operating system.
Installing a second copy (and getting all the other libraries you want
to use installed under _both_ Python versions) is a real pain.

There's nothing to stop people re-packaging 'astropy', or whatever the
Python-only collection gets called, along with SExtractor, IRAF, numpy,
and everything else, if there is demand for an 'all in one' collection.

I'm more used to catering for the other end of the user base - the
people who, at worst, _only_ have access to software that's part of the
default OS install, or at best, packages maintained in the ubuntu/fedora
software systems. For example, I wrote a pyfits-like FITS file library
in pure Python for people who couldn't get pyfits installed on the
machine they had to use...

Another point - if this 'astropy' collection is maintained as a
ubuntu/fedora package (plus the OSX equivalent, I forget the name), it
will expand the available user base hugely - an awful lot of people use
computers they don't have root access for. It's really, really hard to
convince most sysadmins to download some random piece of software off
the internet and install it on a machine unless it's maintained in the
software packaging system for that OS.

The more things you add, like SExtractor and numpy (which are both
already distinct ubuntu packages), the harder it will be to package it
properly that way.

Andrew

_______________________________________________
AstroPy mailing list
AstroPy at scipy.org
http://mail.scipy.org/mailman/listinfo/astropy


From erik.tollerud at gmail.com  Wed Jun 15 01:06:59 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Wed, 15 Jun 2011 01:06:59 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <28E322A1-C259-4F50-B4D3-E928D063AAB7@stsci.edu>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<28E322A1-C259-4F50-B4D3-E928D063AAB7@stsci.edu>
Message-ID: <BANLkTimSKOT8vmWMEZhOApmE8y0uoVANfA@mail.gmail.com>

I mostly agree with Thomas' sentiment that it's important that the
code base itself be on neutral ground.  If nothing else, that makes it
easier to work with and manage by community members not at that
institution, and makes it more firmly community-driven. (Oh, and to be
clear, I don't mean to imply any suspicion or criticism of STScI -
this is just a general view that happens to apply in this particular
case.)

> I don't think this is quite true. From what I understand, Enthought
> does host the source code repositories for both numpy and scipy
> (scipy.org servers belong to Enthought). Why, they host this mailing
> list ;-).

Actually, all of numpy and scipy (except maybe some of the scikits?)
development is on github now.  The same is true of ipython... that's
part of the reason why my personal slant is that github is the way to
go (see below).  The web servers are hosted by Enthought though, and I
think no one disagrees that it would be great for institutions like
STScI (or individuals at those institutions) to take leading roles...
But scipy, numpy, and ipython moved to github because it's far easier
for the community to get involved using tools like those available on
that site.  Even moreso than for those packages, it's crucial that
there be a lot of community buy-in here, because of the
"roll-your-own" culture that seems to pervade much of astronomy.


> It is true that we fund the astrolib repository, and that it currently
> isn't a free hosting, it is hosted off of STScI machines. And it is a
> generic SVN repository which could easily be relocated should that
> commercial hosting service disappear. (We regularly backup the
> repository locally). And astrolib.org is a generic url with no
> intrinsic STScI link. Doing both of these was to try to avoid the
> institutional ownership issue that you raise.
>
> But we will use whatever repository system people will use the most.
> If that would be git, that would be fine with us (or me anyway :-)
> It's not clear yet to me if that is git (votes?).

And it's certainly great for the community that you're willing to do
things like this, don't get me wrong!  But the people who made github
and bitbucket have spent a long time (and a *lot* more money)
developing web sites to solve exactly the problem we're facing here,
so I think it would be a bit strange not to take advantage of it.  I
promise once you get used to git or mercurial, you won't understand
how you could ever have used SVN/CVS!

On the other hand, as Thomas pointed out in another post, I think now
may not be the time to decide this quite yet - we need a clearer
vision of the overall structure before we try to implement anything.
But certainly in the near future, a poll is in order.


-- 
Erik Tollerud


From erik.tollerud at gmail.com  Wed Jun 15 01:36:55 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Wed, 15 Jun 2011 01:36:55 -0400
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 16
In-Reply-To: <BANLkTin4G1o0qm3GukW2q4Mm1Cvc3UGvSA@mail.gmail.com>
References: <mailman.1152.1307992991.1086.astropy@scipy.org>
	<5F9765B9429443B59CB9D0C5DEAEF363@gmail.com>
	<4DF7A077.7050408@gemini.edu>
	<BA702C32DF5249DA835206A0EE4AEE68@gmail.com>
	<4DF7C8FD.7090705@stsci.edu>
	<F9796FAB890A4CBB83D3D03F5A2C2990@gmail.com>
	<BANLkTin4G1o0qm3GukW2q4Mm1Cvc3UGvSA@mail.gmail.com>
Message-ID: <BANLkTimrpMZ9UuvrMKbjo2jPrFq_DzVm1A@mail.gmail.com>

> One thing I would bring up is that there have been any number of
> yak-shaving discussions about rebasing and history rewriting and
> dropping things from the reflog after the matplotlib, numpy, cython
> and scipy projects have moved. ?Hopefully AstroPy will be able to
> avoid these by allowing or favoring a branchy model of development.
> One thing I have noticed is that these discussions seem to happen much
> more commonly with git than hg, and I'm not entirely clear on why that
> is; I suspect it may just be that hg encourages one not to rewrite
> history, and so the hydra-approach is more accepted. ?With Enzo
> specifically branchy development has been very handy.

I whole-heartedly agree with Matthew's sentiment here - branching
development is something that is far better suited to the problem we
face here of trying to connect together rather disparate tools.  As
Perry, Thomas, and Tommy all pointed out in slightly different ways,
the problem in astronomy is more complicated because it will probably
*always* be a bunch of continually evolving pieces that need to be
integrated in changing ways. It's hard to imagine how to sanely manage
this without persistent branches. (And I think git and github do a
somewhat better job of managing branches than hg/bitbucket do,
although that may be mostly a matter of opinion).





> Yes - if the new repository requires a different VCS, then some people
> will have to learn to use the new VCS. ?In the case of a DVCS, there is
> a new work flow that you need to explain to potential developers. ?For
> example, I could have to support both svn and git for my developers.
>
> On the plus side, numpy is using git. ?If you choose git, then I don't
> need _two_ new VCS.
>
> On the minus side, during my last major software build, I did not
> succeed in compiling git on my linux machines. ?But then, the last time
> I tried to compile subverion, it didn't work either. ?The dependencies
> of both are nasty.

This is an important point, *BUT* there is actually a big difference
between a DVCS and a non-D VCS in ease-of-use over the lifetime of the
project (which we are hoping is very long!).  So the question is more
whether or not it's worth the extra time to get people to learn the
distributed type.  I personally have not found it to be that difficult
to learn both hg and git, and actually I think it's *easier* in many
ways to teach those to people that don't know any sort of VCS at all.
And the advantages of a DVCS in the case of a community-driven project
will pay for the initial extra effort many times over.

>> ?By using a distributed versioning system (e.g. git or hg) and a free hosting solution (neutral territory, e.g. github or mercurial) we can ensure that the project will live beyond the funding for any given institution, and the distributed aspect means that if the hosting solution goes out of business, moving to another will be easy.
>
> In fact, a DVCS makes little difference to this case. ?You can make your
> own private copy of the astrolib subversion repository at any time. ?I
> keep daily backups of all my off-site subversion repositories (astrolib
> is not the only thing I have at Assembla) as part of my disaster
> recovery plan. ?I know the backups are valid because my CI system checks
> out code from the backups, so the backups are routinely tested.

Technically true, but more important is the fact that DVCSs inherently
*always* do this whenever you check something out.  This has the added
advantage of allowing any number of local commits and forks and ease
of merging them back in.  That's a big part of why it's great for a
far-flung community like any astronomy library will inevitably be.

And the fact that this is part of the philosophy is why the
distributed code hosting services (github and bitbucket, primarily)
seem to blow most others away in terms of feature sets (for the
developer, at least, if not always the admin) and easy-of-use...
they're designed specifically to suit the needs of a distributed
community.

Then again, I did say earlier that this decision should wait until we
have an agreed-on vision.  I guess what I've been writing is based
mostly on my vision. (and perhaps that's why there seems to be some
disagreement here... or perhaps there's no disagreement... it's a bit
hard to tell :)


-- 
Erik Tollerud


From erik.tollerud at gmail.com  Wed Jun 15 02:29:20 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Wed, 15 Jun 2011 02:29:20 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>
	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
Message-ID: <BANLkTi=0z8daRhjNqEpdkzKOYd8y=cm9cQ@mail.gmail.com>

I also agree that another python distribution is not the way to go, at
least for a time, and that we should be aiming for "expert" users.  In
fact, I think most people working in astronomy are in some weird
middle ground between users and developers, so they're primarily in
that category.

I also really don't think a *directly* EPD-based solution is a good
thing to do.  It should definitely be kept compatible with EPD, but if
someone wants to by hand install and compile only the packages that
are needed for the astronomy package, that should also be an option.
That's the whole point of packaging, after all, and it would be
foolish to try to do it ourselves when all the tools that have been
mentioned already are available and work just fine (well,
sometimes...).


Larger scale, I see two different problems that are being discussed here:

1) What's the best way to package up a new python library with it's
python dependencies (possibly with standalone C-extensions)?
To me the answer is clear: PyPI with standard python tools. If
desired, it's easy to plug into those tools, bootstrapping the library
to install dependencies as needed.

2) How can we provide everything anyone using a python library might
want, even if it's not in python?
This is a *much* harder problem, and while it would be great to solve
this problem, for now I think the resources should be devoted mostly
to solving #1, given that there are other package management tools
exist. It's fairly straightforward to make the library link into
whatever extensions are present, and present helpful messages as to
where to get them if they are not.


PyPI and it's associated tools pretty much work, and it's quite easy
to write tools to make use of it to download and install arbitrary
packages.  I've done exactly this as part of astropysics, and it has
worked fine on Mac and two different Linux distros (although perhaps
there are Windows complications?)...  If the astro library goes in a
direction of a "scikits" approach, having everything on PyPI will make
it much easier to make tools that auto-install whatever kits are
desired (something scipy/scikits seems to have trouble with, because
the kit concept was tacked on later). Furthermore, a lot of the
packaging people who work on the big packaging projects use PyPI as an
easy way to monitor and update for new releases, so it's in some sense
using PyPI is two birds with one stone.

PyPI also handles binaries, although admittedly not as cleanly.  But
aren't most of the "expert" users going to have compilers? In that
case tools like pip easily install anything with a C-extension from
source.  If the concern is about binary *dependencies* that aren't in
the python ecosystem at all, the dizzying array of possible platforms
and versions to support makes me think that it's a problem that we
don't have the people-power to solve.

If someone wants the astronomy python library to call SExtractor, some
random C-code with a command line interface, or even some IDL routine,
it's actually quite easy to do that using inter-process communication
(subprocess.Popen and the like), and then it's left to the user to
install the outside requirement however it wants to be installed.
I've used this approach multiple times, and it has always worked fine
(even across different platforms if you do the python code right, as
long as the 2nd package works on said platform).  If a direct link to
some C-extension is needed, that is probably something to be avoided
as much as possible (or rendered optional a la the way the GUI and
OpenGL pieces of Traits are optional).






On Wed, Jun 15, 2011 at 12:50 AM, Vicki Laidler <laidler at stsci.edu> wrote:
> I too think that the goal of this effort should neither start with yet another python distribution, nor produce a mixed general-purpose astronomy library containing IRAF, SExtractor, or whathaveyou. Both those items produce significant barriers at both ends (the developer/deployment end, and the user/installation end).
>
> Let's at least start out by shooting for a scikit-like python library (C extensions allowed(?)) so that people who really just want a coherent python astronomy library (a la the Astron library in IDL) can have one, and people who want to make a more general astronomy library (like the unified release) can include the resulting product as part of it.
>
> Meanwhile, we can work on collecting some survey data from our potential users, segmented by the various user communities of interest -- I thought Andrew's point that it's the more expert users we want to attract to the project was a very good one; and if need be, revisit the question when we have actual measured data rather than anecdata on which to base the decision.
>
> Vicki
>
> ________________________________________
> From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Andrew Williams [andrew at physics.uwa.edu.au]
> Sent: Wednesday, June 15, 2011 12:01 AM
> To: astropy at scipy.org
> Subject: Re: [AstroPy] Deployment and packaging
>
> On 15/06/2011 10:43 AM, Tommy Grav wrote:
>>
>> On Jun 14, 2011, at 10:11 PM, Perry Greenfield wrote:
>>> IRAF is only one example. SExtractor is another. And there are
>>> others. There isn't yet enough to replace these, and won't be for
>>> some years (if ever, it is probably silly to think everything we
>>> need will be in Python, or that there won't be some very useful
>>> things out there not written in Python).
>>
>> My opinion is that those tools belong in as separate packages. To me
>> they just don't
>
> I strongly agree - people who work with tools like these already will be
> _less_ likely to adopt an 'astropy' package if it also includes an extra
> copy of SExtractor, IRAF, and every other tool they might possibly want,
> generating conflicts with anything that's already installed.
>
> 'All in one' setups like this make life easier for people setting up new
> computers for training workshops or for new postgrads, but they make
> life a lot harder for people who already have the tools they need set
> up, and don't want to break them by installing some monolithic
> collection of loosely coupled packages. It's that latter set of expert
> users who are the ones you most want to get involved using and
> contributing to the collection...
>
> For the same reason, I'm also very much against the idea of turning this
> into another complete python distribution. Apart from Windows users,
> most people already have Python installed with the operating system.
> Installing a second copy (and getting all the other libraries you want
> to use installed under _both_ Python versions) is a real pain.
>
> There's nothing to stop people re-packaging 'astropy', or whatever the
> Python-only collection gets called, along with SExtractor, IRAF, numpy,
> and everything else, if there is demand for an 'all in one' collection.
>
> I'm more used to catering for the other end of the user base - the
> people who, at worst, _only_ have access to software that's part of the
> default OS install, or at best, packages maintained in the ubuntu/fedora
> software systems. For example, I wrote a pyfits-like FITS file library
> in pure Python for people who couldn't get pyfits installed on the
> machine they had to use...
>
> Another point - if this 'astropy' collection is maintained as a
> ubuntu/fedora package (plus the OSX equivalent, I forget the name), it
> will expand the available user base hugely - an awful lot of people use
> computers they don't have root access for. It's really, really hard to
> convince most sysadmins to download some random piece of software off
> the internet and install it on a machine unless it's maintained in the
> software packaging system for that OS.
>
> The more things you add, like SExtractor and numpy (which are both
> already distinct ubuntu packages), the harder it will be to package it
> properly that way.
>
> Andrew
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 
Erik Tollerud


From stefan.schwarzburg at googlemail.com  Wed Jun 15 06:01:51 2011
From: stefan.schwarzburg at googlemail.com (Stefan Schwarzburg)
Date: Wed, 15 Jun 2011 12:01:51 +0200
Subject: [AstroPy] pyfits - memory leak in new_table / 'hello,
	i am new here'
In-Reply-To: <6DDFB054-38E3-4C22-A76C-BEF594FEFE6E@desy.de>
References: <6DDFB054-38E3-4C22-A76C-BEF594FEFE6E@desy.de>
Message-ID: <BANLkTikQWUwGYTbPXgck2iDhgjvvKSq59A@mail.gmail.com>

Hi Martin,

I tried your script with revision 851 and the memory leak is there.
Then I tried with trunk (currently 925) and there is no more memory leak.

The problem you had with the installation happened to me as well, but it
worked for me with:

sudo ./setup.py install

instead of

sudo python setup.py install


I have not checked why, but anyway it works. So maybe you just try that.

Cheers,
Stefan

On Tue, Jun 14, 2011 at 12:45, Martin Raue <martin.raue at desy.de> wrote:

> Hello all,
>
> I am new here. I work in very high energy gamma-ray astronomy with HESS and
> CTA and currently developing some high level analysis tools for CTA using
> python (the classical numpy, scipy, matplotlib, pyfits).
>
> I stumbled upon a problem with memory leakage in pyfits when using
> new_table.
>
> Something similar has been reported here
>
> http://physicsnlinux.wordpress.com/2011/03/28/pyfits-memory-leak-in-new_table/#more-103
> and should be fixed in
> http://trac6.assembla.com/pyfits/changeset/844
>
> But I still have problems in 844 and 851. Unfortunately, I have problems
> installing the latest trunk on my system (some output attached).
>
> Any suggestion and help would be very welcome!
>
> Best wishes,
> Martin
>
> ==============
> Example script
>
> import numpy as np
> import pyfits
> import gc
>
> for i in range(20) :
>
>    # create dummy data
>    a = np.linspace(-1., 1., 1000000)
>    b = a ** 3.
>
>    a_col = pyfits.Column(name='ACOL', format='1E', array=a)
>    b_col = pyfits.Column(name='BCOL', format='1E', array=b)
>
>    coldefs_new = pyfits.ColDefs([a_col, b_col])
>
>    newtable = pyfits.new_table(coldefs_new)
>
>    # try to clean memory
>    del newtable
>    del coldefs_new
>    gc.collect()
>
> gc.collect()
>
> =======================================================
> setup.py output when trying to install the pyfits trunk
>
> Something wrong in the installer modules ...
>
> [uh2macastro04:~/tmp/trunk] $ python setup.py install
> Downloading
> http://pypi.python.org/packages/source/d/distribute/distribute-0.6.15.tar.gz
> Extracting in /var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/tmp4nv73s
> Now working in
> /var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/tmp4nv73s/distribute-0.6.15
> Building a Distribute egg in /Users/mraue/tmp/trunk
> /Users/mraue/tmp/trunk/distribute-0.6.15-py2.6.egg
> zip_safe flag not set; analyzing archive contents...
>
> Installed
> /private/var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/easy_install-NTCEXF/stsci.distutils-0.2dev-r13055/d2to1-0.1.5-py2.6.egg
> zip_safe flag not set; analyzing archive contents...
> stsci.__init__: module references __path__
>
> Installed /Users/mraue/tmp/trunk/stsci.distutils-0.2dev_r13055-py2.6.egg
> Searching for d2to1>=0.1.5
> Reading http://pypi.python.org/simple/d2to1/
> Best match: d2to1 0.1.5
> Downloading
> http://pypi.python.org/packages/source/d/d2to1/d2to1-0.1.5.tar.gz#md5=34129f809b503449b283771b5c3f038b
> Processing d2to1-0.1.5.tar.gz
> Running d2to1-0.1.5/setup.py -q bdist_egg --dist-dir
> /var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/easy_install-ABs3GN/d2to1-0.1.5/egg-dist-tmp-oSIXxB
> Traceback (most recent call last):
>  File "setup.py", line 33, in <module>
>    use_2to3=True
>  File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py",
> line 113, in setup
>    _setup_distribution = dist = klass(attrs)
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 221, in
> __init__
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 245, in
> fetch_build_eggs
>  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 544, in
> resolve
>  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 786, in
> best_match
>  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 798, in
> obtain
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 293, in
> fetch_build_egg
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 584, in easy_install
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 614, in install_item
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 804, in install_eggs
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 1081, in build_and_install
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 1070, in run_setup
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 29, in
> run_setup
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 70, in
> run
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 31, in
> <lambda>
>  File "setup.py", line 21, in <module>
>    except ImportError:
>  File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py",
> line 152, in setup
>    dist.run_commands()
>  File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py",
> line 975, in run_commands
>    self.run_command(cmd)
>  File
> "/private/var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/easy_install-NTCEXF/stsci.distutils-0.2dev-r13055/d2to1-0.1.5-py2.6.egg/d2to1/core.py",
> line 111, in run_command
> AttributeError: 'NoneType' object has no attribute 'info'
>
> [uh2macastro04:~/tmp/trunk] $ python
> Python 2.6.5 (r265:79359, Mar 24 2010, 01:32:55)
> [GCC 4.0.1 (Apple Inc. build 5493)] on darwin
>
> -----------------------------------
>
> Martin Raue
>
> University of Hamburg
> Institute for Experimental Physics
> Luruper Chaussee 149
> D-22761 Hamburg
> Germany
>
> Phone  : +49-40-8998-5216
> Fax    : +49-40-8998-2170
> E-Mail : martin.raue at desy.de
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 
Institut f?r Astronomie und Astrophysik
Eberhard Karls Universit?t T?bingen
Sand 1   -  D-72076 T?bingen
Tel.: 07071/29-78605
-----------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110615/62f20cdf/attachment.html>

From martin.raue at desy.de  Wed Jun 15 08:38:17 2011
From: martin.raue at desy.de (Martin Raue)
Date: Wed, 15 Jun 2011 14:38:17 +0200
Subject: [AstroPy] pyfits - memory leak in new_table / 'hello,
	i 	am new here'
Message-ID: <990C4526-395D-4342-A306-94AB4A2988A5@desy.de>

Hi Stefan,

> I tried your script with revision 851 and the memory leak is there.
> Then I tried with trunk (currently 925) and there is no more memory leak.

Thanks for checking!

> 
> The problem you had with the installation happened to me as well, but it
> worked for me with:
> 
> sudo ./setup.py install
> 

Unfortunately, that produces the same error (also r925). Erik Bray wrote me that he will have a look at the installer.

In your case, it could be that your user and your superuser have different python version (and/or setups) installed or that /usr/bin/env points toward a different version of python then the python command.

Cheers, m

[uh2macastro04:~/tmp/trunk] $ sudo ./setup.py install
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.15.tar.gz
Extracting in /tmp/tmpFIh03D
Now working in /tmp/tmpFIh03D/distribute-0.6.15
Building a Distribute egg in /Users/mraue/tmp/trunk
/Users/mraue/tmp/trunk/distribute-0.6.15-py2.6.egg
zip_safe flag not set; analyzing archive contents...

Installed /private/tmp/easy_install-l3cxoi/stsci.distutils-0.2dev-r13055/d2to1-0.1.5-py2.6.egg
zip_safe flag not set; analyzing archive contents...
stsci.__init__: module references __path__

Installed /Users/mraue/tmp/trunk/stsci.distutils-0.2dev_r13055-py2.6.egg
Searching for d2to1>=0.1.5
Reading http://pypi.python.org/simple/d2to1/
Best match: d2to1 0.1.5
Downloading http://pypi.python.org/packages/source/d/d2to1/d2to1-0.1.5.tar.gz#md5=34129f809b503449b283771b5c3f038b
Processing d2to1-0.1.5.tar.gz
Running d2to1-0.1.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-L6aIqn/d2to1-0.1.5/egg-dist-tmp-XIqk56
Traceback (most recent call last):
  File "./setup.py", line 33, in <module>
    use_2to3=True
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 113, in setup
    _setup_distribution = dist = klass(attrs)
  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 221, in __init__
  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 245, in fetch_build_eggs
  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 544, in resolve
  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 786, in best_match
  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 798, in obtain
  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 293, in fetch_build_egg
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 584, in easy_install
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 614, in install_item
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 804, in install_eggs
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 1081, in build_and_install
  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py", line 1070, in run_setup
  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 29, in run_setup
  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 70, in run
  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 31, in <lambda>
  File "setup.py", line 21, in <module>
    except ImportError:
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "/private/tmp/easy_install-l3cxoi/stsci.distutils-0.2dev-r13055/d2to1-0.1.5-py2.6.egg/d2to1/core.py", line 111, in run_command
AttributeError: 'NoneType' object has no attribute 'info'
[uh2macastro04:~/tmp/trunk] $ 

-----------------------------------

Martin Raue

University of Hamburg
Institute for Experimental Physics
Luruper Chaussee 149
D-22761 Hamburg
Germany

Phone  : +49-40-8998-5216
Fax    : +49-40-8998-2170
E-Mail : martin.raue at desy.de





From thomas.robitaille at gmail.com  Wed Jun 15 08:53:21 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Wed, 15 Jun 2011 08:53:21 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTi=0z8daRhjNqEpdkzKOYd8y=cm9cQ@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>
	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
	<BANLkTi=0z8daRhjNqEpdkzKOYd8y=cm9cQ@mail.gmail.com>
Message-ID: <7250934D05D542D98FEC858244052FD9@gmail.com>

Just to add a small comment, I think this discussion should not be confused with the astrolib/astropy discussion (which is why Tom A. started this in a different thread). I think it's obvious that any new package we create should be available as a 'standard' python package that can be manually installed into any Python distribution. The issue of packaging/bundling is separate, because it applies regardless of whether or not we manage to achieve this common package.

Cheers,
Tom


On Wednesday, June 15, 2011 at 2:29 AM, Erik Tollerud wrote:

> I also agree that another python distribution is not the way to go, at
> least for a time, and that we should be aiming for "expert" users. In
> fact, I think most people working in astronomy are in some weird
> middle ground between users and developers, so they're primarily in
> that category.
> 
> I also really don't think a *directly* EPD-based solution is a good
> thing to do. It should definitely be kept compatible with EPD, but if
> someone wants to by hand install and compile only the packages that
> are needed for the astronomy package, that should also be an option.
> That's the whole point of packaging, after all, and it would be
> foolish to try to do it ourselves when all the tools that have been
> mentioned already are available and work just fine (well,
> sometimes...).
> 
> 
> Larger scale, I see two different problems that are being discussed here:
> 
> 1) What's the best way to package up a new python library with it's
> python dependencies (possibly with standalone C-extensions)?
> To me the answer is clear: PyPI with standard python tools. If
> desired, it's easy to plug into those tools, bootstrapping the library
> to install dependencies as needed.
> 
> 2) How can we provide everything anyone using a python library might
> want, even if it's not in python?
> This is a *much* harder problem, and while it would be great to solve
> this problem, for now I think the resources should be devoted mostly
> to solving #1, given that there are other package management tools
> exist. It's fairly straightforward to make the library link into
> whatever extensions are present, and present helpful messages as to
> where to get them if they are not.
> 
> 
> PyPI and it's associated tools pretty much work, and it's quite easy
> to write tools to make use of it to download and install arbitrary
> packages. I've done exactly this as part of astropysics, and it has
> worked fine on Mac and two different Linux distros (although perhaps
> there are Windows complications?)... If the astro library goes in a
> direction of a "scikits" approach, having everything on PyPI will make
> it much easier to make tools that auto-install whatever kits are
> desired (something scipy/scikits seems to have trouble with, because
> the kit concept was tacked on later). Furthermore, a lot of the
> packaging people who work on the big packaging projects use PyPI as an
> easy way to monitor and update for new releases, so it's in some sense
> using PyPI is two birds with one stone.
> 
> PyPI also handles binaries, although admittedly not as cleanly. But
> aren't most of the "expert" users going to have compilers? In that
> case tools like pip easily install anything with a C-extension from
> source. If the concern is about binary *dependencies* that aren't in
> the python ecosystem at all, the dizzying array of possible platforms
> and versions to support makes me think that it's a problem that we
> don't have the people-power to solve.
> 
> If someone wants the astronomy python library to call SExtractor, some
> random C-code with a command line interface, or even some IDL routine,
> it's actually quite easy to do that using inter-process communication
> (subprocess.Popen and the like), and then it's left to the user to
> install the outside requirement however it wants to be installed.
> I've used this approach multiple times, and it has always worked fine
> (even across different platforms if you do the python code right, as
> long as the 2nd package works on said platform). If a direct link to
> some C-extension is needed, that is probably something to be avoided
> as much as possible (or rendered optional a la the way the GUI and
> OpenGL pieces of Traits are optional).
> 
> 
> 
> 
> 
> 
> On Wed, Jun 15, 2011 at 12:50 AM, Vicki Laidler <laidler at stsci.edu (mailto:laidler at stsci.edu)> wrote:
> > I too think that the goal of this effort should neither start with yet another python distribution, nor produce a mixed general-purpose astronomy library containing IRAF, SExtractor, or whathaveyou. Both those items produce significant barriers at both ends (the developer/deployment end, and the user/installation end).
> > 
> > Let's at least start out by shooting for a scikit-like python library (C extensions allowed(?)) so that people who really just want a coherent python astronomy library (a la the Astron library in IDL) can have one, and people who want to make a more general astronomy library (like the unified release) can include the resulting product as part of it.
> > 
> > Meanwhile, we can work on collecting some survey data from our potential users, segmented by the various user communities of interest -- I thought Andrew's point that it's the more expert users we want to attract to the project was a very good one; and if need be, revisit the question when we have actual measured data rather than anecdata on which to base the decision.
> > 
> > Vicki
> > 
> > ________________________________________
> > From: astropy-bounces at scipy.org (mailto:astropy-bounces at scipy.org) [astropy-bounces at scipy.org (mailto:astropy-bounces at scipy.org)] on behalf of Andrew Williams [andrew at physics.uwa.edu.au (mailto:andrew at physics.uwa.edu.au)]
> > Sent: Wednesday, June 15, 2011 12:01 AM
> > To: astropy at scipy.org (mailto:astropy at scipy.org)
> > Subject: Re: [AstroPy] Deployment and packaging
> > 
> > On 15/06/2011 10:43 AM, Tommy Grav wrote:
> > > 
> > > On Jun 14, 2011, at 10:11 PM, Perry Greenfield wrote:
> > > > IRAF is only one example. SExtractor is another. And there are
> > > > others. There isn't yet enough to replace these, and won't be for
> > > > some years (if ever, it is probably silly to think everything we
> > > > need will be in Python, or that there won't be some very useful
> > > > things out there not written in Python).
> > > 
> > > My opinion is that those tools belong in as separate packages. To me
> > > they just don't
> > 
> > I strongly agree - people who work with tools like these already will be
> > _less_ likely to adopt an 'astropy' package if it also includes an extra
> > copy of SExtractor, IRAF, and every other tool they might possibly want,
> > generating conflicts with anything that's already installed.
> > 
> > 'All in one' setups like this make life easier for people setting up new
> > computers for training workshops or for new postgrads, but they make
> > life a lot harder for people who already have the tools they need set
> > up, and don't want to break them by installing some monolithic
> > collection of loosely coupled packages. It's that latter set of expert
> > users who are the ones you most want to get involved using and
> > contributing to the collection...
> > 
> > For the same reason, I'm also very much against the idea of turning this
> > into another complete python distribution. Apart from Windows users,
> > most people already have Python installed with the operating system.
> > Installing a second copy (and getting all the other libraries you want
> > to use installed under _both_ Python versions) is a real pain.
> > 
> > There's nothing to stop people re-packaging 'astropy', or whatever the
> > Python-only collection gets called, along with SExtractor, IRAF, numpy,
> > and everything else, if there is demand for an 'all in one' collection.
> > 
> > I'm more used to catering for the other end of the user base - the
> > people who, at worst, _only_ have access to software that's part of the
> > default OS install, or at best, packages maintained in the ubuntu/fedora
> > software systems. For example, I wrote a pyfits-like FITS file library
> > in pure Python for people who couldn't get pyfits installed on the
> > machine they had to use...
> > 
> > Another point - if this 'astropy' collection is maintained as a
> > ubuntu/fedora package (plus the OSX equivalent, I forget the name), it
> > will expand the available user base hugely - an awful lot of people use
> > computers they don't have root access for. It's really, really hard to
> > convince most sysadmins to download some random piece of software off
> > the internet and install it on a machine unless it's maintained in the
> > software packaging system for that OS.
> > 
> > The more things you add, like SExtractor and numpy (which are both
> > already distinct ubuntu packages), the harder it will be to package it
> > properly that way.
> > 
> > Andrew
> > 
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> > http://mail.scipy.org/mailman/listinfo/astropy
> 
> 
> 
> -- 
> Erik Tollerud
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
> http://mail.scipy.org/mailman/listinfo/astropy




From embray at stsci.edu  Wed Jun 15 09:25:02 2011
From: embray at stsci.edu (Erik Bray)
Date: Wed, 15 Jun 2011 09:25:02 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>
	<4DF64C59.3010501@gemini.edu>	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>	<4DF678BF.3070505@gemini.edu>	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
Message-ID: <4DF8B2AE.9070505@stsci.edu>

On 06/14/2011 11:20 AM, Erik Tollerud wrote:
> Personally, I'm rather strongly inclined against SciPy given that I
> (and I suspect others who might attend) wouldn't be able to re-arrange
> travel this quickly. I would think AAS might be a bit better given
> that many people might be there anyway, although it might be easier to
> run if it's a separate workshop... But the best thing to do is to
> simply get a vote by interested parties as to which they might be able
> to attend.

Where SciPy is concerned I would definitely suggest an informal meeting 
for anyone interested in this.  I've never been to SciPy before, but I 
image there's BoF board or something of the like?

But I agree that it's too soon for more comprehensive meeting.

Erik


> Are there any options other than the 4 I've listed above?  If not,
> then Thomas, perhaps you can pot a poll on the wikispaces site for
> which is the preferred option (wikispaces has a surveymonkey widget
> you can embed into the page)?
>
>
> On Mon, Jun 13, 2011 at 5:25 PM, Perry Greenfield<perry at stsci.edu>  wrote:
>> I think we could probably host as well (I have to check the
>> logistics). Fall is probably good for us as well but we could probably
>> do it earlier if there was interest in doing it sooner than fall.
>>
>> Perry
>>
>> On Jun 13, 2011, at 5:14 PM, Thomas Robitaille wrote:
>>
>>> Just to put the suggestion out there, Tom A. and I would be happy to
>>> try and host the meeting at CfA if there was interest for this,
>>> preferably in the fall. We have webcast capabilities for the talks
>>> (I like the idea of 'lightning' talks), though it would be harder to
>>> have remote participation for the hands-on part.
>>>
>>> Of course, we're also open to other locations, so if anyone else
>>> interested in hosting this, please feel free to suggest!
>>>
>>> Cheers,
>>> Tom
>>>
>>> On Monday, June 13, 2011 at 4:53 PM, James Turner wrote:
>>>
>>>>> I agree with the idea of a workshop! (but with few talks, and
>>>>> discussing/deciding/planning/coding would be the majority). I think
>>>>> face to face would be much better than teleconference. I also like
>>>>> the idea of a dedicated workshop, not a splinter.
>>>>
>>>> We could do both (a face-to-face workshop with video connections or
>>>> whatever to whomever else wants to dial in; I wouldn't want to shut
>>>> out students or people in far-flung continents).
>>>>
>>>> I agree that we'd keep the talks brief, just for context (1 day?)
>>>> and focus on things like how to contribute, actually doing so and
>>>> organization.
>>>>
>>>>> I suggest that those of us interested in a small workshop see what
>>>>> would be possible in our own institutions. I think in the end we
>>>>> should keep this focused on the development (rather than a general
>>>>> python in astronomy conference). Keeping it small will also make it
>>>>> easier to organize and manage.
>>>>
>>>> Chile would be nice but is probably a bit too far for people(?). As
>>>> someone pointed out, Baltimore and Harvard seem relatively accessible
>>>> for people from the US and Europe.
>>>>
>>>> Cheers,
>>>>
>>>> James.
>>>
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>
>
>


From embray at stsci.edu  Wed Jun 15 09:50:48 2011
From: embray at stsci.edu (Erik Bray)
Date: Wed, 15 Jun 2011 09:50:48 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTimg=PhAyQ+phbNPLo7P+pAMaWi8DQ@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>
	<4DF64C59.3010501@gemini.edu>	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>	<4DF678BF.3070505@gemini.edu>	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>	<4DF7B874.80103@stsci.edu>	<95F4D6CAC0074BAD9DC6DEE80730067A@gmail.com>
	<BANLkTimg=PhAyQ+phbNPLo7P+pAMaWi8DQ@mail.gmail.com>
Message-ID: <4DF8B8B8.803@stsci.edu>

On 06/14/2011 04:50 PM, Erik Tollerud wrote:
> Actually, I think it might be better to have a smallish group do this
> (3-5), assuming enough are willing.   But otherwise, I think the
> wikispace-based nomination process makes sense.
>
> I think more than one is better mainly because the BDFL/BDFN model may
> not be entirely fitting here.  It works well for Python because Guido
> wrote the initial code base himself, so it already had a clear vision.
>   In this case, we're talking about combining a number of disparate
> approaches and ways of working, so a group with somewhat different
> perspectives makes sense.  But as long as it's small enough (3 is
> probably ideal), it hopefully will still be able to quickly agree on a
> unified approach.

+1 to this.  I was thinking it might be a bit awkward to have one person 
who didn't write or even originate 99% of the code we're talking about 
dictating those projects.  Of course, even with 3 or 5 people you still 
have people making decisions for projects that they did not create.  But 
making it a handful of people instead of just one person is still an 
improvement.  It makes the personality of the BDFL less of an issue.

Erik


From sienkiew at stsci.edu  Wed Jun 15 10:28:24 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Wed, 15 Jun 2011 10:28:24 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF8B8B8.803@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>	<4DF64C59.3010501@gemini.edu>	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>	<4DF678BF.3070505@gemini.edu>	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>	<4DF7B874.80103@stsci.edu>	<95F4D6CAC0074BAD9DC6DEE80730067A@gmail.com>	<BANLkTimg=PhAyQ+phbNPLo7P+pAMaWi8DQ@mail.gmail.com>
	<4DF8B8B8.803@stsci.edu>
Message-ID: <4DF8C188.9070609@stsci.edu>

Erik Bray wrote:
> On 06/14/2011 04:50 PM, Erik Tollerud wrote:
>   
>> I think more than one is better mainly because the BDFL/BDFN model may
>> not be entirely fitting here.  It works well for Python because Guido
>> wrote the initial code base himself, so it already had a clear vision.
>>   In this case, we're talking about combining a number of disparate
>> approaches and ways of working, so a group with somewhat different
>> perspectives makes sense.  But as long as it's small enough (3 is
>> probably ideal), it hopefully will still be able to quickly agree on a
>> unified approach.
>>     
>
> +1 to this.  I was thinking it might be a bit awkward to have one person 
> who didn't write or even originate 99% of the code we're talking about 
> dictating

I reject your choice of the word "dictating".  The purpose of the 
organizer is to provide the service of organizing the group.  If 20 
people propose 20 mutually incompatible solutions, the organizer must 
aid the group in discussing the advantages/disadvantages of all the 
solutions.  The organizer must aid the group in negotiating to the point 
of choosing a single solution.  Sometimes there will not be agreement, 
and the organizer must aid the group in choosing which of the 
alternatives to implement.  And finally, the organizer must make a clear 
statement when a decision has been reached.

There is no dictating involved, except when the organizer acts as a 
tie-breaker at the end of this process.

The organizer has very little power.  That's ok, because the organizer 
doesn't _need_ much power to get the job done -- he needs other people 
to recognize that he is doing them a favor by doing all this.

I'm making such a point of this because it really is a key part of 
understanding how the system has to work.  If you try to think of the 
organizer in terms of military or business command/control, you will 
miss the point completely.  In that respect, BDFL is a horribly 
misleading term, but AFAIK it is the only term that would be widely 
understood in the python community.


>  those projects.  Of course, even with 3 or 5 people you still 
> have people making decisions for projects that they did not create.  But 
> making it a handful of people instead of just one person is still an 
> improvement.

The alternative is that nobody makes those decisions at all, so I think 
it is an acceptable tradeoff.


>   It makes the personality of the BDFL less of an issue.
>   

For it to work well, the 3 to 5 people acting as organizers need 
basically the same personality traits as a single organizer would, but 
it is hard to find people with those ideal characteristics.  It can 
sometimes happen that having a group will help compensate for non-ideal 
characteristics, as different members will have different strengths and 
be able to help each other out.

I wonder if we will have the luxury of 3 volunteers to take on the 
organizing job.




From thomas.robitaille at gmail.com  Wed Jun 15 10:40:29 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Wed, 15 Jun 2011 10:40:29 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF8C188.9070609@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
	<4DF7B874.80103@stsci.edu>
	<95F4D6CAC0074BAD9DC6DEE80730067A@gmail.com>
	<BANLkTimg=PhAyQ+phbNPLo7P+pAMaWi8DQ@mail.gmail.com>
	<4DF8B8B8.803@stsci.edu> <4DF8C188.9070609@stsci.edu>
Message-ID: <BANLkTikAQjy3J3a9Wy2BmupnTuc0DdXFFw@mail.gmail.com>

> I reject your choice of the word "dictating". ?The purpose of the
> organizer is to provide the service of organizing the group. ?If 20
> people propose 20 mutually incompatible solutions, the organizer must
> aid the group in discussing the advantages/disadvantages of all the
> solutions. ?The organizer must aid the group in negotiating to the point
> of choosing a single solution. ?Sometimes there will not be agreement,
> and the organizer must aid the group in choosing which of the
> alternatives to implement. ?And finally, the organizer must make a clear
> statement when a decision has been reached.

I completely agree with Mark, and this is why I phrased it as
"Candidates to temporarily moderate the discussions/decisions" on the
wiki. Think moderator, not dictator.

> I wonder if we will have the luxury of 3 volunteers to take on the
> organizing job.

It seems we already have three volunteers, so the idea of a group of 3
is already possible.

Cheers,
Tom

>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From jturner at gemini.edu  Wed Jun 15 10:55:56 2011
From: jturner at gemini.edu (James Turner)
Date: Wed, 15 Jun 2011 10:55:56 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF8B2AE.9070505@stsci.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>	<4DF64C59.3010501@gemini.edu>	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>	<4DF678BF.3070505@gemini.edu>	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
	<4DF8B2AE.9070505@stsci.edu>
Message-ID: <4DF8C7FC.7010407@gemini.edu>

> Where SciPy is concerned I would definitely suggest an informal meeting
> for anyone interested in this.  I've never been to SciPy before, but I
> image there's BoF board or something of the like?

I've organized the SciPy astronomy BoF before. Just email the
organizers or the list (mainly a question of who does it first).

Can I suggest adding a section to the wiki for SciPy atendees to
add their name? I was going to add one last night, but noticed that
the wiki says "email us to suggest a new section" or something
like that.

Cheers,

James.


From thomas.robitaille at gmail.com  Wed Jun 15 10:59:45 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Wed, 15 Jun 2011 10:59:45 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <4DF8C7FC.7010407@gemini.edu>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>
	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>
	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>
	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>
	<4DF647F9.2060409@gemini.edu> <4DF64C59.3010501@gemini.edu>
	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>
	<4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
	<4DF8B2AE.9070505@stsci.edu> <4DF8C7FC.7010407@gemini.edu>
Message-ID: <BANLkTimc7q1ANf2d7DDz8=YQgFY+BBE6Ew@mail.gmail.com>

> Can I suggest adding a section to the wiki for SciPy atendees to
> add their name? I was going to add one last night, but noticed that
> the wiki says "email us to suggest a new section" or something
> like that.

I created a page:

http://astropy.wikispaces.com/SciPy+2011

In future, feel free to create this kind of page directly if you can
(not sure whether you need special permissions)

Cheers,
Tom


From embray at stsci.edu  Wed Jun 15 11:00:10 2011
From: embray at stsci.edu (Erik Bray)
Date: Wed, 15 Jun 2011 11:00:10 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <4DF82EA8.5010105@physics.uwa.edu.au>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>
Message-ID: <4DF8C8FA.50206@stsci.edu>

Hello all,

I'm just getting caught up on this discussion, though Andrew's message 
already pretty much encapsulates my thoughts on this thread so far.  I 
do have some additional notes inlined below.  But first I'd just like to 
add that packaging/deployment is of particular interest to me, and it's 
something I have a bit of experience in.

I've pulled together some large-ish projects from disjoint pieces and 
made them work quite nicely on different platforms.  This includes pure 
Python projects, as well as projects that include other big system 
dependencies including libraries, web servers, etc.  (Actually, to be 
specific, one such project involved my own Ubuntu fork based on a 
specific set of packages, so I'm quite familiar with 
Ubuntu/Debian/apt/etc.)  Unfortunately Macs are a weak point for me.

I also have pretty deep knowledge of Python's packaging mechanisms, 
including the new 'packaging' module (formerly known as distutils2).  So 
I think I can contribute there.

I agree with Andrew that an AstroPy effort should focus on being based 
on one or more "pure" Python packages.  By "pure" I don't mean to 
exclude C extensions of course; and pre-built binary packages should be 
maintained for as many platforms and Python versions as possible, to 
make things easier for users who don't have development tools on their 
systems.  This is, of course, especially important for Windows.  But 
it's important for all other platforms too--just the other day I 
couldn't get matplotlib to build due to a dependency on libpng, for 
example.  Numpy can also be tricky.  But fortunately Numpy and 
matplotlib are already available as binary packages for most platforms.

As for packages that have external, non-Python dependencies (PyRAF for 
example), I do not by any means think they should be excluded 
altogether.  But they should be able to work on a multitude of system 
configurations, and should be excluded from any installation where their 
dependencies are not bet.  At any rate, where IRAF is concerned the goal 
should be to gradually replace its pieces with Python modules.  PyRAF is 
invaluable in the meantime, but in a "perfect" world it wouldn't have to 
exist at all :)

Now if there was some desire to work on a Python + numpy + IRAF + 
SExtractor + ... all encompassing distribution (as I know there is) that 
*could* also be organized under the AstroPy banner.  But I think it 
should consist of one or more separate sub-projects, loosely coupled so 
that the best practices can be followed for different target platforms 
(i.e. .deb distributions, rpm distributions).  The pure Python part may 
have certain dependencies, but it should not be targeted for any one 
specific system layout.


On 06/15/2011 12:01 AM, Andrew Williams wrote:
> 'All in one' setups like this make life easier for people setting up new
> computers for training workshops or for new postgrads, but they make
> life a lot harder for people who already have the tools they need set
> up, and don't want to break them by installing some monolithic
> collection of loosely coupled packages. It's that latter set of expert
> users who are the ones you most want to get involved using and
> contributing to the collection...

Exactly.  We use such 'all in one' setups here, but I never use them for 
development because they're installed as root!  I have my own copies of 
them to hack on, but most of the time it's inconvenient.

> For the same reason, I'm also very much against the idea of turning this
> into another complete python distribution. Apart from Windows users,
> most people already have Python installed with the operating system.
> Installing a second copy (and getting all the other libraries you want
> to use installed under _both_ Python versions) is a real pain.
>
> There's nothing to stop people re-packaging 'astropy', or whatever the
> Python-only collection gets called, along with SExtractor, IRAF, numpy,
> and everything else, if there is demand for an 'all in one' collection.

On Ubuntu/Debian this could be a meta-package that installs all 
requirements at the system level.  I definitely think this should be 
considered, but it should not be required either.

> I'm more used to catering for the other end of the user base - the
> people who, at worst, _only_ have access to software that's part of the
> default OS install, or at best, packages maintained in the ubuntu/fedora
> software systems. For example, I wrote a pyfits-like FITS file library
> in pure Python for people who couldn't get pyfits installed on the
> machine they had to use...
>
> Another point - if this 'astropy' collection is maintained as a
> ubuntu/fedora package (plus the OSX equivalent, I forget the name), it
> will expand the available user base hugely - an awful lot of people use
> computers they don't have root access for. It's really, really hard to
> convince most sysadmins to download some random piece of software off
> the internet and install it on a machine unless it's maintained in the
> software packaging system for that OS.
>
> The more things you add, like SExtractor and numpy (which are both
> already distinct ubuntu packages), the harder it will be to package it
> properly that way.

Which is why I think any software collections should be designed for 
specific platforms in mind.  If we're talking Ubuntu, most of the 
required packages already exist.  So the only work would be making 
packages for those requirements that don't already exist, and managing 
the dependencies.  On a different platform, say, Windows, it might be 
necessary to include more software in the distribution.  But I think 
it's futile to make one exactly common base of installed software for 
all possible platforms, and to me this kind of defeats the purpose of 
basing our efforts around Python (where most of the platform 
interoperability issues should be managed at the Python level).

Eri


From embray at stsci.edu  Wed Jun 15 11:05:45 2011
From: embray at stsci.edu (Erik Bray)
Date: Wed, 15 Jun 2011 11:05:45 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTi=0z8daRhjNqEpdkzKOYd8y=cm9cQ@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>	<4DF82EA8.5010105@physics.uwa.edu.au>	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
	<BANLkTi=0z8daRhjNqEpdkzKOYd8y=cm9cQ@mail.gmail.com>
Message-ID: <4DF8CA49.2030002@stsci.edu>

On 06/15/2011 02:29 AM, Erik Tollerud wrote:
> PyPI also handles binaries, although admittedly not as cleanly.  But
> aren't most of the "expert" users going to have compilers? In that
> case tools like pip easily install anything with a C-extension from
> source.  If the concern is about binary *dependencies* that aren't in
> the python ecosystem at all, the dizzying array of possible platforms
> and versions to support makes me think that it's a problem that we
> don't have the people-power to solve.

Right--it's impossible to try to support all possible platforms 
directly.  However, I don't think it's a bad idea in principle to 
organize "platform x" support under the AstroPy banner, but as 
sub-projects related to each platform.  Those platforms would have to 
have a champion interested in maintaining support for them.

Erik


From laidler at stsci.edu  Wed Jun 15 11:10:34 2011
From: laidler at stsci.edu (Vicki Laidler)
Date: Wed, 15 Jun 2011 15:10:34 +0000
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <4DF8C8FA.50206@stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>,<4DF8C8FA.50206@stsci.edu>
Message-ID: <173C0F068F6E49449797E90DF0B59EF8B70B46@EXCHMAIL1.stsci.edu>

Just a quick comment sparked by Erik's comment about PyRAF:

>As for packages that have external, non-Python dependencies (PyRAF for
>example), I do not by any means think they should be excluded
>altogether.  But they should be able to work on a multitude of system
>configurations, and should be excluded from any installation where their
>dependencies are not bet.  At any rate, where IRAF is concerned the goal
>should be to gradually replace its pieces with Python modules.  PyRAF is
>invaluable in the meantime, but in a "perfect" world it wouldn't have to
>exist at all :)

Let's remember that there is a segment of the astronomy community that not only does not use IRAF, but wants nothing to do with IRAF, and will actively run away from any project that needs IRAF to be installed. There seems to be an IRAF vs IDL split in the community (though it is by no means hard and fast), and I'd argue that we want to attract the IDL users. The IRAF/PyRAF users already have an on-ramp to Python, and the IDL Astron-library users are our natural user base given that the project goal has repeatedly been stated (not just in this specific discussion, but from the start) in terms of "something like the IDL Astron library for Python".

Vicki



From embray at stsci.edu  Wed Jun 15 11:22:55 2011
From: embray at stsci.edu (Erik Bray)
Date: Wed, 15 Jun 2011 11:22:55 -0400
Subject: [AstroPy] pyfits - memory leak in new_table / 'hello,
 i 	am new here'
In-Reply-To: <990C4526-395D-4342-A306-94AB4A2988A5@desy.de>
References: <990C4526-395D-4342-A306-94AB4A2988A5@desy.de>
Message-ID: <4DF8CE4F.3030400@stsci.edu>

On 06/15/2011 08:38 AM, Martin Raue wrote:
> Hi Stefan,
>
>> I tried your script with revision 851 and the memory leak is there.
>> Then I tried with trunk (currently 925) and there is no more memory leak.
>
> Thanks for checking!
>
>>
>> The problem you had with the installation happened to me as well, but it
>> worked for me with:
>>
>> sudo ./setup.py install
>>
>
> Unfortunately, that produces the same error (also r925). Erik Bray wrote me that he will have a look at the installer.
>
> In your case, it could be that your user and your superuser have different python version (and/or setups) installed or that /usr/bin/env points toward a different version of python then the python command.
>
> Cheers, m

Yes, the problem here is with a semi-experimental tool called d2to1 that 
I developed and am using in the packaging of pyfits.

(For those interested, its purpose is to make distutils2/packaging-style 
setup.cfg files work with setuptools/distribute).

That particular bug can only happen if you didn't already have d2to1 
installed.  I should have a fix for it checked in shortly.

As for the actual memory issue:  I confirmed that on trunk there is no 
memory leak.  But it is worth pointing out that pyfits.new_table() uses 
more memory than one might expect.  This is because it allocates a new 
array that has space for all the columns that will be added to it.  The 
original column arrays can't be use directly, because they have to be 
interleaved.

But if you're having trouble with memory usage when building tables, 
there are ways of dealing with this.  I can make specific suggestions if 
you're looking for ideas on how to go about it.

Erik


From sontag at stsci.edu  Wed Jun 15 11:46:21 2011
From: sontag at stsci.edu (Chris Sontag)
Date: Wed, 15 Jun 2011 11:46:21 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <173C0F068F6E49449797E90DF0B59EF8B70B46@EXCHMAIL1.stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>	<4DF82EA8.5010105@physics.uwa.edu.au>,
	<4DF8C8FA.50206@stsci.edu>
	<173C0F068F6E49449797E90DF0B59EF8B70B46@EXCHMAIL1.stsci.edu>
Message-ID: <4DF8D3CD.6040101@stsci.edu>

Since PyRAF is coming up in this discussion, I thought I would clarify a technical point that many of you may know, but that may not be obvious to others.

At installation time, PyRAF has no dependency on IRAF.  PyRAF can even be run without IRAF.  True, most of the things you want to do in PyRAF will not work without IRAF, but STScI is actually working on a few that do (e.g. multidrizzle).

I just thought it should be clarified, concerning installation time dependencies.  This will become more important as more IRAF parts are replaced with Python versions.

Chris


On 6/15/11 11:10 AM, Vicki Laidler wrote:
> Just a quick comment sparked by Erik's comment about PyRAF:
>
>> As for packages that have external, non-Python dependencies (PyRAF for
>> example), I do not by any means think they should be excluded
>> altogether.  But they should be able to work on a multitude of system
>> configurations, and should be excluded from any installation where their
>> dependencies are not bet.  At any rate, where IRAF is concerned the goal
>> should be to gradually replace its pieces with Python modules.  PyRAF is
>> invaluable in the meantime, but in a "perfect" world it wouldn't have to
>> exist at all :)
> Let's remember that there is a segment of the astronomy community that not only does not use IRAF, but wants nothing to do with IRAF, and will actively run away from any project that needs IRAF to be installed. There seems to be an IRAF vs IDL split in the community (though it is by no means hard and fast), and I'd argue that we want to attract the IDL users. The IRAF/PyRAF users already have an on-ramp to Python, and the IDL Astron-library users are our natural user base given that the project goal has repeatedly been stated (not just in this specific discussion, but from the start) in terms of "something like the IDL Astron library for Python".
>
> Vicki
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110615/e31a5b4c/attachment.html>

From sienkiew at stsci.edu  Wed Jun 15 12:19:13 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Wed, 15 Jun 2011 12:19:13 -0400
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTimc7q1ANf2d7DDz8=YQgFY+BBE6Ew@mail.gmail.com>
References: <4DF0FAC3.6050505@hs.uni-hamburg.de>	<C0AECAED-9701-4A76-B225-CA2716CD8430@stsci.edu>	<E6A03D3547F54C77AD5F2BF190A2CAFE@gmail.com>	<75493BE8-D47D-45E0-8293-4E0603349DE7@stsci.edu>	<4DF647F9.2060409@gemini.edu>
	<4DF64C59.3010501@gemini.edu>	<E9110030FE0E4F10A0804CC1A8C4F3D5@gmail.com>	<4DF678BF.3070505@gemini.edu>	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>	<4DF8B2AE.9070505@stsci.edu>
	<4DF8C7FC.7010407@gemini.edu>
	<BANLkTimc7q1ANf2d7DDz8=YQgFY+BBE6Ew@mail.gmail.com>
Message-ID: <4DF8DB81.50803@stsci.edu>

Thomas Robitaille wrote:
> http://astropy.wikispaces.com/SciPy+2011
>   

I put a link from www.astrolib.org to these wiki pages, mostly because I 
kept forgetting the wikispaces URL.





From erwin at mpe.mpg.de  Wed Jun 15 12:30:55 2011
From: erwin at mpe.mpg.de (Peter Erwin)
Date: Wed, 15 Jun 2011 18:30:55 +0200
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <4DF8C8FA.50206@stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au> <4DF8C8FA.50206@stsci.edu>
Message-ID: <427B9D9F-A216-4415-854E-477BD21BC803@mpe.mpg.de>


On Jun 15, 2011, at 5:00 PM, Erik Bray wrote:


> Now if there was some desire to work on a Python + numpy + IRAF + 
> SExtractor + ... all encompassing distribution (as I know there is) that 
> *could* also be organized under the AstroPy banner.  But I think it 
> should consist of one or more separate sub-projects, loosely coupled so 
> that the best practices can be followed for different target platforms 
> (i.e. .deb distributions, rpm distributions).  The pure Python part may 
> have certain dependencies, but it should not be targeted for any one 
> specific system layout.
> 


It occurs to me that ESO's scisoft is an example of such an "all encompassing
distribution", and of the problems that can entail. For one thing, the more disparate
things you try to assemble together -- and need to test to make sure they
play well together -- the longer it takes to make a new release, and the slower
the whole release process is. Which means people who rely on such a distribution
are more likely to get stuck with outdated software -- e.g., the current version
of scisoft has Python 2.5.4[*], numpy 1.3.0, and scipy 0.7.

[*] which, at least in Scisoft 7.4, was built without the bz2 module for some weird
reason, even though that's supposed to be part of the standard Python library.

   -- Peter

=============================================================
Peter Erwin                   Max-Planck-Insitute for Extraterrestrial 
erwin at mpe.mpg.de              Physics, Giessenbachstrasse
tel. +49 (0)89 30000 3695     85748 Garching, Germany
fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin





From jturner at gemini.edu  Wed Jun 15 13:39:07 2011
From: jturner at gemini.edu (James Turner)
Date: Wed, 15 Jun 2011 13:39:07 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <7250934D05D542D98FEC858244052FD9@gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>	<4DF82EA8.5010105@physics.uwa.edu.au>	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>	<BANLkTi=0z8daRhjNqEpdkzKOYd8y=cm9cQ@mail.gmail.com>
	<7250934D05D542D98FEC858244052FD9@gmail.com>
Message-ID: <4DF8EE3B.1030206@gemini.edu>

Argh, can't keep up with this thread :-).

I agree that packaging/distributing Python dependencies is a separate
problem from collaborating on a common library of modules (except that
the two things could be associated in future and maybe even share a
name like AstroPy -- but only if we can agree on a solution for
packaging, which we're nowhere near doing yet).

I think the community includes some pretty strongly opposing views
on how to tackle packaging and I'm sure there will be room for more
than one answer. For my part, I intend to remain focused on getting
our Sage-like/based distribution out of the door. We (Gemini and
STScI) only barely have the resources to complete that project and
any change of direction would certainly derail it. Maybe the community
will accept our solution and find it helpful and maybe it won't (more
likely the preference will be split), but at least we will learn
something concrete and I have no doubt that it will be useful
internally and for some users either way. It sounds like putting stuff
on PyPI will also be an important / easy to do part of the solution.

Regarding EPD, we did look briefly at using that for our effort but
had a few significant concerns. Licencing is an obvious one. Some of
us (eg. observatories) don't even qualify for the academic licence.
That's not just a question of cost, but also practicality (eg. can't
give it to someone and say "try this"). The distribution comes pre-
assembled which makes it harder to modify for our own needs. There
were also a couple of issues like it requiring special privileges to
install on a Mac (any user will be able to install what we're
working on).

Cheers,

James.


On 15/06/11 08:53, Thomas Robitaille wrote:
> Just to add a small comment, I think this discussion should not be confused with the astrolib/astropy discussion (which is why Tom A. started this in a different thread). I think it's obvious that any new package we create should be available as a 'standard' python package that can be manually installed into any Python distribution. The issue of packaging/bundling is separate, because it applies regardless of whether or not we manage to achieve this common package.
>
> Cheers,
> Tom
>
>
> On Wednesday, June 15, 2011 at 2:29 AM, Erik Tollerud wrote:
>
>> I also agree that another python distribution is not the way to go, at
>> least for a time, and that we should be aiming for "expert" users. In
>> fact, I think most people working in astronomy are in some weird
>> middle ground between users and developers, so they're primarily in
>> that category.
>>
>> I also really don't think a *directly* EPD-based solution is a good
>> thing to do. It should definitely be kept compatible with EPD, but if
>> someone wants to by hand install and compile only the packages that
>> are needed for the astronomy package, that should also be an option.
>> That's the whole point of packaging, after all, and it would be
>> foolish to try to do it ourselves when all the tools that have been
>> mentioned already are available and work just fine (well,
>> sometimes...).
>>
>>
>> Larger scale, I see two different problems that are being discussed here:
>>
>> 1) What's the best way to package up a new python library with it's
>> python dependencies (possibly with standalone C-extensions)?
>> To me the answer is clear: PyPI with standard python tools. If
>> desired, it's easy to plug into those tools, bootstrapping the library
>> to install dependencies as needed.
>>
>> 2) How can we provide everything anyone using a python library might
>> want, even if it's not in python?
>> This is a *much* harder problem, and while it would be great to solve
>> this problem, for now I think the resources should be devoted mostly
>> to solving #1, given that there are other package management tools
>> exist. It's fairly straightforward to make the library link into
>> whatever extensions are present, and present helpful messages as to
>> where to get them if they are not.
>>
>>
>> PyPI and it's associated tools pretty much work, and it's quite easy
>> to write tools to make use of it to download and install arbitrary
>> packages. I've done exactly this as part of astropysics, and it has
>> worked fine on Mac and two different Linux distros (although perhaps
>> there are Windows complications?)... If the astro library goes in a
>> direction of a "scikits" approach, having everything on PyPI will make
>> it much easier to make tools that auto-install whatever kits are
>> desired (something scipy/scikits seems to have trouble with, because
>> the kit concept was tacked on later). Furthermore, a lot of the
>> packaging people who work on the big packaging projects use PyPI as an
>> easy way to monitor and update for new releases, so it's in some sense
>> using PyPI is two birds with one stone.
>>
>> PyPI also handles binaries, although admittedly not as cleanly. But
>> aren't most of the "expert" users going to have compilers? In that
>> case tools like pip easily install anything with a C-extension from
>> source. If the concern is about binary *dependencies* that aren't in
>> the python ecosystem at all, the dizzying array of possible platforms
>> and versions to support makes me think that it's a problem that we
>> don't have the people-power to solve.
>>
>> If someone wants the astronomy python library to call SExtractor, some
>> random C-code with a command line interface, or even some IDL routine,
>> it's actually quite easy to do that using inter-process communication
>> (subprocess.Popen and the like), and then it's left to the user to
>> install the outside requirement however it wants to be installed.
>> I've used this approach multiple times, and it has always worked fine
>> (even across different platforms if you do the python code right, as
>> long as the 2nd package works on said platform). If a direct link to
>> some C-extension is needed, that is probably something to be avoided
>> as much as possible (or rendered optional a la the way the GUI and
>> OpenGL pieces of Traits are optional).
>>
>>
>>
>>
>>
>>
>> On Wed, Jun 15, 2011 at 12:50 AM, Vicki Laidler<laidler at stsci.edu (mailto:laidler at stsci.edu)>  wrote:
>>> I too think that the goal of this effort should neither start with yet another python distribution, nor produce a mixed general-purpose astronomy library containing IRAF, SExtractor, or whathaveyou. Both those items produce significant barriers at both ends (the developer/deployment end, and the user/installation end).
>>>
>>> Let's at least start out by shooting for a scikit-like python library (C extensions allowed(?)) so that people who really just want a coherent python astronomy library (a la the Astron library in IDL) can have one, and people who want to make a more general astronomy library (like the unified release) can include the resulting product as part of it.
>>>
>>> Meanwhile, we can work on collecting some survey data from our potential users, segmented by the various user communities of interest -- I thought Andrew's point that it's the more expert users we want to attract to the project was a very good one; and if need be, revisit the question when we have actual measured data rather than anecdata on which to base the decision.
>>>
>>> Vicki
>>>
>>> ________________________________________
>>> From: astropy-bounces at scipy.org (mailto:astropy-bounces at scipy.org) [astropy-bounces at scipy.org (mailto:astropy-bounces at scipy.org)] on behalf of Andrew Williams [andrew at physics.uwa.edu.au (mailto:andrew at physics.uwa.edu.au)]
>>> Sent: Wednesday, June 15, 2011 12:01 AM
>>> To: astropy at scipy.org (mailto:astropy at scipy.org)
>>> Subject: Re: [AstroPy] Deployment and packaging
>>>
>>> On 15/06/2011 10:43 AM, Tommy Grav wrote:
>>>>
>>>> On Jun 14, 2011, at 10:11 PM, Perry Greenfield wrote:
>>>>> IRAF is only one example. SExtractor is another. And there are
>>>>> others. There isn't yet enough to replace these, and won't be for
>>>>> some years (if ever, it is probably silly to think everything we
>>>>> need will be in Python, or that there won't be some very useful
>>>>> things out there not written in Python).
>>>>
>>>> My opinion is that those tools belong in as separate packages. To me
>>>> they just don't
>>>
>>> I strongly agree - people who work with tools like these already will be
>>> _less_ likely to adopt an 'astropy' package if it also includes an extra
>>> copy of SExtractor, IRAF, and every other tool they might possibly want,
>>> generating conflicts with anything that's already installed.
>>>
>>> 'All in one' setups like this make life easier for people setting up new
>>> computers for training workshops or for new postgrads, but they make
>>> life a lot harder for people who already have the tools they need set
>>> up, and don't want to break them by installing some monolithic
>>> collection of loosely coupled packages. It's that latter set of expert
>>> users who are the ones you most want to get involved using and
>>> contributing to the collection...
>>>
>>> For the same reason, I'm also very much against the idea of turning this
>>> into another complete python distribution. Apart from Windows users,
>>> most people already have Python installed with the operating system.
>>> Installing a second copy (and getting all the other libraries you want
>>> to use installed under _both_ Python versions) is a real pain.
>>>
>>> There's nothing to stop people re-packaging 'astropy', or whatever the
>>> Python-only collection gets called, along with SExtractor, IRAF, numpy,
>>> and everything else, if there is demand for an 'all in one' collection.
>>>
>>> I'm more used to catering for the other end of the user base - the
>>> people who, at worst, _only_ have access to software that's part of the
>>> default OS install, or at best, packages maintained in the ubuntu/fedora
>>> software systems. For example, I wrote a pyfits-like FITS file library
>>> in pure Python for people who couldn't get pyfits installed on the
>>> machine they had to use...
>>>
>>> Another point - if this 'astropy' collection is maintained as a
>>> ubuntu/fedora package (plus the OSX equivalent, I forget the name), it
>>> will expand the available user base hugely - an awful lot of people use
>>> computers they don't have root access for. It's really, really hard to
>>> convince most sysadmins to download some random piece of software off
>>> the internet and install it on a machine unless it's maintained in the
>>> software packaging system for that OS.
>>>
>>> The more things you add, like SExtractor and numpy (which are both
>>> already distinct ubuntu packages), the harder it will be to package it
>>> properly that way.
>>>
>>> Andrew
>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>>
>>
>> --
>> Erik Tollerud
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org (mailto:AstroPy at scipy.org)
>> http://mail.scipy.org/mailman/listinfo/astropy
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


From stefan.czesla at hs.uni-hamburg.de  Wed Jun 15 17:23:58 2011
From: stefan.czesla at hs.uni-hamburg.de (Stefan Czesla)
Date: Wed, 15 Jun 2011 23:23:58 +0200
Subject: [AstroPy] Proliferating py-astro-libs
In-Reply-To: <BANLkTikAQjy3J3a9Wy2BmupnTuc0DdXFFw@mail.gmail.com>
References: <4DF678BF.3070505@gemini.edu>
	<2A288F6CE58C40CE9E0824285A9C82B6@gmail.com>
	<3296B214-71D7-4BEC-9EE3-F45690D15F7B@stsci.edu>
	<BANLkTinrSyGVkv_MJzEnvhew38jbQ7fosA@mail.gmail.com>
	<4DF7B874.80103@stsci.edu>
	<95F4D6CAC0074BAD9DC6DEE80730067A@gmail.com>
	<BANLkTimg=PhAyQ+phbNPLo7P+pAMaWi8DQ@mail.gmail.com>
	<4DF8B8B8.803@stsci.edu> <4DF8C188.9070609@stsci.edu>
	<BANLkTikAQjy3J3a9Wy2BmupnTuc0DdXFFw@mail.gmail.com>
Message-ID: <20110615212358.GB7833@hspc61.hs.uni-hamburg.de>

Hi,

as we have been talking about leadership structure, moderators,
dictators, and the role of the community, the Boost libraries
came into my mind, and I think that we can learn something from
their experiences.

For those who are not familiar with them:
Boost is a collection of advanced, high quality c++ libraries.
For a new library to be accepted into Boost, the library has
to pass a peer review process. Every reviewer is asked to
comment on the following questions:

   - What is your evaluation of the design?
   - What is your evaluation of the implementation?
   - What is your evaluation of the documentation?
   - What is your evaluation of the potential usefulness of XXX?
   - Did you try to use it?  With what compiler?  Did you have any problems?
   - How much effort did you put into your evaluation? A glance? A
     quick reading? In-depth study?
   - Are you knowledgeable about the problem domain?

   - Do you think that XXX should be accepted into Boost?
     Be sure to say this explicitly so that your other comments don't
     obscure your overall opinion.

Whether the library is accepted or rejected and what adaptations have still to
be made is a result of the review process.

Recently, Boost announced the formation of a ``steering committee'', which takes
responsibility of ``executive decision making'' to reach a higher effectiveness
in making major decisions (for those interested, have a look at this:
  http://tinyurl.com/3l27o3q
it is worth it.)

It is clear that we have no figure such as Guido or Linus among us,
who obtained the authority to lead the project through brilliant work *and*
by creating it. Therefore, I fear that every group or individual will have
a hard time to gather a horde of self-confident developers behind ideas they
do not support 100%.

I like the idea of a slightly formal review process like in Boost (not necessarily only of
libraries, I do not anticipate any definite structure of what is to be created here),
because it leaves the major responsibility to the (participating part of) the community and
lets the ``moderator'' only carry out a majority decision, which is a very good
legitimization. Evaluating a list discussion may be much more subjective and
controversial, which leaves a lot more room for trouble. This would also ensure
that certain standards, e.g., of code and documentation quality can be met. I see, however,
that such a formalism may hamper community participation, because many may feel
intimidated or simply incapable of contributing at that level. So -- I am only fantasizing
now, because little has been settled so far -- if there were such a library review
process, then participating in accepted libraries should be as simple as possible.
There may also be ``unstable'' branches were people can try out things.

Whatever the details, I believe that some inspiration from Boost and, in particular,
a slightly formal, transparent decision making mechanism could help a lot in uniting
developers behind decisions.

Stefan




From aldcroft at head.cfa.harvard.edu  Wed Jun 15 20:47:43 2011
From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft)
Date: Wed, 15 Jun 2011 20:47:43 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>
	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
Message-ID: <BANLkTimRu3_DjkA5dRuJEBKaU2OcNV10cg@mail.gmail.com>

On Wed, Jun 15, 2011 at 12:50 AM, Vicki Laidler <laidler at stsci.edu> wrote:
> I too think that the goal of this effort should neither start with yet another python distribution, nor produce a mixed general-purpose astronomy library containing IRAF, SExtractor, or whathaveyou. Both those items produce significant barriers at both ends (the developer/deployment end, and the user/installation end).

I definitely agree that we don't want to produce a distribution that
contains all that. I was talking about something on the order of EPD,
e.g. with ipython, HDF5 / pytables, py4mpi, etc.  Just python stuff
that is very handy and not necessarily easy to build.  My original
point, which seems to have gotten lost, is that there is a continuing
proliferation of distributions that large projects are producing in
the form of binary installs for their analysis packages.  I'd like to
see all those large projects coalesce on a single common Python
distribution.  That's more of a political than technical issue, but
the benefit to end users would be tremendous.

>
>
> Meanwhile, we can work on collecting some survey data from our potential users, segmented by the
> various user communities of interest -- I thought Andrew's point that it's the more expert users we want
> to attract to the project was a very good one; and if need be, revisit the question when we have actual
>measured data rather than anecdata on which to base the decision.

Here are some relevant statistics from CfA:

- Number of PhD scientists, grad students, and staff programmers: ~400
(yes, CfA is huge)
- People subscribed to the CfA python users mailing list: 143
- People subscribed to the IDL users mailing list: 42
- Last new subscription to IDL list: Jan 10, 2010.  The tide is turning!  :-)

In February we (myself, Thomas R, Gus Muench, Brian Refsdal) announced
the 7-part "Practical Python for Astronomers" workshop series to a
CfA-wide distribution.  Here are some numbers:

- People attending the initial introductory talk: ~75
- Percentage of those with "advanced" programming skills,
   i.e. use classes, write modules, etc (by show of hands): ~15%
- "Median" programming level: basic control structures and functions
   in single-use scripts
- Most common issue on the CfA python users list: installation problems
- Most exuberant positive feedback from the Workshop:
  "The instructions for installing Python were fantastic.  I've toyed
with Python for
   years but never gotten over the initial barrier because I could
never get everything
   installed in a way that worked.  What I found by googling was
always confusing.
   Now I'm really going to start using Python."

Personally I'm most interested in developing tools that the "median
research astronomer" can use.  A big part of that is making Python
easy to use.  The "expert" users will take care of themselves and
constitute only a small fraction of successful paper-writing,
grant-winning astronomers.  It always surprises me but I see no
correlation at all between programming ability and research success.

I would find it truly disappointing if all the AstroPy effort was
really aimed at the expert end of the spectrum.  There is a huge
interest in Python within the astronomy community, but from my
experience most of that comes from people that are far from expert or
developer.

- Tom


From mrdavis at stsci.edu  Wed Jun 15 21:30:33 2011
From: mrdavis at stsci.edu (Matt Davis)
Date: Thu, 16 Jun 2011 01:30:33 +0000
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTimRu3_DjkA5dRuJEBKaU2OcNV10cg@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>
	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
	<BANLkTimRu3_DjkA5dRuJEBKaU2OcNV10cg@mail.gmail.com>
Message-ID: <8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>

I can see that there seems to be substantial interest in producing an easy Python installation usable by astronomers. Something like EPD but free, or maybe like Python(x,y) for Windows. In fact, there's no reason something like that needs to be specific to astronomy. All the tools you list would be great for someone in geoscience, medicine, engineering, or many other fields. I think it would be fantastic if we could fill this need, but I don't think it's a need specific to astronomy and for that reason it should be a separate project from AstroPy (with substantial staff overlap, to be sure). I have worked in Earth sciences and they face the same challenges to Python adoption that astronomers do. And there is much to be gained by helping other fields join the Python fold, especially when it comes to things like handling large data sets, statistics, pipelines, parallel processing, imaging, etc.

AstroPy, I think, should be a project separate from being packaged with anything else. It can have dependencies, of course, and some modules may have more dependencies than others, but the fact would be that someone could install AstroPy tools into a Python environment they made themselves, into stsci_python, into EPD, into some Python distribution we (as a community) make, or into any other Python environment that supports our distribution method.

- Matt

From oneaufs at gmail.com  Thu Jun 16 04:24:21 2011
From: oneaufs at gmail.com (Prasanth)
Date: Thu, 16 Jun 2011 13:54:21 +0530
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>
	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
	<BANLkTimRu3_DjkA5dRuJEBKaU2OcNV10cg@mail.gmail.com>
	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
Message-ID: <BANLkTi=P+pHM0KD8t_VfQf0_fU7tRSGPYg@mail.gmail.com>

Hello,

I strongly agree with Matt Davis and Thomas Robitaille here.

Many of the packages like asciidata, vo.table, ATpy, APLpy, stsci_python
etc., are easier to install, if we have a distribution that satisfies the
basic requirements.

Right now I think the easiest way for a non expert person with no root
access to quickly get started with using Python, is by first downloading
EPD/Python(x,y) and then the astronomy package of interest. If there is an
EPD like free package with most of the core dependencies, not specifically
astronomy related, then it might be easier for a wider range of non expert
users to get started easily.

As Matt points out, this could possibly help other user communities as well.

Prasanth

On Thu, Jun 16, 2011 at 7:00 AM, Matt Davis <mrdavis at stsci.edu> wrote:

> I can see that there seems to be substantial interest in producing an easy
> Python installation usable by astronomers. Something like EPD but free, or
> maybe like Python(x,y) for Windows. In fact, there's no reason something
> like that needs to be specific to astronomy. All the tools you list would be
> great for someone in geoscience, medicine, engineering, or many other
> fields. I think it would be fantastic if we could fill this need, but I
> don't think it's a need specific to astronomy and for that reason it should
> be a separate project from AstroPy (with substantial staff overlap, to be
> sure). I have worked in Earth sciences and they face the same challenges to
> Python adoption that astronomers do. And there is much to be gained by
> helping other fields join the Python fold, especially when it comes to
> things like handling large data sets, statistics, pipelines, parallel
> processing, imaging, etc.
>
> AstroPy, I think, should be a project separate from being packaged with
> anything else. It can have dependencies, of course, and some modules may
> have more dependencies than others, but the fact would be that someone could
> install AstroPy tools into a Python environment they made themselves, into
> stsci_python, into EPD, into some Python distribution we (as a community)
> make, or into any other Python environment that supports our distribution
> method.
>
> - Matt
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110616/147ce85f/attachment.html>

From luigi at lambrate.inaf.it  Thu Jun 16 05:55:21 2011
From: luigi at lambrate.inaf.it (Luigi Paioro)
Date: Thu, 16 Jun 2011 11:55:21 +0200
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTi=P+pHM0KD8t_VfQf0_fU7tRSGPYg@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>	<4DF82EA8.5010105@physics.uwa.edu.au>	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>	<BANLkTimRu3_DjkA5dRuJEBKaU2OcNV10cg@mail.gmail.com>	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
	<BANLkTi=P+pHM0KD8t_VfQf0_fU7tRSGPYg@mail.gmail.com>
Message-ID: <4DF9D309.3070909@lambrate.inaf.it>

Hello,

   I agree with Matt, Robitaille and Prasanth.

I see four different points raised during this huge thread:

1. Build up one (and just one) package (or set of non duplicated 
packages) containing the most complete and interoperable Python software 
for astronomy;

2. With respect to 1, how to handle third party dependencies, in 
particular if it is not Python stuff, e.g. legacy software like 
SExtractor, Galfit, IRAF, etc.;

3. For the astronomical Python package(s), which packaging adopt;

4. Whether or not make an effort to produce a reference Python & C. 
distribution (maybe binary) like EPD etc.


With respect to point 1. I certainly think that it would be great, but I 
would prefer having several small and well focused interoperable 
packages than a single huge package all comprehensive. Provided that all 
packages are developed by the same community (but each single package by 
a smaller team), an packaged and developed following the same 
"astronomical framework", then, in my opinion, with this approach the 
target would be centred in a more flexible and effective way,


With respect to point 2, I think that for the time being we can assume 
that the third party software is installed separately as a requirement, 
and we just provide the Python bindings (by the way, I have already 
produced a Python wrapper to SExtractor as a demo within FASE project, 
but I assume that SExtractor is installed and in the PATH). In any case 
I think that, once we have this Python framework well defined and 
stable, then we can try to collaborate with the legacy software 
developers in order to find a shared strategy for the software distribution.


Point 3... no doubts: distutils, pip and PyPI (or a separate 
implementation of PyPI).


With respect to point 4. I bring to you my experience along several 
years distributing Python software within the PANDORA initiative (R.I.P.).

We started producing software with mixed Python and C code. There were a 
lot of dependencies with third party libraries and we decided to build 
up a detailed documentation of all requirements. We didn't distribute 
the binaries of our software, but just the source to be compiled.
Result: most of the end users were discouraged at the first sight of 
such documentation and gave up; those balder that decided to give a try, 
frequently had a lot of installation problems with the third party 
software, and in those rare case where they didn't give up, they asked 
us to solve their problems (sometimes leaving to us their PC!!!); The 
remaining (very few) users that arrived to install our software, usually 
had a lot of problems 'cause there wasn't a complete compatibility 
between the libraries they installed and our software.

A mess.

Then we decided to select the versions of the dependences and build up a 
package installing all the third party software we needed (the add-ons, 
which is more or less the same idea of EPD, Sage, etc.). The add-ons 
solved the problem of incompatibility between our software and the third 
party libraries, but two new problems raised:
a) since we distributed the add-ons as sources which were automatically 
locally compiled by a script we set up, there were still a lot of 
compiling problems of the add-ons, and the users still asked us a lot of 
support, up to giving us their computers (again)!
b) after some years the add-ons were no more compatible with the most 
recent platforms and then we had to bug fix the third party software in 
order to solve their incompatibility (!!!) or upgrade it, with the 
unpleasant consequence of running back all the following 
incompatibilities in our software.

Another mess.

Presently my opinion is that there cannot be a final solution, but, at 
least to simplify the end-user life, it would be nice having an open and 
free reference Python & C. binary distribution (one for platform) a la 
EPD/CASA/etc., and developing the astro-python-packages taking into 
account this reference distribution, and leaving as a recommendation to 
support other contexts. It is clear that producing and maintaining such 
a reference distribution is a job by itself, and it requires a specific 
team of developers and maintainers willing to do this dirty job. 
Moreover, it is clear that this reference distribution must be upgraded 
periodically in order to support the new OSs, and this will implicate a 
periodical upgrade of the astro-python-packages (where needed).


Sorry for this long mail.

Luigi



Il 06/16/11 10:24, Prasanth ha scritto:
> Hello,
>
> I strongly agree with Matt Davis and Thomas Robitaille here.
>
> Many of the packages like asciidata, vo.table, ATpy, APLpy, stsci_python
> etc., are easier to install, if we have a distribution that satisfies
> the basic requirements.
>
> Right now I think the easiest way for a non expert person with no root
> access to quickly get started with using Python, is by first downloading
> EPD/Python(x,y) and then the astronomy package of interest. If there is
> an EPD like free package with most of the core dependencies, not
> specifically astronomy related, then it might be easier for a wider
> range of non expert users to get started easily.
>
> As Matt points out, this could possibly help other user communities as well.
>
> Prasanth
>
> On Thu, Jun 16, 2011 at 7:00 AM, Matt Davis <mrdavis at stsci.edu
> <mailto:mrdavis at stsci.edu>> wrote:
>
>     I can see that there seems to be substantial interest in producing
>     an easy Python installation usable by astronomers. Something like
>     EPD but free, or maybe like Python(x,y) for Windows. In fact,
>     there's no reason something like that needs to be specific to
>     astronomy. All the tools you list would be great for someone in
>     geoscience, medicine, engineering, or many other fields. I think it
>     would be fantastic if we could fill this need, but I don't think
>     it's a need specific to astronomy and for that reason it should be a
>     separate project from AstroPy (with substantial staff overlap, to be
>     sure). I have worked in Earth sciences and they face the same
>     challenges to Python adoption that astronomers do. And there is much
>     to be gained by helping other fields join the Python fold,
>     especially when it comes to things like handling large data sets,
>     statistics, pipelines, parallel processing, imaging, etc.
>
>     AstroPy, I think, should be a project separate from being packaged
>     with anything else. It can have dependencies, of course, and some
>     modules may have more dependencies than others, but the fact would
>     be that someone could install AstroPy tools into a Python
>     environment they made themselves, into stsci_python, into EPD, into
>     some Python distribution we (as a community) make, or into any other
>     Python environment that supports our distribution method.
>
>     - Matt


From erwin at mpe.mpg.de  Thu Jun 16 06:47:25 2011
From: erwin at mpe.mpg.de (Peter Erwin)
Date: Thu, 16 Jun 2011 12:47:25 +0200
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>
	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
	<BANLkTimRu3_DjkA5dRuJEBKaU2OcNV10cg@mail.gmail.com>
	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
Message-ID: <C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>


On Jun 16, 2011, at 3:30 AM, Matt Davis wrote:

> I can see that there seems to be substantial interest in producing an easy Python installation usable by astronomers. Something like EPD but free, or maybe like Python(x,y) for Windows. In fact, there's no reason something like that needs to be specific to astronomy. All the tools you list would be great for someone in geoscience, medicine, engineering, or many other fields. I think it would be fantastic if we could fill this need, but I don't think it's a need specific to astronomy and for that reason it should be a separate project from AstroPy (with substantial staff overlap, to be sure). I have worked in Earth sciences and they face the same challenges to Python adoption that astronomers do. And there is much to be gained by helping other fields join the Python fold, especially when it comes to things like handling large data sets, statistics, pipelines, parallel processing, imaging, etc.

Or vice-versa, even -- a little googling turns up a project called BioPython, which has apparently been operating
since ~ 2000 (where "bio" in this case appears to mean "bioinformatics", rather than some quixotic attempt to cover all of
biology!):

www.biopython.org

Here's a somewhat recent (2009) overview of the package:
http://bioinformatics.oxfordjournals.org/content/25/11/1422.full

Those who are interested in Grand Unified Packages might look into how that's been done, or even
contact some of the folks behind BioPython to see if they have any wisdom to share.
(It looks, from my very brief inspection, as if they *don't* attempt to include their own Python
distribution, but instead provide links and notes for how to install Python.)



I'll confess I'm not sure why the standard Python distributions available at www.python.org
aren't considered valid "easy Python installations" -- at least for Mac and Windows machines. I know 
that the standard binary distributions of numpy, scipy, and matplotlib for Macs are designed to install
into the www.python.org Python distributions (and I assume the same is true for the Windows
binary versions of those packages). The www.python.org installations *are* the standard, up-to-date
versions, after all, and they have no peculiar license restrictions.

Is Linux, then, the main problem as far as "easy Python [+numpy/scipy/matplotlib] installations" go?

   -- Peter

=============================================================
Peter Erwin                   Max-Planck-Insitute for Extraterrestrial 
erwin at mpe.mpg.de              Physics, Giessenbachstrasse
tel. +49 (0)89 30000 3695     85748 Garching, Germany
fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin





From crawford at saao.ac.za  Thu Jun 16 07:00:04 2011
From: crawford at saao.ac.za (Steve Crawford)
Date: Thu, 16 Jun 2011 13:00:04 +0200 (SAST)
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <4DF9D309.3070909@lambrate.inaf.it>
Message-ID: <1af23792-78fc-4f66-a1fa-e2db103e6e27@mailcape>


I think my main concerns have been mentioned by others, but I'd just like to add to the discussion some thought to the discussion:

1.  In my experience, the simpler the install procedure, the more likely the adoption.  The first time someone runs into a problem with the install, they will give up and go back to using their old software.   In this light, I would be very supportive of the proposed two package approach:  1.  A pure python/c package with the minimum dependence on other libraries and easily installed using standard tools.   2. A wrapper package for accessing other useful astronomy software.   

2.  My biggest headache in users installing my software is when they have this other monolithic packages installed that install everything.  If your package isn't in the everything, then it can be a nightmare to get your software to install on that system with all the conflicts that can be generated.  

3.  I may be mis-understanding this part of the conversation, but I can not support any packaging which would require paid-licensing at any level in order to install.  Personally, I'm all for supporting Enthought, but it would not be feasible in my community as many African facilities cannot afford or justify licensing to our funding agencies beyond what is really, truly necessary.  That's one of the primary reasons we adopted python as our language of choice and not IDL.    Its free.  


From eric at depagne.org  Thu Jun 16 07:01:26 2011
From: eric at depagne.org (=?iso-8859-1?q?=C9ric_Depagne?=)
Date: Thu, 16 Jun 2011 13:01:26 +0200
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
	<C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>
Message-ID: <201106161301.27468.eric@depagne.org>

> Is Linux, then, the main problem as far as "easy Python
> [+numpy/scipy/matplotlib] installations" go?
To my knowledge of various distributions, installing python/numpy/scipy is 
very easy when running linux, it's just a matter of asking your package 
installer to do so, and here it goes.
Of course, one has to know that it's done this way, but I suspect that it's 
not the biggest hurdle (maybe I'm wrong, I do not know).

?ric.
> 
>    -- Peter
> 
> =============================================================
> Peter Erwin                   Max-Planck-Insitute for Extraterrestrial
> erwin at mpe.mpg.de              Physics, Giessenbachstrasse
> tel. +49 (0)89 30000 3695     85748 Garching, Germany
> fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin
> 
> 
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-- 
Un clavier azerty en vaut deux
----------------------------------------------------------
?ric Depagne                            eric at depagne.org


From aldcroft at head.cfa.harvard.edu  Thu Jun 16 07:59:56 2011
From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft)
Date: Thu, 16 Jun 2011 07:59:56 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <201106161301.27468.eric@depagne.org>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
	<C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>
	<201106161301.27468.eric@depagne.org>
Message-ID: <BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>

On Thu, Jun 16, 2011 at 7:01 AM, ?ric Depagne <eric at depagne.org> wrote:
>> Is Linux, then, the main problem as far as "easy Python
>> [+numpy/scipy/matplotlib] installations" go?
> To my knowledge of various distributions, installing python/numpy/scipy is
> very easy when running linux, it's just a matter of asking your package
> installer to do so, and here it goes.
> Of course, one has to know that it's done this way, but I suspect that it's
> not the biggest hurdle (maybe I'm wrong, I do not know).
>

This requires root access.  At CfA all the system-managed linux
machines (hundreds) run CentOS-5, which includes Python 2.4 as the
native python.  The RPM installs for numpy / scipy will be ancient
versions.  Using CentOS-5 (as I am forced to do) is a challenge since
the libraries are likewise ancient.  Getting PyGTK to build from
source was a major project in archaeology, trying to guess which
versions of the gtk dependencies would build.

The upshot is that to provide a universally applicable solution
requires a non-root installable distribution.

I think everyone agrees that EPD itself is not acceptable because of
license restrictions.  Probably 2/3 of people at CfA would not qualify
under the academic license, myself included.

- Tom A


From eric at depagne.org  Thu Jun 16 08:19:10 2011
From: eric at depagne.org (=?iso-8859-1?q?=C9ric_Depagne?=)
Date: Thu, 16 Jun 2011 14:19:10 +0200
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<201106161301.27468.eric@depagne.org>
	<BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>
Message-ID: <201106161419.10422.eric@depagne.org>

Le jeudi 16 juin 2011 13:59:56, Tom Aldcroft a ?crit :
> On Thu, Jun 16, 2011 at 7:01 AM, ?ric Depagne <eric at depagne.org> wrote:
> >> Is Linux, then, the main problem as far as "easy Python
> >> [+numpy/scipy/matplotlib] installations" go?
> > 
> > To my knowledge of various distributions, installing python/numpy/scipy
> > is very easy when running linux, it's just a matter of asking your
> > package installer to do so, and here it goes.
> > Of course, one has to know that it's done this way, but I suspect that
> > it's not the biggest hurdle (maybe I'm wrong, I do not know).
> 
> This requires root access.  At CfA all the system-managed linux
> machines (hundreds) run CentOS-5, which includes Python 2.4 as the
> native python.  The RPM installs for numpy / scipy will be ancient
> versions.  Using CentOS-5 (as I am forced to do) is a challenge since
> the libraries are likewise ancient.  Getting PyGTK to build from
> source was a major project in archaeology, trying to guess which
> versions of the gtk dependencies would build.
> 
Yes, you are right, the problem is that usually, institution have officially 
supported versions that are old. Andit's always difficult to deal with that, 
since it's very unlikely that this will change anytime soon (for good and bad 
reasons).

> The upshot is that to provide a universally applicable solution
> requires a non-root installable distribution.
> 
Or that we ask everyone to adminstrate their own machine (just kidding).

> I think everyone agrees that EPD itself is not acceptable because of
> license restrictions.  Probably 2/3 of people at CfA would not qualify
> under the academic license, myself included.

Most of the distribution provide enthought-packages so is it a real problem ? 
( that's a true question. I myself uses the mdv packages since I was never 
able ton install the epd itself on my cmoputer)

?ric.
> 
> - Tom A

-- 
Un clavier azerty en vaut deux
----------------------------------------------------------
?ric Depagne                            eric at depagne.org


From luigi at lambrate.inaf.it  Thu Jun 16 08:26:38 2011
From: luigi at lambrate.inaf.it (Luigi Paioro)
Date: Thu, 16 Jun 2011 14:26:38 +0200
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>	<C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>	<201106161301.27468.eric@depagne.org>
	<BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>
Message-ID: <4DF9F67E.6080303@lambrate.inaf.it>

Il 06/16/11 13:59, Tom Aldcroft ha scritto:
> On Thu, Jun 16, 2011 at 7:01 AM, ?ric Depagne<eric at depagne.org>  wrote:
>>> Is Linux, then, the main problem as far as "easy Python
>>> [+numpy/scipy/matplotlib] installations" go?
>> To my knowledge of various distributions, installing python/numpy/scipy is
>> very easy when running linux, it's just a matter of asking your package
>> installer to do so, and here it goes.
>> Of course, one has to know that it's done this way, but I suspect that it's
>> not the biggest hurdle (maybe I'm wrong, I do not know).
>>
>
> This requires root access.  At CfA all the system-managed linux
> machines (hundreds) run CentOS-5, which includes Python 2.4 as the
> native python.  The RPM installs for numpy / scipy will be ancient
> versions.  Using CentOS-5 (as I am forced to do) is a challenge since
> the libraries are likewise ancient.  Getting PyGTK to build from
> source was a major project in archaeology, trying to guess which
> versions of the gtk dependencies would build.


Gosh, PyGTK remembers me pangocairo and family... a nightmare!

My experience is that the most tricky libraries to be installed are just 
those related with a graphical interface system (we used Tk and GTK, but 
now we would prefer Qt): installing python, numpy and matplotlib is a 
minor problem (concerning scipy I'm not so sure it is so easy though).


> The upshot is that to provide a universally applicable solution
> requires a non-root installable distribution.

I agree.


> I think everyone agrees that EPD itself is not acceptable because of
> license restrictions.  Probably 2/3 of people at CfA would not qualify
> under the academic license, myself included.

It is not acceptable, but certainly comfortable.


Luigi


From mrdavis at stsci.edu  Thu Jun 16 08:37:01 2011
From: mrdavis at stsci.edu (Matt Davis)
Date: Thu, 16 Jun 2011 12:37:01 +0000
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>
	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
	<BANLkTimRu3_DjkA5dRuJEBKaU2OcNV10cg@mail.gmail.com>
	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
	<C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>
Message-ID: <90507CFD-B052-44BB-BE86-5BC2B2604850@stsci.edu>

> I'll confess I'm not sure why the standard Python distributions available at www.python.org
> aren't considered valid "easy Python installations" -- at least for Mac and Windows machines. I know 
> that the standard binary distributions of numpy, scipy, and matplotlib for Macs are designed to install
> into the www.python.org Python distributions (and I assume the same is true for the Windows
> binary versions of those packages). The www.python.org installations *are* the standard, up-to-date
> versions, after all, and they have no peculiar license restrictions.
> 
> Is Linux, then, the main problem as far as "easy Python [+numpy/scipy/matplotlib] installations" go?

I don't know why more people don't use the python.org installers, but as easy as it may seem to us, having to find and download 4 different Mac binaries to get Python/Numpy/Scipy/Matplotlib actually does discourage some people. Then they have to install ipython, easy_install/pip, file readers... Some good directions might help. That seems to be the BioPython method and as long as we're making an AstroPy website we might as well provide those good directions too.

As Tom notes there are users with restricted permissions who could be helped by a nice installer, and sys admins might appreciate it too.

- Matt

From thomas.robitaille at gmail.com  Thu Jun 16 08:37:05 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Thu, 16 Jun 2011 08:37:05 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
	<C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>
	<201106161301.27468.eric@depagne.org>
	<BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>
Message-ID: <BANLkTi=JxyZr9OA4eYGJ65aHyACo+Xb1Ng@mail.gmail.com>

I think it's time that we start formally keeping track of all these
options and pros/cons, so we can reach a consensus on what the best
ways are to get things out to the community (and there is nothing to
exclude multiple methods). I have created a wiki page that it would be
great if those of you interested in packaging could start filling out:

http://astropy.wikispaces.com/distribution

Of course, please feel free to continue the conversation by email too,
but it'd be nice to start to see some consensus on what the options
are (and not necessarily endorsement) and have a record of that.

Cheers,
Tom


From sergiopr at fis.ucm.es  Thu Jun 16 11:54:51 2011
From: sergiopr at fis.ucm.es (Sergio Pascual)
Date: Thu, 16 Jun 2011 17:54:51 +0200
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTi=JxyZr9OA4eYGJ65aHyACo+Xb1Ng@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
	<C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>
	<201106161301.27468.eric@depagne.org>
	<BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>
	<BANLkTi=JxyZr9OA4eYGJ65aHyACo+Xb1Ng@mail.gmail.com>
Message-ID: <BANLkTinPRt15enqDRhm9+fUuXHrrVmcOog@mail.gmail.com>

Hi all, just muy 2 cents from a linux packager (disclaimer: I create
packages for Fedora).

Most (if not all) linux distributions manage dependencies between
packages automatically. If you package needs Sextractor (for example),
it's downloaded and installed together with your package. That's not a
problem at all (of course, if your dependence is packaged also).

What is a problem is when upstream releases a package with bundled
libraries or other packages. Duplicated programs or libraries are
considered bad practice and not allowed in general, so packages that
come with it's own version of system libraries are patched to use the
system library.

>From my point of view, the problem of packaging is mostly solved, just
using the native tools for package management of the operating system.
I try to include new astronomy packages in Fedora (last one is
asciitable), but it takes some time.

For faster creation of linux packages we could use a service such as
build.opensuse.org. It allows to create packages for several linux
distributions.

For Mac I'm sure there are package management systems also. It's just
a a matter of creating the appropriated packages.

Regards, Sergio

-- 
Sergio Pascual ? ? ? ? ? ? ? ? ? ? ? ?http://guaix.fis.ucm.es/~spr
gpg fingerprint: 5203 B42D 86A0 5649 410A F4AC A35F D465 F263 BCCC
Departamento de Astrof?sica -- Universidad Complutense de Madrid (Spain)


From sienkiew at stsci.edu  Thu Jun 16 12:50:26 2011
From: sienkiew at stsci.edu (Mark Sienkiewicz)
Date: Thu, 16 Jun 2011 12:50:26 -0400
Subject: [AstroPy] ratholes
In-Reply-To: <BANLkTinPRt15enqDRhm9+fUuXHrrVmcOog@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>	<C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>	<201106161301.27468.eric@depagne.org>	<BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>	<BANLkTi=JxyZr9OA4eYGJ65aHyACo+Xb1Ng@mail.gmail.com>
	<BANLkTinPRt15enqDRhm9+fUuXHrrVmcOog@mail.gmail.com>
Message-ID: <4DFA3452.20202@stsci.edu>

I note that the discussion of packaging has completely displaced any 
discussion of what astropy is, what it does, or what might be included.

I also note that the packaging mechanism does not matter if we have no 
software to package.  Let me offer this:

- astropy can be installed with "python setup.py install".

- astropy can be installed with "pip install astropy".

- Anything that can be installed via those methods can also be 
distributed via _any_ of the other packaging mechanisms under 
discussion, subject to someone committing to routinely perform the 
packaging work at each release.

I suggest that we declare these three statements as "good enough for 
now" and get back to answering the question "What is astropy?"

Mark S.



From embray at stsci.edu  Thu Jun 16 13:01:59 2011
From: embray at stsci.edu (Erik Bray)
Date: Thu, 16 Jun 2011 13:01:59 -0400
Subject: [AstroPy] ratholes
In-Reply-To: <4DFA3452.20202@stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>	<C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>	<201106161301.27468.eric@depagne.org>	<BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>	<BANLkTi=JxyZr9OA4eYGJ65aHyACo+Xb1Ng@mail.gmail.com>	<BANLkTinPRt15enqDRhm9+fUuXHrrVmcOog@mail.gmail.com>
	<4DFA3452.20202@stsci.edu>
Message-ID: <4DFA3707.3060202@stsci.edu>

On 06/16/2011 12:50 PM, Mark Sienkiewicz wrote:
> I note that the discussion of packaging has completely displaced any
> discussion of what astropy is, what it does, or what might be included.
>
> I also note that the packaging mechanism does not matter if we have no
> software to package.  Let me offer this:
>
> - astropy can be installed with "python setup.py install".
>
> - astropy can be installed with "pip install astropy".
>
> - Anything that can be installed via those methods can also be
> distributed via _any_ of the other packaging mechanisms under
> discussion, subject to someone committing to routinely perform the
> packaging work at each release.
>
> I suggest that we declare these three statements as "good enough for
> now" and get back to answering the question "What is astropy?"

This is a point I tried to make at some point yesterday, only not as 
clearly :)  I think that such straightforward installation should be the 
goal here.  Beyond that there's really nothing more to say until we know 
what we're installing.

As for "what is astropy"--something I would like to add is that I agree 
with Luigi Paioro that a single "astropy" package is undoubtedly 
overkill.  It would be huge and difficult to maintain.  A more loosely 
coupled Scikits-like approach is better.  In this case "astropy" would 
be a namespace package, and one would almost never `pip install astropy` 
unless they really wanted *everything*.

Each sub-package would relate to astropy in so far as it follows the 
same coding standards as, and have interfaces similar to every other 
package in astropy.  Also with as little duplication of functionality as 
possible.

Erik


From lisa.winter at Colorado.EDU  Thu Jun 16 13:07:41 2011
From: lisa.winter at Colorado.EDU (Lisa M Winter)
Date: Thu, 16 Jun 2011 11:07:41 -0600
Subject: [AstroPy] Deployment and packaging
Message-ID: <C677FBE4-1E40-4579-9D2D-F0CA04936CF5@colorado.edu>

In response to the issue of ease of installation for numpy/scipy/ 
matplotlib on a Mac, this is not a trivial feat for the average user.   
You need to install the correct versions of C, fortran, etc., in order  
to get things to work properly.  I hadn't found any easy binary  
installs either.  While looking for installations for my undergrad  
research students this summer, we tried the standard installs, got  
frustrated, and turned to the extremely easy method of installing  
everything with Enthought (a packaged python distribution).  A decent  
set of instructions will be crucial for the packaging of the data so  
that standard users like myself can get things running smoothly  
without spending days trying to get everything set up (which happened  
the first few times I tried installing on my own).

Sincerely,
Lisa Winter


From thomas.robitaille at gmail.com  Thu Jun 16 13:15:25 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Thu, 16 Jun 2011 13:15:25 -0400
Subject: [AstroPy] ratholes
In-Reply-To: <4DFA3452.20202@stsci.edu>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<8BF90BBE-45D5-46E6-A1CF-60F84CACD3DD@stsci.edu>
	<C9F9AAF4-FB80-4E17-9490-8877ADA1B0E1@mpe.mpg.de>
	<201106161301.27468.eric@depagne.org>
	<BANLkTi=QwYV3JFUzfqk_2gym_Y2+TPc6rw@mail.gmail.com>
	<BANLkTi=JxyZr9OA4eYGJ65aHyACo+Xb1Ng@mail.gmail.com>
	<BANLkTinPRt15enqDRhm9+fUuXHrrVmcOog@mail.gmail.com>
	<4DFA3452.20202@stsci.edu>
Message-ID: <BANLkTimDD2SV-XYbYi5v-WK0mCu4Ko8HZA@mail.gmail.com>

I also agree with Mark.

As I mentioned before, I view these are completely unrelated
discussions. The packaging discussion is relevant for existing
packages, not just for this 'common package idea'. I tried to clarify
this at the start of the 'distribution' wiki page:

"This page keeps track of the various options with pros/cons for
packaging python Astronomy modules. The common astronomy package
discussed elsewhere should be installable in the standard Pythonic
way, so the packaging discussion can be kept separate: it applies
whether or not the common astronomy package sees the light of day.
There is nothing to exclude multiple distribution channels for
astronomy software (e.g. package managers, all-in-one), but at least
we should agree on what these options are."

So I think it's ok for people to discuss packaging options, and how to
help beginners get set up, so long as people realize this is separate
from the issue of a common Python package. In the end, the common
package will just be another standard Python package (or packages)
that can fit in to any of the distribution channels.

Once we know who's interested in moderating the discussions (after
this evening), we can start that discussion in earnest about what
astrolib/astropy should be and make progress in that direction.

Cheers,
Tom



On 16 June 2011 12:50, Mark Sienkiewicz <sienkiew at stsci.edu> wrote:
> I note that the discussion of packaging has completely displaced any
> discussion of what astropy is, what it does, or what might be included.
>
> I also note that the packaging mechanism does not matter if we have no
> software to package. ?Let me offer this:
>
> - astropy can be installed with "python setup.py install".
>
> - astropy can be installed with "pip install astropy".
>
> - Anything that can be installed via those methods can also be
> distributed via _any_ of the other packaging mechanisms under
> discussion, subject to someone committing to routinely perform the
> packaging work at each release.
>
> I suggest that we declare these three statements as "good enough for
> now" and get back to answering the question "What is astropy?"
>
> Mark S.
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From erwin at mpe.mpg.de  Thu Jun 16 13:33:20 2011
From: erwin at mpe.mpg.de (Peter Erwin)
Date: Thu, 16 Jun 2011 19:33:20 +0200
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <C677FBE4-1E40-4579-9D2D-F0CA04936CF5@colorado.edu>
References: <C677FBE4-1E40-4579-9D2D-F0CA04936CF5@colorado.edu>
Message-ID: <7E2114C7-8BCD-4650-9304-F1E1E61A5CA6@mpe.mpg.de>


On Jun 16, 2011, at 7:07 PM, Lisa M Winter wrote:

> In response to the issue of ease of installation for numpy/scipy/ 
> matplotlib on a Mac, this is not a trivial feat for the average user.   
> You need to install the correct versions of C, fortran, etc., in order  
> to get things to work properly.  I hadn't found any easy binary  
> installs either.  While looking for installations for my undergrad  
> research students this summer, we tried the standard installs, got  
> frustrated, and turned to the extremely easy method of installing  
> everything with Enthought (a packaged python distribution).  A decent  
> set of instructions will be crucial for the packaging of the data so  
> that standard users like myself can get things running smoothly  
> without spending days trying to get everything set up (which happened  
> the first few times I tried installing on my own).

I'm puzzled by this, since the current version of numpy, scipy, and matplotlib
all have standard binary installers in disk-image format for Mac OS X, none of which
require that you have C or Fortran compilers. I've had no trouble installing these on
my various Mac laptops.  E.g., the current release of numpy (1.6.0) has
the following three binary installers:
http://sourceforge.net/projects/numpy/files/NumPy/1.6.0/

numpy-1.6.0-py2.5-python.org-macosx10.3.dmg
numpy-1.6.0-py2.6-python.org-macosx10.3.dmg
numpy-1.6.0-py2.7-python.org-macosx10.3.dmg

depending on whether you want to install it into Python 2.5, 2.6, or 2.7.
The situtation is identical for scipy 0.9, and almost the same for matplotlib
(the latter doesn't have a Mac binary installer for Python 2.5).

(Things used to be much hairier, and I can remember pulling my hair out trying
to install matplotlib from source a few years ago, but the situation in the last year or so has been
pretty simple, in my experience.)


They *do* assume that you have the standard www.python.org "framework"
versions of Python installed -- the ones you find at

http://www.python.org/download/

The general consensus in the Python-on-Mac world is that those are the way to go, and
that you don't want to use the built-in (and outdated) versions of Python that come as part
of Mac OS X.

   -- Peter


=============================================================
Peter Erwin                   Max-Planck-Insitute for Extraterrestrial 
erwin at mpe.mpg.de              Physics, Giessenbachstrasse
tel. +49 (0)89 30000 3695     85748 Garching, Germany
fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin





From aldcroft at head.cfa.harvard.edu  Thu Jun 16 13:56:13 2011
From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft)
Date: Thu, 16 Jun 2011 13:56:13 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <7E2114C7-8BCD-4650-9304-F1E1E61A5CA6@mpe.mpg.de>
References: <C677FBE4-1E40-4579-9D2D-F0CA04936CF5@colorado.edu>
	<7E2114C7-8BCD-4650-9304-F1E1E61A5CA6@mpe.mpg.de>
Message-ID: <BANLkTi=f8p2criK0Ph3Kff4=rZqgP43hNA@mail.gmail.com>

On Thu, Jun 16, 2011 at 1:33 PM, Peter Erwin <erwin at mpe.mpg.de> wrote:
>
> On Jun 16, 2011, at 7:07 PM, Lisa M Winter wrote:
>
>> In response to the issue of ease of installation for numpy/scipy/
>> matplotlib on a Mac, this is not a trivial feat for the average user.
>> You need to install the correct versions of C, fortran, etc., in order
>> to get things to work properly. ?I hadn't found any easy binary
>> installs either. ?While looking for installations for my undergrad
>> research students this summer, we tried the standard installs, got
>> frustrated, and turned to the extremely easy method of installing
>> everything with Enthought (a packaged python distribution). ?A decent
>> set of instructions will be crucial for the packaging of the data so
>> that standard users like myself can get things running smoothly
>> without spending days trying to get everything set up (which happened
>> the first few times I tried installing on my own).
>
> I'm puzzled by this, since the current version of numpy, scipy, and matplotlib
> all have standard binary installers in disk-image format for Mac OS X, none of which
> require that you have C or Fortran compilers. I've had no trouble installing these on
> my various Mac laptops. ?E.g., the current release of numpy (1.6.0) has
> the following three binary installers:
> http://sourceforge.net/projects/numpy/files/NumPy/1.6.0/

The problem is the blizzard of available options and conflicting
opinions.  Developers and experts have no trouble sorting through the
options and finding a clean solution that works for them on their
computer.  But Lisa's perspective and troubles need to be taken
seriously as the reality of astronomers coming to Python.  Clear
instructions are very much needed.

- Tom A


From perry at stsci.edu  Thu Jun 16 13:59:35 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Thu, 16 Jun 2011 13:59:35 -0400
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <BANLkTimRu3_DjkA5dRuJEBKaU2OcNV10cg@mail.gmail.com>
References: <BANLkTi=JNYrs4f90u0hTMPB3qAyxFi5SQw@mail.gmail.com>
	<C78D98B3-9BB6-4F6F-A770-BF7CA4556078@stsci.edu>
	<EF6534DD-9A89-424F-9B5E-48C627BC537B@mac.com>
	<3FD70BF1-C767-4A29-870A-AE6E41D02F4D@stsci.edu>
	<8C2369A6-9D54-4FA2-BCA5-B83420B4A09E@mac.com>
	<4DF82EA8.5010105@physics.uwa.edu.au>
	<173C0F068F6E49449797E90DF0B59EF8B709C3@EXCHMAIL1.stsci.edu>
	<BANLkTimRu3_DjkA5dRuJEBKaU2OcNV10cg@mail.gmail.com>
Message-ID: <3D23E5D5-F3B4-4764-9307-87DE5009A5F5@stsci.edu>

Some points I would like to make

1) It's clear that deployment and packaging should not be directly  
tied to the effort to develop more general and consistent astronomical  
modules for Python. This  really is a different issue than the other  
effort and nothing about this should impede that.

2) I'd say that I think all (or almost all, anyway) agree that these  
modules should support standard Python installation schemes (e.g.,  
PyPI support), and that one should not have to use some monolithic  
binary install or anything like that if one doesn't want to install  
them that way. In some respects, this is an extension of point 1.

3) It's good to understand that the typical expert on this list  
doesn't represent the average user that might want to use this  
software. Typically those participating in this discussion have  
significantly more expertise in doing software installations than the  
typical user does.

4) It's also good to understand that many users out there don't have  
root access, and don't necessary have support staff to do what they  
would like as far as root installations as Tom mentions. It's true at  
CFA and STScI, for example. This means that tools that may work  
wonderfully for you, such as RPMs, are useless to these people. Some  
have railed at the stupidity of not allowing them root, it's a fact of  
life at a number of institutions, and it isn't likely to change.

5) Besides the root issue, what works well for you, doesn't  
necessarily work well for everyone else. There is a great variety of  
systems out there that confound standard installations. Unless you  
have experience distributing software that has lots of dependences,  
you may not be aware of all the problems you may run into. As some  
have mentioned, most users will give up at the first installation  
problem. The greater the number of separate installations one has to  
do, the greater the variety of the installation scheme (from make  
variants, to distutils, to...) the more likely problems are to occur  
and the greater the likelihood of the user giving up.

6) With the standard, individualized installations one can run into  
dependency conflicts where the different packages have inconsistent  
dependency requirements. Hopefully coordinated development can avoid  
this (though there may be occasions where it is unavoidable), but  
really what is needed is regular regression testing of the packages  
with each other. This is something we (STScI) are planning on as part  
of our distribution, but it is an activity that needs to be done in  
any event.

7) Speaking of dependencies, distribution systems out there (e.g.,  
RPMs) help, but they also are subject to dependency conflicts. This is  
an area that really has no easy solution. The only existing, reliable  
solution is to control what versions are packaged together explicitly,  
and test them. This leads to two conflicting desires. The packaging  
and testing together takes time, and the more one includes, the more  
the effort (one reason why things like Linux distributions run behind  
in versions by substantial amounts). It's reliability and consistency  
vs. how current it is.

8) The scisoft distribution has some issues that people have pointed  
out already. But a lots of people have used it just because it is so  
simple to install. I have been asked often to support something like  
that for our software. But it isn't easy to update items and that a  
significant problem.

9) Is a larger package a real problem for people? Suppose it includes  
X, Y, and Z for which you have no interest in having. So long as they  
don't interfere with the installation, or break software you do care  
about, why do you care if there are there? (so long as the space  
required isn't massive or the download time isn't horrible).

10) One objection is a big binary install isn't flexible. You get what  
you get, and you can't  update it. We do plan to make it possible to  
update modules within it, and add modules to it. It is true that doing  
so uses a different scheme than the standard approach and it is more  
work. But...

My final point is: So what alternate packaged binary (or similar)  
solution that exists now that we should be using that doesn't require  
root, and doesn't require N different versions for M platforms? My  
impression from all the discussion that there isn't one. It isn't like  
we are going our own way when there is already a good solution out  
there. People don't like (for good reasons): scisoft, EPD, Sage,  
etc... So why object to trying something else? If it doesn't work, it  
won't hurt the standard way of installing software, will it? If it  
does work, it gives those that don't want to manage 18 separate  
installations something that should work in the majority of cases with  
little pain.

Perry









From lisa.winter at Colorado.EDU  Thu Jun 16 14:00:20 2011
From: lisa.winter at Colorado.EDU (Lisa M Winter)
Date: Thu, 16 Jun 2011 12:00:20 -0600
Subject: [AstroPy] Deployment and packaging
Message-ID: <6D354D08-EFCD-4CB9-8F06-B770BD6DEF30@colorado.edu>

Peter, it looks like you are using Mac OS 10.3?  I hadn't found any  
binary installations that work with Snowleopard... and now of course  
we will need to worry about Lion.  Then there was the issue of how  
this all works on a 64-bit machine.

There is a tutorial on: http://blog.hyperjeff.net/?p=160, which I had  
tried.  However, I ended up with errors in several points going  
through this.

I think that the average user is likely to see a website like this and  
give up at some point through the process if they get too many errors  
and continue to use whatever package they are used to (e.g., IDL).

Also, I agree with Brad, matplotlib was difficult to install, until  
you use a distribution like Enthought that just gets everything  
working.  I didn't like using Scisoft, however, because it installs  
multiple copies of some applications I've already installed (like iraf/ 
pyraf).  Plus, I'm not sure all the distributions are up to date.

Lisa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110616/70a4fde7/attachment.html>

From erwin at mpe.mpg.de  Thu Jun 16 14:14:15 2011
From: erwin at mpe.mpg.de (Peter Erwin)
Date: Thu, 16 Jun 2011 20:14:15 +0200
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <6D354D08-EFCD-4CB9-8F06-B770BD6DEF30@colorado.edu>
References: <6D354D08-EFCD-4CB9-8F06-B770BD6DEF30@colorado.edu>
Message-ID: <14B30E39-66D2-4399-A840-F8EBD467AC27@mpe.mpg.de>


On Jun 16, 2011, at 8:00 PM, Lisa M Winter wrote:

> Peter, it looks like you are using Mac OS 10.3?  I hadn't found any binary installations that work with Snowleopard... and now of course we will need to worry about Lion.  Then there was the issue of how this all works on a 64-bit machine.
> 

Ah, I see the problem. The naming scheme for those installer disk images is confusing; the "10.3" 
really means "10.3 and more recent" -- so they're meant to be installed on Macs running OS X 10.3, 
10.4, 10.5, *or* 10.6. They *are* meant to work with Snow Leopard (and earlier versions).
(I'm using 10.6.7 on my two main laptops, with an older laptop that's still running 10.5.)

The 64-bit issue *is* a bit of a problem; for the time being, I'm keeping all my Python 2.x installations
32-bit only, at least on my Macs. (In the Mac OS X world, the 32-bit versions of Python work just fine on 64-bit
*machines*; you only run into problems if you try to, e.g., install a 64-bit version of Numpy into a 32-bit-only
version of Python.)  So I'd steer clear of the "Building 64bit NumPy/SciPy/PyLab on Snow Leopard" idea
unless you *really* need to access > 4 GB of memory within your Python session.

At www.python.org, you have the option of 32-bit-only versions of Python 2.7 -- which can be installed
on Mac OS X 10.3 through 10.6 -- or a "fat" 32-bit/64-bit version, which can only be installed on 10.6.
Personally, I'm sticking with the 32-bit-only version.


> There is a tutorial on: http://blog.hyperjeff.net/?p=160, which I had tried.  However, I ended up with errors in several points going through this.
> 
> I think that the average user is likely to see a website like this and give up at some point through the process if they get too many errors and continue to use whatever package they are used to (e.g., IDL).
> 
> Also, I agree with Brad, matplotlib was difficult to install, until you use a distribution like Enthought that just gets everything working.  I didn't like using Scisoft, however, because it installs multiple copies of some applications I've already installed (like iraf/pyraf).  Plus, I'm not sure all the distributions are up to date.

I've added some text to the astropy-distribution wiki page concerning Scisoft. It's *not* very up-to-date;
the official ESO version (Fedora Linux only) only has Python 2.5.4 (plus numpy 1.3, scipy 0.7, et.c), while the
unofficial Mac OS X version now has Python 2.6 (though I'm not sure how you're supposed to know this
without downloading the entire installer and going through its contents, like I just did...).

I share your dislike of Scisoft; we've had some problems with the Linux version on our local
Linux boxes, having to do with outdated libraries.

   -- Peter

=============================================================
Peter Erwin                   Max-Planck-Insitute for Extraterrestrial 
erwin at mpe.mpg.de              Physics, Giessenbachstrasse
tel. +49 (0)89 30000 3695     85748 Garching, Germany
fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin





From holden at ucolick.org  Thu Jun 16 14:32:39 2011
From: holden at ucolick.org (Brad Holden)
Date: Thu, 16 Jun 2011 11:32:39 -0700
Subject: [AstroPy] Deployment and packaging
In-Reply-To: <14B30E39-66D2-4399-A840-F8EBD467AC27@mpe.mpg.de>
References: <6D354D08-EFCD-4CB9-8F06-B770BD6DEF30@colorado.edu>
	<14B30E39-66D2-4399-A840-F8EBD467AC27@mpe.mpg.de>
Message-ID: <C2A7BE8F-4259-4146-AB47-7ACB08220BF2@ucolick.org>

The MacOSX version of scisoft is independent of the ESO one.  
For example, it has 1.5.1 of numpy.

That said, it does bring another 700 Mb of stuff with it that you may not want.
(It is how I install IRAF, so that part I do not mind.)


On Jun 16, 2011, at 11:14 AM, Peter Erwin wrote:

> 
> On Jun 16, 2011, at 8:00 PM, Lisa M Winter wrote:
> 
>> Peter, it looks like you are using Mac OS 10.3?  I hadn't found any binary installations that work with Snowleopard... and now of course we will need to worry about Lion.  Then there was the issue of how this all works on a 64-bit machine.
>> 
> 
> Ah, I see the problem. The naming scheme for those installer disk images is confusing; the "10.3" 
> really means "10.3 and more recent" -- so they're meant to be installed on Macs running OS X 10.3, 
> 10.4, 10.5, *or* 10.6. They *are* meant to work with Snow Leopard (and earlier versions).
> (I'm using 10.6.7 on my two main laptops, with an older laptop that's still running 10.5.)
> 
> The 64-bit issue *is* a bit of a problem; for the time being, I'm keeping all my Python 2.x installations
> 32-bit only, at least on my Macs. (In the Mac OS X world, the 32-bit versions of Python work just fine on 64-bit
> *machines*; you only run into problems if you try to, e.g., install a 64-bit version of Numpy into a 32-bit-only
> version of Python.)  So I'd steer clear of the "Building 64bit NumPy/SciPy/PyLab on Snow Leopard" idea
> unless you *really* need to access > 4 GB of memory within your Python session.
> 
> At www.python.org, you have the option of 32-bit-only versions of Python 2.7 -- which can be installed
> on Mac OS X 10.3 through 10.6 -- or a "fat" 32-bit/64-bit version, which can only be installed on 10.6.
> Personally, I'm sticking with the 32-bit-only version.
> 
> 
>> There is a tutorial on: http://blog.hyperjeff.net/?p=160, which I had tried.  However, I ended up with errors in several points going through this.
>> 
>> I think that the average user is likely to see a website like this and give up at some point through the process if they get too many errors and continue to use whatever package they are used to (e.g., IDL).
>> 
>> Also, I agree with Brad, matplotlib was difficult to install, until you use a distribution like Enthought that just gets everything working.  I didn't like using Scisoft, however, because it installs multiple copies of some applications I've already installed (like iraf/pyraf).  Plus, I'm not sure all the distributions are up to date.
> 
> I've added some text to the astropy-distribution wiki page concerning Scisoft. It's *not* very up-to-date;
> the official ESO version (Fedora Linux only) only has Python 2.5.4 (plus numpy 1.3, scipy 0.7, et.c), while the
> unofficial Mac OS X version now has Python 2.6 (though I'm not sure how you're supposed to know this
> without downloading the entire installer and going through its contents, like I just did...).
> 
> I share your dislike of Scisoft; we've had some problems with the Linux version on our local
> Linux boxes, having to do with outdated libraries.
> 
>   -- Peter
> 
> =============================================================
> Peter Erwin                   Max-Planck-Insitute for Extraterrestrial 
> erwin at mpe.mpg.de              Physics, Giessenbachstrasse
> tel. +49 (0)89 30000 3695     85748 Garching, Germany
> fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin
> 
> 
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
> 
> !DSPAM:859,4dfa47fe17433315015342!
> 

Bradford Holden
UCO/Lick Observatory
Santa Cruz, CA 95064
1-831-459-3387
www.ucolick.org/~holden









From magnusp at snm.ku.dk  Thu Jun 16 14:42:55 2011
From: magnusp at snm.ku.dk (Magnus V Persson)
Date: Thu, 16 Jun 2011 20:42:55 +0200
Subject: [AstroPy] unsubscribe
References: <mailman.1381.1308248044.1086.astropy@scipy.org>
Message-ID: <0F0DE1E3EF57814CAC181AC227738A61AD6F67@MAILSERVER2.akizci.ku.dk>

A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 1880 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20110616/99d5dc1f/attachment.bin>

From thomas.robitaille at gmail.com  Fri Jun 17 00:22:59 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Fri, 17 Jun 2011 00:22:59 -0400
Subject: [AstroPy] POLL: moderation/coordination
Message-ID: <BANLkTikDkrgTMrMsUzz9xBd=6KtMhO_W-Q@mail.gmail.com>

Hi everyone,

Erik, Perry, and myself all volunteered to moderate the discussions
and coordinate the efforts towards a common Python astronomy package.
While the original intent was to have a poll to pick a single person,
we would like to suggest that we try operating as as a group of three
instead. We want to check whether this is ok with you, so we've made a
poll, which is located near the top of the developers page:

http://astropy.wikispaces.com/developers

and will remain open until Friday 17th at 9pm EST. Let us know what you think!

Cheers,
Thomas


From stevan.white at googlemail.com  Mon Jun 20 05:44:16 2011
From: stevan.white at googlemail.com (Steve White)
Date: Mon, 20 Jun 2011 11:44:16 +0200
Subject: [AstroPy] tweakshifts in writeCoordFile()--IndexError: invalid
	index to scalar variable
Message-ID: <BANLkTinUEbo_9dx-CxQrikxhxday_0q7Ow@mail.gmail.com>

Hi!

I'm trying to create a shifts file along the lines of the
	Multidrizzle Handbook 5.5.3.6 Tweakshifts
	http://www.stsci.edu/hst/HST_overview/documents/multidrizzle/ch56.html#251052

Tweakshifts gets stuck in a strange place deep in the python code.
I can't tell if I'm missing some step or if there is a bug in the code.

Iraf 2.14.1 is installed (but also tried 2.15.a1) with stsdas and
tables v 3.13 and pytools SVN revision 11593.
Here, for example, the input files are from HST proposal 6431.

First I determined which files correspond to a single pointing, and feed that
list into
	asnt = asnutil.ASNTable( files, output = out_prefix ),
        asnt.create()
        asnt.write()
to create asn table files.

When I run tweakshifts on the ASN files thus produced,
	--> tweakshifts u34l390250_asn.fits undistort=yes mode=h
it does some stuff then dies, as listed below.

I also attach the ASN file (which seems to consist of mixed text and binary).

Thanks!
----------------------------------------------------------
--> tweakshifts u34l390250_asn.fits undistort=yes mode=h
xyxin:  INDEF    xyyin:  INDEF
fitgeometry:  rscale
Tweakshifts Version  0.7.3 (19 Aug 2010)

+ MAKEWCS Version 1.1.7 (6 Jul 2010)
-Updating image  u34l3901m_c0m.fits[sci,1]
- IDCTAB: Distortion model from row 25 for chip 1 : F555W and CLEAR
- OFFTAB: Offset interpolated from rows 9 and 13
- IDCTAB: Distortion model from row 165 for chip 3 : F555W and CLEAR
- OFFTAB: Offset interpolated from rows 11 and 15
...
Setting up output name:  u34l390250_drz.fits
Starting PyDrizzle Version  6.3.7 (3-Jan-2011)  at  10:02:52 (20/06/2011)
- IDCTAB: Distortion model from row 25 for chip 1 : F555W and CLEAR
- OFFTAB: Offset interpolated from rows 9 and 13
- IDCTAB: Distortion model from row 95 for chip 2 : F555W and CLEAR
...
Drizzle parameters have been calculated. Ready to .run()...
Finished calculating parameters at  10:02:55 (20/06/2011)

> WARNING: Pixel stack overflow at position 493,431

    Computing undistorted target positions for  322 objects.
...
    Computing undistorted target positions for  1 objects.
Traceback (innermost last):
  File "<CL script CL1>", line 1, in <module>
  File "/opt/iraf/extern/stsdas/pkg/analysis/dither/tweakshifts_iraf.py",
line 217, in tweak_iraf
    fitbox        = _fitbox)
  File "/opt/iraf/extern/stsdas/python/tweak.py", line 625, in run
    shift_dict = runCatalog(flist,refname,match_pars)
  File "/opt/iraf/extern/stsdas/python/tweak.py", line 861, in runCatalog
    writeCoordFile(cxylist, fluxes, coordname)
  File "/opt/iraf/extern/stsdas/python/tweak.py", line 252, in writeCoordFile
    cstr = str(coord[0])+'    '+str(coord[1])+'    '+str(flux) +'\n'
IndexError: invalid index to scalar variable.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: u34l390250_asn.fits.gz
Type: application/x-gzip
Size: 1324 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20110620/abce4e0b/attachment.bin>

From thomas.robitaille at gmail.com  Mon Jun 20 16:19:34 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Mon, 20 Jun 2011 21:19:34 +0100
Subject: [AstroPy] POLL: moderation/coordination
In-Reply-To: <BANLkTikDkrgTMrMsUzz9xBd=6KtMhO_W-Q@mail.gmail.com>
References: <BANLkTikDkrgTMrMsUzz9xBd=6KtMhO_W-Q@mail.gmail.com>
Message-ID: <BANLkTimsm9G1xuHiB=mL-Xg_EHK2UwaOXg@mail.gmail.com>

Hi everyone,

The results of the poll as of 9pm EST last Friday were 53 votes in
favor, and 2 votes against, so Erik, Perry and I are now making sure
we agree on how to proceed next, and we will send out an email this
week to get things started. Thanks for voting!

Cheers,
Thomas

On 17 June 2011 05:22, Thomas Robitaille <thomas.robitaille at gmail.com> wrote:
> Hi everyone,
>
> Erik, Perry, and myself all volunteered to moderate the discussions
> and coordinate the efforts towards a common Python astronomy package.
> While the original intent was to have a poll to pick a single person,
> we would like to suggest that we try operating as as a group of three
> instead. We want to check whether this is ok with you, so we've made a
> poll, which is located near the top of the developers page:
>
> http://astropy.wikispaces.com/developers
>
> and will remain open until Friday 17th at 9pm EST. Let us know what you think!
>
> Cheers,
> Thomas
>


From hack at stsci.edu  Wed Jun 22 10:43:04 2011
From: hack at stsci.edu (Warren J. Hack)
Date: Wed, 22 Jun 2011 10:43:04 -0400
Subject: [AstroPy] tweakshifts in writeCoordFile()--IndexError: invalid
 index to scalar variable
Message-ID: <4E01FF78.8070203@stsci.edu>

Hello,

This error comes from a problem with how the coordinate lists are 
created within 'tweakshifts' and only shows up when there is only 1 
identified source in a chip given the object finding parameters provided 
when that task was run.  This task will be replaced shortly (end of this 
summer?) with a new Python, WCSLIB-based task, so no effort will be 
going into fixing any bugs found in using this task.  There are a couple 
of parameters in 'tweakshift' that provide some control over the object 
finding algorithm in 'daofind' (and 'sextractor', depending on which you 
use); namely, fwhmpsf, sigma, datamin, datamax, threshold and nsigma.  
Varying these can usually result in more than a single object being 
detected, assuming more than 1 object exists in the field of view at 
all.   This will then allow 'tweakshifts' to at least get past this 
problem.

As a side note, there are many other ways of specifying the names of the 
input images other than building an association (ASN) table, most of 
which are easier to use and manage.  Please review the help provided for 
this task, something which is made easiest by using the EPAR GUI and 
clicking on the 'tweakshift help' button at the top.  For example, you 
can run 'tweakshifts' with '*flt.fits' as the input. The ASN table 
actually gets stored as a binary FITS table for cases where you want to 
keep track of the shifts computed in the same file, rather relying on 
than the more convenient ASCII shift file that gets written by 
'tweakshifts'.

If you have any further questions or problems with this task, please 
feel free to email them 'help at stsci.edu' and I will be glad to continue 
working with you to help you use this task as successfully as possible.

Warren Hack
Science Software Branch
Space Telescope Science Institute

> From: Steve White <stevan.white at googlemail.com>
> Date: June 20, 2011 5:44:16 AM EDT
> To: <astropy at scipy.org>
> Subject: [AstroPy] tweakshifts in writeCoordFile()--IndexError: 
> invalid index to scalar variable
>
> Hi!
>
> I'm trying to create a shifts file along the lines of the
>     Multidrizzle Handbook 5.5.3.6 Tweakshifts
> http://www.stsci.edu/hst/HST_overview/documents/multidrizzle/ch56.html#251052 
>
>
> Tweakshifts gets stuck in a strange place deep in the python code.
> I can't tell if I'm missing some step or if there is a bug in the code.
>
> Iraf 2.14.1 is installed (but also tried 2.15.a1) with stsdas and
> tables v 3.13 and pytools SVN revision 11593.
> Here, for example, the input files are from HST proposal 6431.
>
> First I determined which files correspond to a single pointing, and 
> feed that
> list into
>     asnt = asnutil.ASNTable( files, output = out_prefix ),
>        asnt.create()
>        asnt.write()
> to create asn table files.
>
> When I run tweakshifts on the ASN files thus produced,
>     --> tweakshifts u34l390250_asn.fits undistort=yes mode=h
> it does some stuff then dies, as listed below.
>
> I also attach the ASN file (which seems to consist of mixed text and 
> binary).
>
> Thanks!
> ----------------------------------------------------------
> --> tweakshifts u34l390250_asn.fits undistort=yes mode=h
> xyxin:  INDEF    xyyin:  INDEF
> fitgeometry:  rscale
> Tweakshifts Version  0.7.3 (19 Aug 2010)
>
> + MAKEWCS Version 1.1.7 (6 Jul 2010)
> -Updating image  u34l3901m_c0m.fits[sci,1]
> - IDCTAB: Distortion model from row 25 for chip 1 : F555W and CLEAR
> - OFFTAB: Offset interpolated from rows 9 and 13
> - IDCTAB: Distortion model from row 165 for chip 3 : F555W and CLEAR
> - OFFTAB: Offset interpolated from rows 11 and 15
> ...
> Setting up output name:  u34l390250_drz.fits
> Starting PyDrizzle Version  6.3.7 (3-Jan-2011)  at  10:02:52 (20/06/2011)
> - IDCTAB: Distortion model from row 25 for chip 1 : F555W and CLEAR
> - OFFTAB: Offset interpolated from rows 9 and 13
> - IDCTAB: Distortion model from row 95 for chip 2 : F555W and CLEAR
> ...
> Drizzle parameters have been calculated. Ready to .run()...
> Finished calculating parameters at  10:02:55 (20/06/2011)
>
>> WARNING: Pixel stack overflow at position 493,431
>
>    Computing undistorted target positions for  322 objects.
> ...
>    Computing undistorted target positions for  1 objects.
> Traceback (innermost last):
>  File "<CL script CL1>", line 1, in <module>
>  File "/opt/iraf/extern/stsdas/pkg/analysis/dither/tweakshifts_iraf.py",
> line 217, in tweak_iraf
>    fitbox        = _fitbox)
>  File "/opt/iraf/extern/stsdas/python/tweak.py", line 625, in run
>    shift_dict = runCatalog(flist,refname,match_pars)
>  File "/opt/iraf/extern/stsdas/python/tweak.py", line 861, in runCatalog
>    writeCoordFile(cxylist, fluxes, coordname)
>  File "/opt/iraf/extern/stsdas/python/tweak.py", line 252, in 
> writeCoordFile
>    cstr = str(coord[0])+'    '+str(coord[1])+'    '+str(flux) +'\n'
> IndexError: invalid index to scalar variable.
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110622/a86e0421/attachment.html>

From thomas.robitaille at gmail.com  Mon Jun 27 16:16:43 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Mon, 27 Jun 2011 22:16:43 +0200
Subject: [AstroPy] POLL: vision for a common Astronomy package
Message-ID: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>

Hi everyone,

In the last week, Erik, Perry, and myself have been discussing how
best to coordinate the development of the common Python Astronomy
package. We have now converged on a common vision, and would now like
to know whether you would be happy with it too. The vision and a
poll are available at the following pages:

vision: http://astropy.wikispaces.com/vision

poll: http://astropy.wikispaces.com/vision-polls

In addition, 'astropy' has been suggested by several people as a name
for this common package, so rather than creating a multi-option poll,
we've created a simple yes/no poll to find out whether you would agree
with this name. The idea is to have a name that does not endorse any
specific existing project, is in line with numpy/scipy, and reflects
its initial development via this mailing list. The poll is located at
the same URL as before.

The polls will be open unti Friday 1st July at 9pm EST.

Thanks!
Thomas


From thomas.robitaille at gmail.com  Mon Jun 27 16:22:06 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Mon, 27 Jun 2011 22:22:06 +0200
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
Message-ID: <BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>

Just a quick note to avoid any confusion - the two polls are
independent, so you will need to click 'vote' for each one in turn.

Thanks,
Tom

On 27 June 2011 22:16, Thomas Robitaille <thomas.robitaille at gmail.com> wrote:
> Hi everyone,
>
> In the last week, Erik, Perry, and myself have been discussing how
> best to coordinate the development of the common Python Astronomy
> package. We have now converged on a common vision, and would now like
> to know whether you would be happy with it too. The vision and a
> poll are available at the following pages:
>
> vision: http://astropy.wikispaces.com/vision
>
> poll: http://astropy.wikispaces.com/vision-polls
>
> In addition, 'astropy' has been suggested by several people as a name
> for this common package, so rather than creating a multi-option poll,
> we've created a simple yes/no poll to find out whether you would agree
> with this name. The idea is to have a name that does not endorse any
> specific existing project, is in line with numpy/scipy, and reflects
> its initial development via this mailing list. The poll is located at
> the same URL as before.
>
> The polls will be open unti Friday 1st July at 9pm EST.
>
> Thanks!
> Thomas
>


From mperrin at stsci.edu  Mon Jun 27 16:47:31 2011
From: mperrin at stsci.edu (Marshall Perrin)
Date: Mon, 27 Jun 2011 20:47:31 +0000
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>,
	<BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>
Message-ID: <A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>


Structuring this as a binary yes/no vote seems a little restrictive to me. Is there any room for discussion or comments prior to this vote? I would have expected something a little more like Python's PEP workflow (http://www.python.org/dev/peps/pep-0001/#pep-work-flow) in which community feedback is applied to a document developed by one or more leaders. 


(Nothing in this email should be taken to imply approval or lack thereof of anything in the draft text. I'm more just stating my own opinion that binary votes are not necessarily the best decision making process for complex issues, where 'yes, but' or 'yes and also' or 'no unless' can at times be very valuable in drawing on the community's expertise...)

 - Marshall


________________________________________
From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Thomas Robitaille [thomas.robitaille at gmail.com]
Sent: Monday, June 27, 2011 4:22 PM
To: astropy at scipy.org
Subject: Re: [AstroPy] POLL: vision for a common Astronomy package

Just a quick note to avoid any confusion - the two polls are
independent, so you will need to click 'vote' for each one in turn.

Thanks,
Tom

On 27 June 2011 22:16, Thomas Robitaille <thomas.robitaille at gmail.com> wrote:
> Hi everyone,
>
> In the last week, Erik, Perry, and myself have been discussing how
> best to coordinate the development of the common Python Astronomy
> package. We have now converged on a common vision, and would now like
> to know whether you would be happy with it too. The vision and a
> poll are available at the following pages:
>
> vision: http://astropy.wikispaces.com/vision
>
> poll: http://astropy.wikispaces.com/vision-polls
>
> In addition, 'astropy' has been suggested by several people as a name
> for this common package, so rather than creating a multi-option poll,
> we've created a simple yes/no poll to find out whether you would agree
> with this name. The idea is to have a name that does not endorse any
> specific existing project, is in line with numpy/scipy, and reflects
> its initial development via this mailing list. The poll is located at
> the same URL as before.
>
> The polls will be open unti Friday 1st July at 9pm EST.
>
> Thanks!
> Thomas
>
_______________________________________________
AstroPy mailing list
AstroPy at scipy.org
http://mail.scipy.org/mailman/listinfo/astropy


From perry at stsci.edu  Mon Jun 27 17:04:12 2011
From: perry at stsci.edu (Perry Greenfield)
Date: Mon, 27 Jun 2011 17:04:12 -0400
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>,
	<BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>
	<A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>
Message-ID: <F8361E4B-75DD-4142-9C54-265F5751562B@stsci.edu>

Nothing stops people from making general comments.

Perry

On Jun 27, 2011, at 4:47 PM, Marshall Perrin wrote:

>
> Structuring this as a binary yes/no vote seems a little restrictive  
> to me. Is there any room for discussion or comments prior to this  
> vote? I would have expected something a little more like Python's  
> PEP workflow (http://www.python.org/dev/peps/pep-0001/#pep-work- 
> flow) in which community feedback is applied to a document developed  
> by one or more leaders.
>
>
> (Nothing in this email should be taken to imply approval or lack  
> thereof of anything in the draft text. I'm more just stating my own  
> opinion that binary votes are not necessarily the best decision  
> making process for complex issues, where 'yes, but' or 'yes and  
> also' or 'no unless' can at times be very valuable in drawing on the  
> community's expertise...)
>
> - Marshall
>
>
> ________________________________________
> From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on  
> behalf of Thomas Robitaille [thomas.robitaille at gmail.com]
> Sent: Monday, June 27, 2011 4:22 PM
> To: astropy at scipy.org
> Subject: Re: [AstroPy] POLL: vision for a common Astronomy package
>
> Just a quick note to avoid any confusion - the two polls are
> independent, so you will need to click 'vote' for each one in turn.
>
> Thanks,
> Tom
>
> On 27 June 2011 22:16, Thomas Robitaille  
> <thomas.robitaille at gmail.com> wrote:
>> Hi everyone,
>>
>> In the last week, Erik, Perry, and myself have been discussing how
>> best to coordinate the development of the common Python Astronomy
>> package. We have now converged on a common vision, and would now like
>> to know whether you would be happy with it too. The vision and a
>> poll are available at the following pages:
>>
>> vision: http://astropy.wikispaces.com/vision
>>
>> poll: http://astropy.wikispaces.com/vision-polls
>>
>> In addition, 'astropy' has been suggested by several people as a name
>> for this common package, so rather than creating a multi-option poll,
>> we've created a simple yes/no poll to find out whether you would  
>> agree
>> with this name. The idea is to have a name that does not endorse any
>> specific existing project, is in line with numpy/scipy, and reflects
>> its initial development via this mailing list. The poll is located at
>> the same URL as before.
>>
>> The polls will be open unti Friday 1st July at 9pm EST.
>>
>> Thanks!
>> Thomas
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy



From thomas.robitaille at gmail.com  Mon Jun 27 17:21:16 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Mon, 27 Jun 2011 23:21:16 +0200
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
	<BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>
	<A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>
Message-ID: <BANLkTimu_haNfuWvXLnU7pELPu6OKM=QwA@mail.gmail.com>

Hi Marshall,

Please do feel free to express any opinions you have in this thread.
The thinking behind the binary vote is that we just want something we
can start from. But once things start gaining momentum, there is no
reason why we couldn't adopt a PEP-like process to propose amendments
to the existing vision/plan :-)

Cheers,
Tom

On 27 June 2011 22:47, Marshall Perrin <mperrin at stsci.edu> wrote:
>
> Structuring this as a binary yes/no vote seems a little restrictive to me. Is there any room for discussion or comments prior to this vote? I would have expected something a little more like Python's PEP workflow (http://www.python.org/dev/peps/pep-0001/#pep-work-flow) in which community feedback is applied to a document developed by one or more leaders.
>
>
> (Nothing in this email should be taken to imply approval or lack thereof of anything in the draft text. I'm more just stating my own opinion that binary votes are not necessarily the best decision making process for complex issues, where 'yes, but' or 'yes and also' or 'no unless' can at times be very valuable in drawing on the community's expertise...)
>
> ?- Marshall
>
>
> ________________________________________
> From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Thomas Robitaille [thomas.robitaille at gmail.com]
> Sent: Monday, June 27, 2011 4:22 PM
> To: astropy at scipy.org
> Subject: Re: [AstroPy] POLL: vision for a common Astronomy package
>
> Just a quick note to avoid any confusion - the two polls are
> independent, so you will need to click 'vote' for each one in turn.
>
> Thanks,
> Tom
>
> On 27 June 2011 22:16, Thomas Robitaille <thomas.robitaille at gmail.com> wrote:
>> Hi everyone,
>>
>> In the last week, Erik, Perry, and myself have been discussing how
>> best to coordinate the development of the common Python Astronomy
>> package. We have now converged on a common vision, and would now like
>> to know whether you would be happy with it too. The vision and a
>> poll are available at the following pages:
>>
>> vision: http://astropy.wikispaces.com/vision
>>
>> poll: http://astropy.wikispaces.com/vision-polls
>>
>> In addition, 'astropy' has been suggested by several people as a name
>> for this common package, so rather than creating a multi-option poll,
>> we've created a simple yes/no poll to find out whether you would agree
>> with this name. The idea is to have a name that does not endorse any
>> specific existing project, is in line with numpy/scipy, and reflects
>> its initial development via this mailing list. The poll is located at
>> the same URL as before.
>>
>> The polls will be open unti Friday 1st July at 9pm EST.
>>
>> Thanks!
>> Thomas
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From jturner at gemini.edu  Mon Jun 27 18:51:15 2011
From: jturner at gemini.edu (James Turner)
Date: Mon, 27 Jun 2011 18:51:15 -0400
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
Message-ID: <4E090963.9030408@gemini.edu>

Sounds pretty good, but I'm balking a little bit at NO external
dependencies. Obviously we can't have an unmanageable proliferation
of them, but having none would seem to imply, for example, that we
can't include anything that uses a database -- unless we're going to
bundle the whole database into *each* "affiliated package" that needs
it. It's no good making that an optional import if the database is
central to what the module does. Perhaps one of you can elaborate a
bit on what you were thinking here? Are you expecting AstroPy only to
include lower-level algorithmic/library functionality? I'd have
thought there is also room for library routines that are closer to an
application level, which a user can interact with more directly.
Maybe I'm looking at this from the wrong angle though (it's quite
late here and I haven't thought about it that long yet)? I won't
suggest an alternative until I understand better what you had in mind.

Yes to calling it AstroPy.

Cheers,

James.


On 27/06/11 16:16, Thomas Robitaille wrote:
> Hi everyone,
>
> In the last week, Erik, Perry, and myself have been discussing how
> best to coordinate the development of the common Python Astronomy
> package. We have now converged on a common vision, and would now like
> to know whether you would be happy with it too. The vision and a
> poll are available at the following pages:
>
> vision: http://astropy.wikispaces.com/vision
>
> poll: http://astropy.wikispaces.com/vision-polls
>
> In addition, 'astropy' has been suggested by several people as a name
> for this common package, so rather than creating a multi-option poll,
> we've created a simple yes/no poll to find out whether you would agree
> with this name. The idea is to have a name that does not endorse any
> specific existing project, is in line with numpy/scipy, and reflects
> its initial development via this mailing list. The poll is located at
> the same URL as before.
>
> The polls will be open unti Friday 1st July at 9pm EST.
>
> Thanks!
> Thomas
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


-- 
James E.H. Turner
Gemini Observatory Southern Operations Centre,
Casilla 603,		Tel. (+56) 51 205609
La Serena, Chile.	Fax. (+56) 51 205650


From jturner at gemini.edu  Mon Jun 27 19:30:17 2011
From: jturner at gemini.edu (James Turner)
Date: Mon, 27 Jun 2011 19:30:17 -0400
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <4E090963.9030408@gemini.edu>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
	<4E090963.9030408@gemini.edu>
Message-ID: <4E091289.7050705@gemini.edu>

Just another (perhaps overly-obvious) thought. When you list the
"Python Standard Library", we probably need to decide what version(s)
of Python we're targeting (can I depend on a standard library module
that's only in Python 2.7?). Likewise for the other allowed
dependencies, otherwise people might depend on conflicting versions
or something.


On 27/06/11 18:51, James Turner wrote:
> Sounds pretty good, but I'm balking a little bit at NO external
> dependencies. Obviously we can't have an unmanageable proliferation
> of them, but having none would seem to imply, for example, that we
> can't include anything that uses a database -- unless we're going to
> bundle the whole database into *each* "affiliated package" that needs
> it. It's no good making that an optional import if the database is
> central to what the module does. Perhaps one of you can elaborate a
> bit on what you were thinking here? Are you expecting AstroPy only to
> include lower-level algorithmic/library functionality? I'd have
> thought there is also room for library routines that are closer to an
> application level, which a user can interact with more directly.
> Maybe I'm looking at this from the wrong angle though (it's quite
> late here and I haven't thought about it that long yet)? I won't
> suggest an alternative until I understand better what you had in mind.
>
> Yes to calling it AstroPy.
>
> Cheers,
>
> James.
>
>
> On 27/06/11 16:16, Thomas Robitaille wrote:
>> Hi everyone,
>>
>> In the last week, Erik, Perry, and myself have been discussing how
>> best to coordinate the development of the common Python Astronomy
>> package. We have now converged on a common vision, and would now like
>> to know whether you would be happy with it too. The vision and a
>> poll are available at the following pages:
>>
>> vision: http://astropy.wikispaces.com/vision
>>
>> poll: http://astropy.wikispaces.com/vision-polls
>>
>> In addition, 'astropy' has been suggested by several people as a name
>> for this common package, so rather than creating a multi-option poll,
>> we've created a simple yes/no poll to find out whether you would agree
>> with this name. The idea is to have a name that does not endorse any
>> specific existing project, is in line with numpy/scipy, and reflects
>> its initial development via this mailing list. The poll is located at
>> the same URL as before.
>>
>> The polls will be open unti Friday 1st July at 9pm EST.
>>
>> Thanks!
>> Thomas
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy


From matthewturk at gmail.com  Mon Jun 27 19:32:36 2011
From: matthewturk at gmail.com (Matthew Turk)
Date: Mon, 27 Jun 2011 16:32:36 -0700
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <4E091289.7050705@gemini.edu>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
	<4E090963.9030408@gemini.edu> <4E091289.7050705@gemini.edu>
Message-ID: <BANLkTi=+nvAgYQaRizNaH+kwsqtp=iKsrw@mail.gmail.com>

Agreed -- since dependencies are a focus of this vision statement, it
might also be worthwhile to state whether or not modules that are
conditionally compiled (sqlite, ctypes, tkinter, although that is
covered under "GUI" in the statement) in CPython are allowable for
dependencies.

-Matt

On Mon, Jun 27, 2011 at 4:30 PM, James Turner <jturner at gemini.edu> wrote:
> Just another (perhaps overly-obvious) thought. When you list the
> "Python Standard Library", we probably need to decide what version(s)
> of Python we're targeting (can I depend on a standard library module
> that's only in Python 2.7?). Likewise for the other allowed
> dependencies, otherwise people might depend on conflicting versions
> or something.
>
>
> On 27/06/11 18:51, James Turner wrote:
>> Sounds pretty good, but I'm balking a little bit at NO external
>> dependencies. Obviously we can't have an unmanageable proliferation
>> of them, but having none would seem to imply, for example, that we
>> can't include anything that uses a database -- unless we're going to
>> bundle the whole database into *each* "affiliated package" that needs
>> it. It's no good making that an optional import if the database is
>> central to what the module does. Perhaps one of you can elaborate a
>> bit on what you were thinking here? Are you expecting AstroPy only to
>> include lower-level algorithmic/library functionality? I'd have
>> thought there is also room for library routines that are closer to an
>> application level, which a user can interact with more directly.
>> Maybe I'm looking at this from the wrong angle though (it's quite
>> late here and I haven't thought about it that long yet)? I won't
>> suggest an alternative until I understand better what you had in mind.
>>
>> Yes to calling it AstroPy.
>>
>> Cheers,
>>
>> James.
>>
>>
>> On 27/06/11 16:16, Thomas Robitaille wrote:
>>> Hi everyone,
>>>
>>> In the last week, Erik, Perry, and myself have been discussing how
>>> best to coordinate the development of the common Python Astronomy
>>> package. We have now converged on a common vision, and would now like
>>> to know whether you would be happy with it too. The vision and a
>>> poll are available at the following pages:
>>>
>>> vision: http://astropy.wikispaces.com/vision
>>>
>>> poll: http://astropy.wikispaces.com/vision-polls
>>>
>>> In addition, 'astropy' has been suggested by several people as a name
>>> for this common package, so rather than creating a multi-option poll,
>>> we've created a simple yes/no poll to find out whether you would agree
>>> with this name. The idea is to have a name that does not endorse any
>>> specific existing project, is in line with numpy/scipy, and reflects
>>> its initial development via this mailing list. The poll is located at
>>> the same URL as before.
>>>
>>> The polls will be open unti Friday 1st July at 9pm EST.
>>>
>>> Thanks!
>>> Thomas
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From erik.tollerud at gmail.com  Tue Jun 28 00:55:12 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Mon, 27 Jun 2011 23:55:12 -0500
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <BANLkTi=+nvAgYQaRizNaH+kwsqtp=iKsrw@mail.gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
	<4E090963.9030408@gemini.edu> <4E091289.7050705@gemini.edu>
	<BANLkTi=+nvAgYQaRizNaH+kwsqtp=iKsrw@mail.gmail.com>
Message-ID: <BANLkTikExbj5RiVknDEx9fEQGnt5d7-d9A@mail.gmail.com>

To clarify, we're not intending  to completely disallow other
dependencies, but only stating that they should not be used
*initially* when the package is imported.  The idea is that someone
should be able to use everything that doesn't specifically require
another dependency without getting an ImportError.  If something
requires an extra dependency, it should only end up issuing a
dependency error when that particular functionality is needed.  For
example, "import astropy" and "import astropy.submodule" should always
work, but "import astropy.submodule.guiapp" or
"astropy.submodule.showgui()" should assue an ImportError if you don't
have the appropriate toolkit installed.

We also plan to keep watch on the external dependencies to make sure
they don't spiral out-of-control, but that will be done case-by-case.

You're both right that we should be a bit clearer about what parts of
the standard library, though.  My opinion is that we should definitely
count everything up to 2.6 (2.6 includes some absolutely crucial
features like ABCs), but 2.7 stdlib and anything that's optionally
included should be put in the same category as external deps
(shouldn't be necessary for the base imports).


On Mon, Jun 27, 2011 at 6:32 PM, Matthew Turk <matthewturk at gmail.com> wrote:
> Agreed -- since dependencies are a focus of this vision statement, it
> might also be worthwhile to state whether or not modules that are
> conditionally compiled (sqlite, ctypes, tkinter, although that is
> covered under "GUI" in the statement) in CPython are allowable for
> dependencies.
>
> -Matt
>
> On Mon, Jun 27, 2011 at 4:30 PM, James Turner <jturner at gemini.edu> wrote:
>> Just another (perhaps overly-obvious) thought. When you list the
>> "Python Standard Library", we probably need to decide what version(s)
>> of Python we're targeting (can I depend on a standard library module
>> that's only in Python 2.7?). Likewise for the other allowed
>> dependencies, otherwise people might depend on conflicting versions
>> or something.
>>
>>
>> On 27/06/11 18:51, James Turner wrote:
>>> Sounds pretty good, but I'm balking a little bit at NO external
>>> dependencies. Obviously we can't have an unmanageable proliferation
>>> of them, but having none would seem to imply, for example, that we
>>> can't include anything that uses a database -- unless we're going to
>>> bundle the whole database into *each* "affiliated package" that needs
>>> it. It's no good making that an optional import if the database is
>>> central to what the module does. Perhaps one of you can elaborate a
>>> bit on what you were thinking here? Are you expecting AstroPy only to
>>> include lower-level algorithmic/library functionality? I'd have
>>> thought there is also room for library routines that are closer to an
>>> application level, which a user can interact with more directly.
>>> Maybe I'm looking at this from the wrong angle though (it's quite
>>> late here and I haven't thought about it that long yet)? I won't
>>> suggest an alternative until I understand better what you had in mind.
>>>
>>> Yes to calling it AstroPy.
>>>
>>> Cheers,
>>>
>>> James.
>>>
>>>
>>> On 27/06/11 16:16, Thomas Robitaille wrote:
>>>> Hi everyone,
>>>>
>>>> In the last week, Erik, Perry, and myself have been discussing how
>>>> best to coordinate the development of the common Python Astronomy
>>>> package. We have now converged on a common vision, and would now like
>>>> to know whether you would be happy with it too. The vision and a
>>>> poll are available at the following pages:
>>>>
>>>> vision: http://astropy.wikispaces.com/vision
>>>>
>>>> poll: http://astropy.wikispaces.com/vision-polls
>>>>
>>>> In addition, 'astropy' has been suggested by several people as a name
>>>> for this common package, so rather than creating a multi-option poll,
>>>> we've created a simple yes/no poll to find out whether you would agree
>>>> with this name. The idea is to have a name that does not endorse any
>>>> specific existing project, is in line with numpy/scipy, and reflects
>>>> its initial development via this mailing list. The poll is located at
>>>> the same URL as before.
>>>>
>>>> The polls will be open unti Friday 1st July at 9pm EST.
>>>>
>>>> Thanks!
>>>> Thomas
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 
Erik Tollerud


From wkerzendorf at googlemail.com  Tue Jun 28 01:44:53 2011
From: wkerzendorf at googlemail.com (Wolfgang Kerzendorf)
Date: Tue, 28 Jun 2011 15:44:53 +1000
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <BANLkTikExbj5RiVknDEx9fEQGnt5d7-d9A@mail.gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>	<4E090963.9030408@gemini.edu>
	<4E091289.7050705@gemini.edu>	<BANLkTi=+nvAgYQaRizNaH+kwsqtp=iKsrw@mail.gmail.com>
	<BANLkTikExbj5RiVknDEx9fEQGnt5d7-d9A@mail.gmail.com>
Message-ID: <4E096A55.6040506@gmail.com>

I think we should write python 3 compliant code as most of it is written 
new anyways. Why not save us future pain.

Cheers
    W
On 28/06/11 2:55 PM, Erik Tollerud wrote:
> To clarify, we're not intending  to completely disallow other
> dependencies, but only stating that they should not be used
> *initially* when the package is imported.  The idea is that someone
> should be able to use everything that doesn't specifically require
> another dependency without getting an ImportError.  If something
> requires an extra dependency, it should only end up issuing a
> dependency error when that particular functionality is needed.  For
> example, "import astropy" and "import astropy.submodule" should always
> work, but "import astropy.submodule.guiapp" or
> "astropy.submodule.showgui()" should assue an ImportError if you don't
> have the appropriate toolkit installed.
>
> We also plan to keep watch on the external dependencies to make sure
> they don't spiral out-of-control, but that will be done case-by-case.
>
> You're both right that we should be a bit clearer about what parts of
> the standard library, though.  My opinion is that we should definitely
> count everything up to 2.6 (2.6 includes some absolutely crucial
> features like ABCs), but 2.7 stdlib and anything that's optionally
> included should be put in the same category as external deps
> (shouldn't be necessary for the base imports).
>
>
> On Mon, Jun 27, 2011 at 6:32 PM, Matthew Turk<matthewturk at gmail.com>  wrote:
>> Agreed -- since dependencies are a focus of this vision statement, it
>> might also be worthwhile to state whether or not modules that are
>> conditionally compiled (sqlite, ctypes, tkinter, although that is
>> covered under "GUI" in the statement) in CPython are allowable for
>> dependencies.
>>
>> -Matt
>>
>> On Mon, Jun 27, 2011 at 4:30 PM, James Turner<jturner at gemini.edu>  wrote:
>>> Just another (perhaps overly-obvious) thought. When you list the
>>> "Python Standard Library", we probably need to decide what version(s)
>>> of Python we're targeting (can I depend on a standard library module
>>> that's only in Python 2.7?). Likewise for the other allowed
>>> dependencies, otherwise people might depend on conflicting versions
>>> or something.
>>>
>>>
>>> On 27/06/11 18:51, James Turner wrote:
>>>> Sounds pretty good, but I'm balking a little bit at NO external
>>>> dependencies. Obviously we can't have an unmanageable proliferation
>>>> of them, but having none would seem to imply, for example, that we
>>>> can't include anything that uses a database -- unless we're going to
>>>> bundle the whole database into *each* "affiliated package" that needs
>>>> it. It's no good making that an optional import if the database is
>>>> central to what the module does. Perhaps one of you can elaborate a
>>>> bit on what you were thinking here? Are you expecting AstroPy only to
>>>> include lower-level algorithmic/library functionality? I'd have
>>>> thought there is also room for library routines that are closer to an
>>>> application level, which a user can interact with more directly.
>>>> Maybe I'm looking at this from the wrong angle though (it's quite
>>>> late here and I haven't thought about it that long yet)? I won't
>>>> suggest an alternative until I understand better what you had in mind.
>>>>
>>>> Yes to calling it AstroPy.
>>>>
>>>> Cheers,
>>>>
>>>> James.
>>>>
>>>>
>>>> On 27/06/11 16:16, Thomas Robitaille wrote:
>>>>> Hi everyone,
>>>>>
>>>>> In the last week, Erik, Perry, and myself have been discussing how
>>>>> best to coordinate the development of the common Python Astronomy
>>>>> package. We have now converged on a common vision, and would now like
>>>>> to know whether you would be happy with it too. The vision and a
>>>>> poll are available at the following pages:
>>>>>
>>>>> vision: http://astropy.wikispaces.com/vision
>>>>>
>>>>> poll: http://astropy.wikispaces.com/vision-polls
>>>>>
>>>>> In addition, 'astropy' has been suggested by several people as a name
>>>>> for this common package, so rather than creating a multi-option poll,
>>>>> we've created a simple yes/no poll to find out whether you would agree
>>>>> with this name. The idea is to have a name that does not endorse any
>>>>> specific existing project, is in line with numpy/scipy, and reflects
>>>>> its initial development via this mailing list. The poll is located at
>>>>> the same URL as before.
>>>>>
>>>>> The polls will be open unti Friday 1st July at 9pm EST.
>>>>>
>>>>> Thanks!
>>>>> Thomas
>>>>> _______________________________________________
>>>>> AstroPy mailing list
>>>>> AstroPy at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
>
>



From thomas.robitaille at gmail.com  Tue Jun 28 04:00:24 2011
From: thomas.robitaille at gmail.com (Thomas Robitaille)
Date: Tue, 28 Jun 2011 10:00:24 +0200
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <4E096A55.6040506@gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
	<4E090963.9030408@gemini.edu> <4E091289.7050705@gemini.edu>
	<BANLkTi=+nvAgYQaRizNaH+kwsqtp=iKsrw@mail.gmail.com>
	<BANLkTikExbj5RiVknDEx9fEQGnt5d7-d9A@mail.gmail.com>
	<4E096A55.6040506@gmail.com>
Message-ID: <BANLkTins3c6F4p_CKpiMF9NFqVfEwvOpdA@mail.gmail.com>

Thanks for all the feedback!

I agree that we'll probably want to require Python 2.6+ as well as
support Python 3. We deliberately left out specific coding
requirements like this for now, as there are many similar specific
issues to consider (docstring format, testing framework, C extensions,
Cython, etc.). We've been keeping a list, and if the vision is
approved and we can move ahead, then we can start listing coding
guidelines such as these on the wiki.

Cheers,
Tom

On 28 June 2011 07:44, Wolfgang Kerzendorf <wkerzendorf at googlemail.com> wrote:
> I think we should write python 3 compliant code as most of it is written
> new anyways. Why not save us future pain.
>
> Cheers
> ? ?W
> On 28/06/11 2:55 PM, Erik Tollerud wrote:
>> To clarify, we're not intending ?to completely disallow other
>> dependencies, but only stating that they should not be used
>> *initially* when the package is imported. ?The idea is that someone
>> should be able to use everything that doesn't specifically require
>> another dependency without getting an ImportError. ?If something
>> requires an extra dependency, it should only end up issuing a
>> dependency error when that particular functionality is needed. ?For
>> example, "import astropy" and "import astropy.submodule" should always
>> work, but "import astropy.submodule.guiapp" or
>> "astropy.submodule.showgui()" should assue an ImportError if you don't
>> have the appropriate toolkit installed.
>>
>> We also plan to keep watch on the external dependencies to make sure
>> they don't spiral out-of-control, but that will be done case-by-case.
>>
>> You're both right that we should be a bit clearer about what parts of
>> the standard library, though. ?My opinion is that we should definitely
>> count everything up to 2.6 (2.6 includes some absolutely crucial
>> features like ABCs), but 2.7 stdlib and anything that's optionally
>> included should be put in the same category as external deps
>> (shouldn't be necessary for the base imports).
>>
>>
>> On Mon, Jun 27, 2011 at 6:32 PM, Matthew Turk<matthewturk at gmail.com> ?wrote:
>>> Agreed -- since dependencies are a focus of this vision statement, it
>>> might also be worthwhile to state whether or not modules that are
>>> conditionally compiled (sqlite, ctypes, tkinter, although that is
>>> covered under "GUI" in the statement) in CPython are allowable for
>>> dependencies.
>>>
>>> -Matt
>>>
>>> On Mon, Jun 27, 2011 at 4:30 PM, James Turner<jturner at gemini.edu> ?wrote:
>>>> Just another (perhaps overly-obvious) thought. When you list the
>>>> "Python Standard Library", we probably need to decide what version(s)
>>>> of Python we're targeting (can I depend on a standard library module
>>>> that's only in Python 2.7?). Likewise for the other allowed
>>>> dependencies, otherwise people might depend on conflicting versions
>>>> or something.
>>>>
>>>>
>>>> On 27/06/11 18:51, James Turner wrote:
>>>>> Sounds pretty good, but I'm balking a little bit at NO external
>>>>> dependencies. Obviously we can't have an unmanageable proliferation
>>>>> of them, but having none would seem to imply, for example, that we
>>>>> can't include anything that uses a database -- unless we're going to
>>>>> bundle the whole database into *each* "affiliated package" that needs
>>>>> it. It's no good making that an optional import if the database is
>>>>> central to what the module does. Perhaps one of you can elaborate a
>>>>> bit on what you were thinking here? Are you expecting AstroPy only to
>>>>> include lower-level algorithmic/library functionality? I'd have
>>>>> thought there is also room for library routines that are closer to an
>>>>> application level, which a user can interact with more directly.
>>>>> Maybe I'm looking at this from the wrong angle though (it's quite
>>>>> late here and I haven't thought about it that long yet)? I won't
>>>>> suggest an alternative until I understand better what you had in mind.
>>>>>
>>>>> Yes to calling it AstroPy.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> James.
>>>>>
>>>>>
>>>>> On 27/06/11 16:16, Thomas Robitaille wrote:
>>>>>> Hi everyone,
>>>>>>
>>>>>> In the last week, Erik, Perry, and myself have been discussing how
>>>>>> best to coordinate the development of the common Python Astronomy
>>>>>> package. We have now converged on a common vision, and would now like
>>>>>> to know whether you would be happy with it too. The vision and a
>>>>>> poll are available at the following pages:
>>>>>>
>>>>>> vision: http://astropy.wikispaces.com/vision
>>>>>>
>>>>>> poll: http://astropy.wikispaces.com/vision-polls
>>>>>>
>>>>>> In addition, 'astropy' has been suggested by several people as a name
>>>>>> for this common package, so rather than creating a multi-option poll,
>>>>>> we've created a simple yes/no poll to find out whether you would agree
>>>>>> with this name. The idea is to have a name that does not endorse any
>>>>>> specific existing project, is in line with numpy/scipy, and reflects
>>>>>> its initial development via this mailing list. The poll is located at
>>>>>> the same URL as before.
>>>>>>
>>>>>> The polls will be open unti Friday 1st July at 9pm EST.
>>>>>>
>>>>>> Thanks!
>>>>>> Thomas
>>>>>> _______________________________________________
>>>>>> AstroPy mailing list
>>>>>> AstroPy at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>> _______________________________________________
>>>> AstroPy mailing list
>>>> AstroPy at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>>
>>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>


From stefan.czesla at hs.uni-hamburg.de  Tue Jun 28 05:32:04 2011
From: stefan.czesla at hs.uni-hamburg.de (Stefan Czesla)
Date: Tue, 28 Jun 2011 11:32:04 +0200
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <BANLkTimu_haNfuWvXLnU7pELPu6OKM=QwA@mail.gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>	<BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>	<A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>
	<BANLkTimu_haNfuWvXLnU7pELPu6OKM=QwA@mail.gmail.com>
Message-ID: <4E099F94.3060200@hs.uni-hamburg.de>

Dear all,

first I would like to thank Tom, Perry, and Eric for their work. I agree 
with Marshall, a binary vote
may be little restrictive to vote on such a complex issue. What 
irritated me a little more, however,
is that the vision was not put up for discussion before the community 
was asked to vote on it. At
least this has not been stated explicitly and the vision does not 
contain a reference to a place (maybe
this list) were the comments can be made. Even though nobody stops 
anybody from making
any comments, this does not mean that they will be considered at any 
time (there hardly would be time
until the vote closes).
I do not think that this is all stuff, which can be done later.
So it is the presentation (not necessarily the vision), which I find a 
little irritating. As we are in
a learning process, it is necessary to put this on the table right now; 
so please take this as
constructive criticism and not complaining.

Cheers,
Stefan


> Hi Marshall,
>
> Please do feel free to express any opinions you have in this thread.
> The thinking behind the binary vote is that we just want something we
> can start from. But once things start gaining momentum, there is no
> reason why we couldn't adopt a PEP-like process to propose amendments
> to the existing vision/plan :-)
>
> Cheers,
> Tom
>
> On 27 June 2011 22:47, Marshall Perrin<mperrin at stsci.edu>  wrote:
>> Structuring this as a binary yes/no vote seems a little restrictive to me. Is there any room for discussion or comments prior to this vote? I would have expected something a little more like Python's PEP workflow (http://www.python.org/dev/peps/pep-0001/#pep-work-flow) in which community feedback is applied to a document developed by one or more leaders.
>>
>>
>> (Nothing in this email should be taken to imply approval or lack thereof of anything in the draft text. I'm more just stating my own opinion that binary votes are not necessarily the best decision making process for complex issues, where 'yes, but' or 'yes and also' or 'no unless' can at times be very valuable in drawing on the community's expertise...)
>>
>>   - Marshall
>>
>>
>> ________________________________________
>> From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of Thomas Robitaille [thomas.robitaille at gmail.com]
>> Sent: Monday, June 27, 2011 4:22 PM
>> To: astropy at scipy.org
>> Subject: Re: [AstroPy] POLL: vision for a common Astronomy package
>>
>> Just a quick note to avoid any confusion - the two polls are
>> independent, so you will need to click 'vote' for each one in turn.
>>
>> Thanks,
>> Tom
>>
>> On 27 June 2011 22:16, Thomas Robitaille<thomas.robitaille at gmail.com>  wrote:
>>> Hi everyone,
>>>
>>> In the last week, Erik, Perry, and myself have been discussing how
>>> best to coordinate the development of the common Python Astronomy
>>> package. We have now converged on a common vision, and would now like
>>> to know whether you would be happy with it too. The vision and a
>>> poll are available at the following pages:
>>>
>>> vision: http://astropy.wikispaces.com/vision
>>>
>>> poll: http://astropy.wikispaces.com/vision-polls
>>>
>>> In addition, 'astropy' has been suggested by several people as a name
>>> for this common package, so rather than creating a multi-option poll,
>>> we've created a simple yes/no poll to find out whether you would agree
>>> with this name. The idea is to have a name that does not endorse any
>>> specific existing project, is in line with numpy/scipy, and reflects
>>> its initial development via this mailing list. The poll is located at
>>> the same URL as before.
>>>
>>> The polls will be open unti Friday 1st July at 9pm EST.
>>>
>>> Thanks!
>>> Thomas
>>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
A non-text attachment was scrubbed...
Name: stefan_czesla.vcf
Type: text/x-vcard
Size: 224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/astropy/attachments/20110628/cf4f3e00/attachment.vcf>

From jturner at gemini.edu  Tue Jun 28 06:14:56 2011
From: jturner at gemini.edu (James Turner)
Date: Tue, 28 Jun 2011 06:14:56 -0400
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <BANLkTikExbj5RiVknDEx9fEQGnt5d7-d9A@mail.gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
	<4E090963.9030408@gemini.edu> <4E091289.7050705@gemini.edu>
	<BANLkTi=+nvAgYQaRizNaH+kwsqtp=iKsrw@mail.gmail.com>
	<BANLkTikExbj5RiVknDEx9fEQGnt5d7-d9A@mail.gmail.com>
Message-ID: <4E09A9A0.6040408@gemini.edu>

> To clarify, we're not intending  to completely disallow other
> dependencies, but only stating that they should not be used
> *initially* when the package is imported.

Oh, more a coding practice than a restriction on what dependencies
are used? I suppose I read this as saying that if a dependency is
essential (so you would normally import it at the top) then it's
not allowed. OK, we'll I'll vote yes too and we can see how it
goes :-).

I agree with the case-by-case review of dependencies. After some
careful selection we should end up with a pretty standard base
platform that astronomers can rely on to be there plus a few
optional features that require extra dependencies.

James.


From luigi at lambrate.inaf.it  Tue Jun 28 06:16:50 2011
From: luigi at lambrate.inaf.it (Luigi Paioro)
Date: Tue, 28 Jun 2011 12:16:50 +0200
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <4E099F94.3060200@hs.uni-hamburg.de>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>	<BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>	<A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>	<BANLkTimu_haNfuWvXLnU7pELPu6OKM=QwA@mail.gmail.com>
	<4E099F94.3060200@hs.uni-hamburg.de>
Message-ID: <4E09AA12.5020704@lambrate.inaf.it>

Dear all,

   I have just voted the global vision giving my general agreement. Yes, 
a bit of discussion would be certainly appreciated, but I guess that 
this is just a starting point and not a final document. For this reason 
I tryed to add a comment to my vote, but the wiki system raised an error 
and thus my comment hasn't been committed. So, my general comment is this:

The vision document says:

"[...] Initially, no dependency on GUI toolkits will be allowed in the 
core package. If the community reaches agrees on a single toolkit that 
could be used, then this toolkit will be allowed (but will only be 
imported as needed)."

While I agree that initially no dependency on GUI toolkits should be 
allowed in the core library, I think that it would be desirable to reach 
an agreement on a single toolkit ASAP.

Maybe GUIs are not really important for the core library, but most of 
the programs we have written so far at INAF-IASF Milano (SAMPy included) 
rely on a GUI toolkit (alas, usually diverse). If I could unify them 
making them compliant with the AstroPy endorsed toolkit, I would be happy.

Think about this.

Cheers,

   Luigi



Il 06/28/11 11:32, Stefan Czesla ha scritto:
> Dear all,
>
> first I would like to thank Tom, Perry, and Eric for their work. I agree
> with Marshall, a binary vote
> may be little restrictive to vote on such a complex issue. What
> irritated me a little more, however,
> is that the vision was not put up for discussion before the community
> was asked to vote on it. At
> least this has not been stated explicitly and the vision does not
> contain a reference to a place (maybe
> this list) were the comments can be made. Even though nobody stops
> anybody from making
> any comments, this does not mean that they will be considered at any
> time (there hardly would be time
> until the vote closes).
> I do not think that this is all stuff, which can be done later.
> So it is the presentation (not necessarily the vision), which I find a
> little irritating. As we are in
> a learning process, it is necessary to put this on the table right now;
> so please take this as
> constructive criticism and not complaining.
>
> Cheers,
> Stefan


From heather625 at gmail.com  Tue Jun 28 14:56:33 2011
From: heather625 at gmail.com (Heather Kelly)
Date: Tue, 28 Jun 2011 14:56:33 -0400
Subject: [AstroPy] trouble installing pywcs-1.10-4.7 on Win XP with python
	2.7.2
Message-ID: <BANLkTin-Go6GKppQQbT5RDEdoVD7ZxjL5Q@mail.gmail.com>

Hi,

I have successfully installed pywcs in the past, most recently with python
2.6.5 on windows XP.  I am now trying to move to python 2.7.2, however pywcs
refuses to install:

C:\>C:\Python27\Scripts\pip.
exe install pywcs
Downloading/unpacking pywcs
  Downloading pywcs-1.10-4.7.tar.gz (2.0Mb): 2.0Mb downloaded
  Running setup.py egg_info for package pywcs

    warning: no files found matching 'doc\source\_templates'
    warning: no files found matching 'doc\source\_static'
    warning: no files found matching 'examples\*.fits'
Installing collected packages: pywcs
  Running setup.py install for pywcs
    building 'pywcs._pywcs' extension
    c:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo
/Ox /M
D /W3 /GS- /DNDEBUG -DECHO -DWCSTRIG_MACRO -DPYWCS_BUILD -D_GNU_SOURCE
-DNDEBUG
-DYY_NO_UNISTD_H -D_CRT_SECURE_NO_WARNINGS -UDEBUG
-IC:\Python27\lib\site-packages\numpy\core\include -I.\wcslib\C -Iwcslib\C
-I.\src -IC:\Python27\include -IC:
\Python27\PC /Tcwcslib\C\flexed/wcsbth.c
/Fobuild\temp.win32-2.7\Release\wcslib\
C\flexed/wcsbth.obj
    wcsbth.c
    .\wcslib\C\wcs.h(1396) : warning C4030: first formal parameter list
longer than the second list
    .\wcslib\C\wcs.h(1396) : warning C4028: formal parameter 1 different
from declaration
    .\wcslib\C\wcs.h(1396) : error C2040: 'wcsset' : 'int (wcsprm *)'
differs in
 levels of indirection from 'int (wcsprm *)'
    error: command '"c:\Program Files\Microsoft Visual Studio
9.0\VC\BIN\cl.exe"
' failed with exit status 2
    Complete output from command C:\Python27\python.exe -c "import
setuptools;__file__='C:\\build\\pywcs\\setup.py';exec(compile(open(__file__).read().replace('
\r\n', '\n'), __file__, 'exec'))" install
--single-version-externally-managed --record
c:\docume~1\heather\locals~1\temp\pip-pezwil-record\install-record.txt:
    running install

running build

running build_py

creating build

creating build\lib.win32-2.7

creating build\lib.win32-2.7\pywcs

copying lib\pywcs.py -> build\lib.win32-2.7\pywcs

copying lib\svn_version.py -> build\lib.win32-2.7\pywcs

copying lib\_docutil.py -> build\lib.win32-2.7\pywcs

copying lib\__init__.py -> build\lib.win32-2.7\pywcs

creating build\lib.win32-2.7\pywcs\tests

copying lib\tests\test.py -> build\lib.win32-2.7\pywcs\tests

copying lib\tests\__init__.py -> build\lib.win32-2.7\pywcs\tests

running build_ext

building 'pywcs._pywcs' extension

creating build\temp.win32-2.7

creating build\temp.win32-2.7\Release

creating build\temp.win32-2.7\Release\wcslib

creating build\temp.win32-2.7\Release\wcslib\C

creating build\temp.win32-2.7\Release\wcslib\C\flexed

creating build\temp.win32-2.7\Release\src

c:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox
/MD /W3 /GS- /DNDEBUG -DECHO -DWCSTRIG_MACRO -DPYWCS_BUILD -D_GNU_SOURCE
-DNDEBUG -DYY
_NO_UNISTD_H -D_CRT_SECURE_NO_WARNINGS -UDEBUG
-IC:\Python27\lib\site-packages\numpy\core\include -I.\wcslib\C -Iwcslib\C
-I.\src -IC:\Python27\include -IC:\Python27\PC /Tcwcslib\C\flexed/wcsbth.c
/Fobuild\temp.win32-2.7\Release\wcslib\C\flexed/wcsbth.obj

wcsbth.c

.\wcslib\C\wcs.h(1396) : warning C4030: first formal parameter list longer
than the second list

.\wcslib\C\wcs.h(1396) : warning C4028: formal parameter 1 different from
declaration

.\wcslib\C\wcs.h(1396) : error C2040: 'wcsset' : 'int (wcsprm *)' differs in
levels of indirection from 'int (wcsprm *)'

error: command '"c:\Program Files\Microsoft Visual Studio
9.0\VC\BIN\cl.exe"' failed with exit status 2

----------------------------------------
Command C:\Python27\python.exe -c "import
setuptools;__file__='C:\\build\\pywcs\\setup.py';exec(compile(open(__file__).read().replace('\r\n',
'\n'), __file__, '
exec'))" install --single-version-externally-managed --record
c:\docume~1\heather\locals~1\temp\pip-pezwil-record\install-record.txt
failed with error code 1
Storing complete log in C:\Documents and Settings\heather\Application
Data\pip\pip.log


I then went back and attempted to install using the tar.gz distribution:

C:\heather\downloads\python\pywcs-1.10-4.7>c:\Python27\python.exe setup.py
install
running install
running build
running build_py
copying lib\svn_version.py -> build\lib.win32-2.7\pywcs
running build_ext
building 'pywcs._pywcs' extension
c:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox
/MD /W
3 /GS- /DNDEBUG -DECHO -DWCSTRIG_MACRO -DPYWCS_BUILD -D_GNU_SOURCE -DNDEBUG
-DYY
_NO_UNISTD_H -D_CRT_SECURE_NO_WARNINGS -UDEBUG
-Ic:\Python27\lib\site-packages\numpy\core\include -I.\wcslib\C -Iwcslib\C
-I.\src -Ic:\Python27\include -Ic:\Python27\PC /Tcwcslib\C\flexed/wcsbth.c
/Fobuild\temp.win32-2.7\Release\wcslib\C\flexed/wcsbth.obj
wcsbth.c
.\wcslib\C\wcs.h(1396) : warning C4030: first formal parameter list longer
thanthe second list
.\wcslib\C\wcs.h(1396) : warning C4028: formal parameter 1 different from
declaration
.\wcslib\C\wcs.h(1396) : error C2040: 'wcsset' : 'int (wcsprm *)' differs in
levels of indirection from 'int (wcsprm *)'
error: command '"c:\Program Files\Microsoft Visual Studio
9.0\VC\BIN\cl.exe"' failed with exit status 2



Has anyone else seen this behavior and is there a workaround?

Thanks,
Heather
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110628/ba75f502/attachment.html>

From nbouche at physics.ucsb.edu  Wed Jun 29 15:46:43 2011
From: nbouche at physics.ucsb.edu (Nicolas Bouche)
Date: Wed, 29 Jun 2011 12:46:43 -0700
Subject: [AstroPy] AstroPy Digest, Vol 58, Issue 45
In-Reply-To: <mailman.1.1309366801.3242.astropy@scipy.org>
References: <mailman.1.1309366801.3242.astropy@scipy.org>
Message-ID: <4E0B8123.6030604@physics.ucsb.edu>


try to install it from here:

http://www.lfd.uci.edu/~gohlke/pythonlibs/

N

On 06/29/2011 10:00 AM, astropy-request at scipy.org wrote:
> Hi,
>
> I have successfully installed pywcs in the past, most recently with python
> 2.6.5 on windows XP.  I am now trying to move to python 2.7.2, however pywcs
> refuses to install:
>

-- 
--------------------------------------------------------------------
Dr Nicolas Bouche				nbouche at physics.ucsb.edu
Marie Curie Fellow
Department of Physics
University of Santa Barbara
Broida						tel: 805-893-2246
Santa Barbara 93106
--------------------------------------------------------------------


From MARQUETT at IAP.FR  Thu Jun 30 13:46:10 2011
From: MARQUETT at IAP.FR (Jean-Baptiste Marquette)
Date: Thu, 30 Jun 2011 19:46:10 +0200
Subject: [AstroPy] Error in vo on Mac OS 10.6.8, Python 2.7
Message-ID: <1E38F6C6-523D-4D0D-9A81-BAD289A99B0F@IAP.FR>

Dear vo gurus,

I use vo through ATpy on VOTables which are well read by TOPCAT. Thus I didn't find any violation of VO specifications.

The traceback is:

bs300.date processing ...
/Volumes/pepperland/erosdata/sexcatcalib/bs/bs300/bs3001/bs3001_6f2079.cat:3:0: W20: No version number specified in file. Assuming 1.0
Traceback (most recent call last):
  File "/Users/marquett/workspace/EpochsCat/src/PutEpochs.py", line 28, in <module>
    Table = atpy.Table(Cat, type='vo', pedantic=False, tid=2)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/atpy/basetable.py", line 167, in __init__
    self.read(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/atpy/basetable.py", line 213, in read
    atpy._readers[table_type](self, *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/atpy/votable.py", line 86, in read
    votable = parse(filename, pedantic=pedantic)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/table.py", line 103, in parse
    return tree.VOTableFile(config=config, pos=(1, 1)).parse(iterator, config)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 2963, in parse
    iterator, tag, data, config, pos)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 2897, in _add_resource
    resource.parse(self, iterator, config)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 2732, in parse
    tag_mapping.get(tag, self._add_unknown_tag)(iterator, tag, data, config, pos)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 2710, in _add_resource
    resource.parse(self._votable, iterator, config)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 2732, in parse
    tag_mapping.get(tag, self._add_unknown_tag)(iterator, tag, data, config, pos)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 2710, in _add_resource
    resource.parse(self._votable, iterator, config)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 2732, in parse
    tag_mapping.get(tag, self._add_unknown_tag)(iterator, tag, data, config, pos)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 2698, in _add_param
    param = Param(self._votable, config=config, pos=pos, **data)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 1232, in __init__
    id=id, config=config, pos=pos)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 915, in __init__
    self._setup(config)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 1251, in _setup
    self.value = self._value
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/tree.py", line 1239, in _set_value
    self._value = self.converter.parse(value, self._config, self._pos)[0]
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/converters.py", line 433, in parse
    config, pos)
  File "/Library/Frameworks/Python.framework/Versions/7.0/lib/python2.7/site-packages/vo/voexceptions.py", line 109, in vo_raise
    raise exc(message)
vo.voexceptions.VOTableSpecError: /Volumes/pepperland/erosdata/sexcatcalib/bs/bs300/bs3001/bs3001_6f2079.cat:87041:3: E02: Incorrect number of elements in array, expected 0, got 1

As I said this table is well loaded by TOPCAT, and was created by Emmanuel Bertin's SExtractor. It is 20 Mb large.
87041 (if it is a rank) is far beyond the index bound.
If necessary, I can provide the table in my ftp, together with my python script.
I checked that the issue remains the same on other tables of the same type.

Thanks for your help,

Jean-Baptiste Marquette

From erik.tollerud at gmail.com  Thu Jun 30 20:02:31 2011
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Thu, 30 Jun 2011 17:02:31 -0700
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <4E099F94.3060200@hs.uni-hamburg.de>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
	<BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>
	<A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>
	<BANLkTimu_haNfuWvXLnU7pELPu6OKM=QwA@mail.gmail.com>
	<4E099F94.3060200@hs.uni-hamburg.de>
Message-ID: <BANLkTikY1YRTuXk37KpeLx0=JSNsjC-STg@mail.gmail.com>

Just to quickly address the process here:  The idea was that the
vision document in particular needed to be drafted with a somewhat
coherent vision, so we wanted to offer it for a vote quickly so as to
get a sense as to whether everyone was on the same page here (so far
at least, in seems most people are, given the votes and clarifications
here).

For future documents such as coding style and other planning
documents, we will likely be issuing drafts and solicit
comments/suggestions before putting a modified version to a vote.  In
this particular case we felt that speed and consistency of vision
would help the process proceed more quickly and stay organized,
especially given that most of the details in the document are still
open for change if it's desired.


On the topic of GUIs, the thought was that it would be difficult to
come to a consensus given how many different toolkits are in use and
the fact that the learning curve is a fair bit steeper... so it would
be great to settle on one, but it might alienate a fair number of
otherwise interested developers, or reduce the knowledge base so much
that we couldn't get off the ground for GUIs.  That said, I personally
favor TraitsUI and the related Enthought tools, but I know there are
other opinions...


On Tue, Jun 28, 2011 at 2:32 AM, Stefan Czesla
<stefan.czesla at hs.uni-hamburg.de> wrote:
> Dear all,
>
> first I would like to thank Tom, Perry, and Eric for their work. I agree
> with Marshall, a binary vote
> may be little restrictive to vote on such a complex issue. What irritated me
> a little more, however,
> is that the vision was not put up for discussion before the community was
> asked to vote on it. At
> least this has not been stated explicitly and the vision does not contain a
> reference to a place (maybe
> this list) were the comments can be made. Even though nobody stops anybody
> from making
> any comments, this does not mean that they will be considered at any time
> (there hardly would be time
> until the vote closes).
> I do not think that this is all stuff, which can be done later.
> So it is the presentation (not necessarily the vision), which I find a
> little irritating. As we are in
> a learning process, it is necessary to put this on the table right now; so
> please take this as
> constructive criticism and not complaining.
>
> Cheers,
> Stefan
>
>
>> Hi Marshall,
>>
>> Please do feel free to express any opinions you have in this thread.
>> The thinking behind the binary vote is that we just want something we
>> can start from. But once things start gaining momentum, there is no
>> reason why we couldn't adopt a PEP-like process to propose amendments
>> to the existing vision/plan :-)
>>
>> Cheers,
>> Tom
>>
>> On 27 June 2011 22:47, Marshall Perrin<mperrin at stsci.edu> ?wrote:
>>>
>>> Structuring this as a binary yes/no vote seems a little restrictive to
>>> me. Is there any room for discussion or comments prior to this vote? I would
>>> have expected something a little more like Python's PEP workflow
>>> (http://www.python.org/dev/peps/pep-0001/#pep-work-flow) in which community
>>> feedback is applied to a document developed by one or more leaders.
>>>
>>>
>>> (Nothing in this email should be taken to imply approval or lack thereof
>>> of anything in the draft text. I'm more just stating my own opinion that
>>> binary votes are not necessarily the best decision making process for
>>> complex issues, where 'yes, but' or 'yes and also' or 'no unless' can at
>>> times be very valuable in drawing on the community's expertise...)
>>>
>>> ?- Marshall
>>>
>>>
>>> ________________________________________
>>> From: astropy-bounces at scipy.org [astropy-bounces at scipy.org] on behalf of
>>> Thomas Robitaille [thomas.robitaille at gmail.com]
>>> Sent: Monday, June 27, 2011 4:22 PM
>>> To: astropy at scipy.org
>>> Subject: Re: [AstroPy] POLL: vision for a common Astronomy package
>>>
>>> Just a quick note to avoid any confusion - the two polls are
>>> independent, so you will need to click 'vote' for each one in turn.
>>>
>>> Thanks,
>>> Tom
>>>
>>> On 27 June 2011 22:16, Thomas Robitaille<thomas.robitaille at gmail.com>
>>> ?wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> In the last week, Erik, Perry, and myself have been discussing how
>>>> best to coordinate the development of the common Python Astronomy
>>>> package. We have now converged on a common vision, and would now like
>>>> to know whether you would be happy with it too. The vision and a
>>>> poll are available at the following pages:
>>>>
>>>> vision: http://astropy.wikispaces.com/vision
>>>>
>>>> poll: http://astropy.wikispaces.com/vision-polls
>>>>
>>>> In addition, 'astropy' has been suggested by several people as a name
>>>> for this common package, so rather than creating a multi-option poll,
>>>> we've created a simple yes/no poll to find out whether you would agree
>>>> with this name. The idea is to have a name that does not endorse any
>>>> specific existing project, is in line with numpy/scipy, and reflects
>>>> its initial development via this mailing list. The poll is located at
>>>> the same URL as before.
>>>>
>>>> The polls will be open unti Friday 1st July at 9pm EST.
>>>>
>>>> Thanks!
>>>> Thomas
>>>>
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>> _______________________________________________
>>> AstroPy mailing list
>>> AstroPy at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/astropy
>>>
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at scipy.org
>> http://mail.scipy.org/mailman/listinfo/astropy
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>
>



-- 
Erik Tollerud


From pebarrett at gmail.com  Thu Jun 30 20:44:20 2011
From: pebarrett at gmail.com (Paul Barrett)
Date: Thu, 30 Jun 2011 20:44:20 -0400
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <BANLkTikY1YRTuXk37KpeLx0=JSNsjC-STg@mail.gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>
	<BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>
	<A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>
	<BANLkTimu_haNfuWvXLnU7pELPu6OKM=QwA@mail.gmail.com>
	<4E099F94.3060200@hs.uni-hamburg.de>
	<BANLkTikY1YRTuXk37KpeLx0=JSNsjC-STg@mail.gmail.com>
Message-ID: <BANLkTikuVSoMU5nH-F4ZLConrae6zEH+5g@mail.gmail.com>

Since, it appears that this vision for an astropy suite will be
adopted, given the overwhelming number of yeses at the current time,
do you have a timeline in mind for how to proceed, i.e., what modules
are being considered for the initial release and when will it occur?

 -- Paul


From bkloppenborg at gmail.com  Thu Jun 30 21:19:32 2011
From: bkloppenborg at gmail.com (Brian Kloppenborg)
Date: Thu, 30 Jun 2011 19:19:32 -0600
Subject: [AstroPy] POLL: vision for a common Astronomy package
In-Reply-To: <BANLkTikuVSoMU5nH-F4ZLConrae6zEH+5g@mail.gmail.com>
References: <BANLkTim1uCDYWd6sndE=_q68Wm2Q+AkxHA@mail.gmail.com>	<BANLkTi=+CT6kqphhrK4xh=De2GWBZwc2hQ@mail.gmail.com>	<A1EC8E0B03630149876AA55EEE18C05CE3B4C9@EXCHMAIL1.stsci.edu>	<BANLkTimu_haNfuWvXLnU7pELPu6OKM=QwA@mail.gmail.com>	<4E099F94.3060200@hs.uni-hamburg.de>	<BANLkTikY1YRTuXk37KpeLx0=JSNsjC-STg@mail.gmail.com>
	<BANLkTikuVSoMU5nH-F4ZLConrae6zEH+5g@mail.gmail.com>
Message-ID: <4E0D20A4.7090507@du.edu>

Greetings,

I have just recently joined the AstroPy listserv and am very happy to 
hear there is an effort to create a unified library for 
astronomy/astrophysics.  I openly welcome such a task and will be happy 
to contribute time coding the routines.

I am, however, a little concerned that the current vision statement 
seems to exclude the use of existing officially sanctioned (i.e. by the 
IAU) routines that are written in other languages.

For instance, the Standards of Fundamental Astronomy 
(http://www.iausofa.org/) package contains IAU-sanctioned routines for 
very common tasks (Calendards, Time Scales, Sidereal times, Ephemerids, 
star motion, star catalog conversion, etc).  This package is written in 
C and is quite easily wrapped using SWIG (see the pysofa package 
http://pypi.python.org/pypi/pysofa).  If the interface layer is 
implemented correctly, the function calls have minimal overhead and, in 
some circumstances, can benefit from being implemented in lower-level 
languages.

A quick example would be a general purpose library for common tasks in 
optical interferometry, in particular image reconstruction.  The code 
uses OpenCL which can leverage multiple CPUs and/or multiple GPUs to 
speed up the most time consuming tasks.  I've consistently demonstrated 
a 243x speed up over traditional CPU methods using the GPU and I really 
don't want to rewrite this in Python for inclusion in astropy.

Is there room for using libraries that are endorsed by a high-level 
governing body but that are written in other languages as a basis for 
some of astropy?  With the use of SWIG and official routines we could 
get (1) rapid development, (2) officially sanctioned routines that we 
don't have to debug/maintain, and (3) an interface to a well-developed 
package that we could enforce through the interface layer.  Clearly the 
detriment is that we would need to keep the packages in sync.  This is a 
common release problem in the programming world and has existing 
solutions though.

Thank you for your consideration,
Brian


On 06/30/2011 06:44 PM, Paul Barrett wrote:
> Since, it appears that this vision for an astropy suite will be
> adopted, given the overwhelming number of yeses at the current time,
> do you have a timeline in mind for how to proceed, i.e., what modules
> are being considered for the initial release and when will it occur?
>
>   -- Paul
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy


From aldcroft at head.cfa.harvard.edu  Thu Jun 30 23:01:56 2011
From: aldcroft at head.cfa.harvard.edu (Tom Aldcroft)
Date: Thu, 30 Jun 2011 23:01:56 -0400
Subject: [AstroPy] asciitable 0.7.0
Message-ID: <BANLkTinKg3n7XLjN3_=6SmemtbnmW6LmuQ@mail.gmail.com>

Version 0.7.0 of asciitable (an extensible module for reading and
writing ASCII tables) is now available.  This release includes the
following key features:

- Added support for reading and writing LaTeX tables
  (contributed by Moritz Guenther).
- Improved the CDS reader by better supporting multi-file
  tables (contributed by Frederic Grollier).
- Refactored the code into a package with functionally distinct modules.
- Added a "type" attribute in the Column class that provides the
  type of a column as IntType, FloatType, or StrType.

Please see: http://cxc.cfa.harvard.edu/contrib/asciitable/

The new Latex and AASTex classes provide the ability to write
publication quality LaTeX tables in both the standard and AAS
"deluxetable" formats.  These classes provide hooks to inject
additional LaTeX commands as needed for more complex tables.

API changes:
- Previously the read(), write(), get_reader() and get_writer()
  raised an exception for unrecognized keyword arguments.  Now
  those extra arguments are passed on to the Reader class
  constructor.  From the user perspective this means you can call
  read()/write() with class initialization arguments (see Cds and
  Latex for examples). For developers it means more flexibility
  in Reader classes.
- One minor API change is not backward compatible.  When
  specifying custom column converters (e.g. to force a column
  that looks like integers to convert to floats) it was
  previously possible to provide either a list of converters or a
  single converter.  Now you must always provide a list of
  converters even it is has only one element.  This was needed to
  support consistent assigment of the Column.type attribute.

Other minor fixes:
- Fixed a bug when the "end_line" parameter is passed a function.
- Fixed a bug where the RDB writer issued incorrect column types.

Cheers,
Tom Aldcroft