[AstroPy] Proliferating py-astro-libs

[Perry Greenfield]

On Jun 10, 2011, at 5:32 PM, Erik Tollerud wrote:
> [...]
>> 1) We've been planning (along with Gemini, and particularly James
>> Turner), to try to develop some Sage-like installation package that
>> would make it easy to install all the basic tools for most
>> astronomers. We had hoped to have a beta version out, but one of the
>> people working on this left at the end of last year, and we've not
>> been able to replace that person. We are going to continue this  
>> effort
>> with existing staff though. Hopefully in a few months we'll have
>> something to try out.
>
> You may already be too far in to change this, but I think it's
> important to make install tools work within the existing python
> ecosystem.  For example, I already have a tool written in astropysics
> that installs needed astronomy-related packages, but it uses the PyPI
> RPC interface to easily identify packages, and then downloads and
> installs them just like, for example, pip
> (http://pypi.python.org/pypi/pip) does, which has been clearly
> endorsed as the future of python package installers.
>
We are planning to do that as well, but it doesn't solve the larger  
issue. Not everything people need to use is in that ecosystem, and  
some of the things in it don't install easily on lots of platforms.  
Unlike the Sage approach, this has to allow updates and additions so I  
wouldn't want people to read too much into the "Sage-like"
>
>
>> 3) More specifically, we are currently focussing on how to handle WCS
>> issues in a more general way than they are handled in FITS. If there
>> is interest, perhaps we should say more about the intended approach.
>> This is particularly important for replacing spectrographic tools in
>> IRAF, and this is where we are starting our effort.
>>
>> 4) We need a way of saving these WCS models, and FITS is not the way.
>> We are looking at an alternate data format, not just for WCS models,
>> but for data in general [gasp!]. Work has begun on this too.
>
> I'm very curious what you have in mind here - are you thinking of a
> new format that's more adaptable, or more tightly coupling the data
> format to the intended applications?  And is there a plan yet for what
> the format is supposed to look like (i.e. XML or otherwise
> standards-based or something different?)
>
Yes to more adaptable, no to tightly coupling (though I suppose that  
is subject to interpretation). We have some ideas. Ideally it could  
map to more than one representation on disk but represent a common  
abstract object system. Perhaps I will circulate a bit more on some of  
the ideas in the near future.
>
>> 5) A lot of our recent work has been on pysynphot and ETCs. We plan  
>> on
>> making the computational part of our ETCs a released tool. But I'm
>> also wondering if we can generalize the pysynphot spectral models for
>> more general use in spectral tools.
>
> I think spectra might be a good "focus area" where there's room for
> consolidation as you mention below...
>
That's a good possibility.
>
>> 6) We have been working on a framework for making pipelines easier to
>> build and configure. That won't be ready for at least a few months,
>> but could well be of general interest and use.
>
> Have you looked at my astropysics.utils.pipeline module?  That's
> already working, although that may not be exactly what you had in
> mind.  I'm working on a traits-based GUI interface for it as well,
> although it's not working yet.  Regardless, one important thing to
> think about here is making sure it's compatible with the parallel
> computing era that we're going into now (it could be nade much easier
> with the new parallel architecture of the 0.11 version of ipython).
>
yes, but this has a somewhat different goal. Reviewing that again, it  
doesn't seem like there is that much overlap in what it does. I think  
pipeline is a fairly overloaded term these days.
>
>> But the community ought to identify one or two areas that are of the
>> most interest in consolidating (let's start small). What should we
>> start with? Focus is important in making any progress in this area.
>
> See my comments above with regard to spectra.  I also think perhaps
> some agreement on some basic interfaces for model-fitting to make them
> cross-compatible is also important, as a lot of people seem to be
> duplicating functionality there, and it's a pretty common need (albiet
> with a lot of complicated and application-specific implementation
> details).  I've started at this with pymodelfit, but that's certainly
> not the only way it could be done...
>
> Also, while I agree focus is important, I think there is also strong
> need for a "vision" - that is, a plan for how to proceed on a larger
> scale. Then the starting pieces in the direction of that vision can be
> determined.  I think my person "vision" for how to deal with this
> consolidation was apparent from my previous message, so I won't repeat
> it here (although I emphasis that the vision does *not* have to be
> based on astropysics - some other package that fulfills the same aims
> and uses github or other robust development models would work just as
> well).

Perhaps. I think vision makes sense on small to medium scales.  
Astronomy software is a pretty large area and I'm not sure anyone in  
particular can do a vision for everything in that regard. I think  
there is room for both broad (top down?) and narrow approaches (bottom  
up coalescing). I've seen lots of failures with the pure top down  
approach. But maybe I'm too small minded :-)

Perry