[AstroPy] Proliferating py-astro-libs

[Erik Tollerud]

A few comments on the excellent points Perry brings up:

> Early on there are some pragmatic needs for different approaches. For
> example, having a fairly "literal" translation of IDL tools into
> Python has its benefit. It is very useful for those that would like to
> migrate IDL code, and given the existing IDL versions, make it much
> easier to test their correctness. But I don't see this as a substitute
> for a good set of modular tools that have a better object design and
> consistent interfaces with other modules. Doing this is more work and
> will take more time. So a need for both approaches is likely. Some
> could argue the same for replacing IRAF tasks.
>
> All this is much easier said than done of course.

I 100% agree that the goal should be to make things work in a way
people are familiar with in IDL, while eventually switching to a clear
object-oriented/pythonic approach.  In my experience, programming
re-writing algorithms in object-based approaches is actually not
actually more time-consuming, once you get used to it - the key is to
then *also* provide a more functional-like interface to make the
transition easier/more practical.


> 1) We've been planning (along with Gemini, and particularly James
> Turner), to try to develop some Sage-like installation package that
> would make it easy to install all the basic tools for most
> astronomers. We had hoped to have a beta version out, but one of the
> people working on this left at the end of last year, and we've not
> been able to replace that person. We are going to continue this effort
> with existing staff though. Hopefully in a few months we'll have
> something to try out.

You may already be too far in to change this, but I think it's
important to make install tools work within the existing python
ecosystem.  For example, I already have a tool written in astropysics
that installs needed astronomy-related packages, but it uses the PyPI
RPC interface to easily identify packages, and then downloads and
installs them just like, for example, pip
(http://pypi.python.org/pypi/pip) does, which has been clearly
endorsed as the future of python package installers.



> 2) There is a recognition that a more serious effort needs to be made
> to replace IRAF functionality. Perhaps one of the benefits of a JWST
> delay is that it will allow us to do some of that work more
> explicitly. But we would not do it by replacing IRAF tasks one-for-one
> but coming up with an entirely different approach which has to start
> from the bottom up (the end result could have applications that mostly
> emulate IRAF tasks, but also provide much more modular tools).

+1 !

> 3) More specifically, we are currently focussing on how to handle WCS
> issues in a more general way than they are handled in FITS. If there
> is interest, perhaps we should say more about the intended approach.
> This is particularly important for replacing spectrographic tools in
> IRAF, and this is where we are starting our effort.
>
> 4) We need a way of saving these WCS models, and FITS is not the way.
> We are looking at an alternate data format, not just for WCS models,
> but for data in general [gasp!]. Work has begun on this too.

I'm very curious what you have in mind here - are you thinking of a
new format that's more adaptable, or more tightly coupling the data
format to the intended applications?  And is there a plan yet for what
the format is supposed to look like (i.e. XML or otherwise
standards-based or something different?)


> 5) A lot of our recent work has been on pysynphot and ETCs. We plan on
> making the computational part of our ETCs a released tool. But I'm
> also wondering if we can generalize the pysynphot spectral models for
> more general use in spectral tools.

I think spectra might be a good "focus area" where there's room for
consolidation as you mention below...


> 6) We have been working on a framework for making pipelines easier to
> build and configure. That won't be ready for at least a few months,
> but could well be of general interest and use.

Have you looked at my astropysics.utils.pipeline module?  That's
already working, although that may not be exactly what you had in
mind.  I'm working on a traits-based GUI interface for it as well,
although it's not working yet.  Regardless, one important thing to
think about here is making sure it's compatible with the parallel
computing era that we're going into now (it could be nade much easier
with the new parallel architecture of the 0.11 version of ipython).


> But the community ought to identify one or two areas that are of the
> most interest in consolidating (let's start small). What should we
> start with? Focus is important in making any progress in this area.

See my comments above with regard to spectra.  I also think perhaps
some agreement on some basic interfaces for model-fitting to make them
cross-compatible is also important, as a lot of people seem to be
duplicating functionality there, and it's a pretty common need (albiet
with a lot of complicated and application-specific implementation
details).  I've started at this with pymodelfit, but that's certainly
not the only way it could be done...

Also, while I agree focus is important, I think there is also strong
need for a "vision" - that is, a plan for how to proceed on a larger
scale. Then the starting pieces in the direction of that vision can be
determined.  I think my person "vision" for how to deal with this
consolidation was apparent from my previous message, so I won't repeat
it here (although I emphasis that the vision does *not* have to be
based on astropysics - some other package that fulfills the same aims
and uses github or other robust development models would work just as
well).


-- 
Erik Tollerud