[AstroPy] Deployment and packaging
Perry Greenfield
perry at stsci.edu
Thu Jun 16 13:59:35 EDT 2011
Some points I would like to make
1) It's clear that deployment and packaging should not be directly
tied to the effort to develop more general and consistent astronomical
modules for Python. This really is a different issue than the other
effort and nothing about this should impede that.
2) I'd say that I think all (or almost all, anyway) agree that these
modules should support standard Python installation schemes (e.g.,
PyPI support), and that one should not have to use some monolithic
binary install or anything like that if one doesn't want to install
them that way. In some respects, this is an extension of point 1.
3) It's good to understand that the typical expert on this list
doesn't represent the average user that might want to use this
software. Typically those participating in this discussion have
significantly more expertise in doing software installations than the
typical user does.
4) It's also good to understand that many users out there don't have
root access, and don't necessary have support staff to do what they
would like as far as root installations as Tom mentions. It's true at
CFA and STScI, for example. This means that tools that may work
wonderfully for you, such as RPMs, are useless to these people. Some
have railed at the stupidity of not allowing them root, it's a fact of
life at a number of institutions, and it isn't likely to change.
5) Besides the root issue, what works well for you, doesn't
necessarily work well for everyone else. There is a great variety of
systems out there that confound standard installations. Unless you
have experience distributing software that has lots of dependences,
you may not be aware of all the problems you may run into. As some
have mentioned, most users will give up at the first installation
problem. The greater the number of separate installations one has to
do, the greater the variety of the installation scheme (from make
variants, to distutils, to...) the more likely problems are to occur
and the greater the likelihood of the user giving up.
6) With the standard, individualized installations one can run into
dependency conflicts where the different packages have inconsistent
dependency requirements. Hopefully coordinated development can avoid
this (though there may be occasions where it is unavoidable), but
really what is needed is regular regression testing of the packages
with each other. This is something we (STScI) are planning on as part
of our distribution, but it is an activity that needs to be done in
any event.
7) Speaking of dependencies, distribution systems out there (e.g.,
RPMs) help, but they also are subject to dependency conflicts. This is
an area that really has no easy solution. The only existing, reliable
solution is to control what versions are packaged together explicitly,
and test them. This leads to two conflicting desires. The packaging
and testing together takes time, and the more one includes, the more
the effort (one reason why things like Linux distributions run behind
in versions by substantial amounts). It's reliability and consistency
vs. how current it is.
8) The scisoft distribution has some issues that people have pointed
out already. But a lots of people have used it just because it is so
simple to install. I have been asked often to support something like
that for our software. But it isn't easy to update items and that a
significant problem.
9) Is a larger package a real problem for people? Suppose it includes
X, Y, and Z for which you have no interest in having. So long as they
don't interfere with the installation, or break software you do care
about, why do you care if there are there? (so long as the space
required isn't massive or the download time isn't horrible).
10) One objection is a big binary install isn't flexible. You get what
you get, and you can't update it. We do plan to make it possible to
update modules within it, and add modules to it. It is true that doing
so uses a different scheme than the standard approach and it is more
work. But...
My final point is: So what alternate packaged binary (or similar)
solution that exists now that we should be using that doesn't require
root, and doesn't require N different versions for M platforms? My
impression from all the discussion that there isn't one. It isn't like
we are going our own way when there is already a good solution out
there. People don't like (for good reasons): scisoft, EPD, Sage,
etc... So why object to trying something else? If it doesn't work, it
won't hurt the standard way of installing software, will it? If it
does work, it gives those that don't want to manage 18 separate
installations something that should work in the majority of cases with
little pain.
Perry
More information about the AstroPy
mailing list