[Distutils] setuptools: package management and explicit version numbers

[Paul Moore]

On 8/10/05, Ian Bicking <ianb at colorstudy.com> wrote:
> Paul Moore wrote:
> > Hence my interest in management tools - installers come with a number
> > of benefits "free", as part of the Windows installer architecture -
> > you get an uninstall "command", you can see a listing of installed
> > packages, and you can see the version numbers of those packages. Until
> > I can do the same with eggs, they are a step backwards for me. So
> > that's why I want to build these features.
> 
> I suspect the management tool has two parts -- one is extending
> pkg_resources so these operations are easy (list, delete).  Then you
> want to integrate that into Windows' installer architecture --
> registering the packages and uninstallers with Windows, maybe making a
> GUI frontend (though you could almost get away without it).  But since
> I'm not using Windows, the frontend I imagine is obviously very
> different.  My frontend is probably really easy, but I have no idea what
> a GUI frontend would entail.

I'm not interested in integrating with the Windows installer stuff per
se. My key concern here is that C:\Python24 (or wherever Python is
installed) is owned by Python - I want no reason or need to use OS
commands directly on its contents. As long as I have a way of getting
the information I'm after, and performing the actions I need to, I'm
happy. On that basis, something like

    easy_install.py list

(with command line output) is perfectly acceptable. Indeed, it's
probably an improvement, to the extent that it can be used in code,
not just read by the user.

BUT (and it's a big but) when easy_install.py list doesn't show me
everything installed in Python's directory, I have a problem. I
exclude stuff installed via python setup.py install (I never do that,
because it's unmanaged) or "just dropped in there" (same reason). But
there's a transitional problem with bdist_wininst installers, which
needs some thinking about (and no, easy_install's "convert a wininst
installer to an egg" feature isn't an answer - it loses things like
cx_Oracle's documentation, and I suspect that it breaks horribly for
pywin32, which uses a postinstall script).

As this issue is transitional only, it's not the end of the world, as
long as eggs really do become the package distribution medium of
choice for Python. I'd like to see some sign that eggs are making
inroads against bdist_wininst, first, though.

> I think eggs are sometimes a distraction when considering features.  Or
> at least they are for me.  I know there's some eggs produced when I
> install stuff with easy_install, but I don't really touch them or care
> about them much.  Right now I don't have any incentive to distribute
> anything but source packages (from sdist); easy_install and setuptools
> do a fine job of using these, and for the users I interact with (other
> programmers) the source package is more usable.

Source distributions, I currently immediately build into Windows
installers (python setup.py bdist_wininst) and install. If they
contain C extensions that I can't build myself (for whatever reason) I
bleat pathetically at the author in the hope that he'll take pity on
me and provide Windows builds, and if that doesn't work, I generally
(have to) give up.

So unless you are building packages with C extensions having complex
build requirements, you'll get no incentives from me - the source
package is fine. But I'll use it to build Windows installers, and
never bother with eggs.

(Is there ever going to be a situation where code *won't work" unless
run from an egg? I can see that being an issue, unless there's a way
availabe to wrap eggs in Windows installers, RPMs, debs, etc - there
will always be people who value "integration with platform standards"
over other benefits).

> But anyway, the feature for me is the installation of different package
> versions.  This makes me less worried about installing things globally.
>  This in turn makes management a lot easier.
> 
> I suspect Windows usage patterns (or just your patterns) aren't the
> same, so you aren't as concerned about this.

Correct. I don't worry about package versions much, beyond getting the
latest one. I tend to treat applications which don't work with "the
latest" versions of dependent packages as broken, and either attempt
to fix them, or avoid using them, depending on time and the importance
of the package to me.

> Another issue is installation of dependencies.  This is still just
> potential for me; I haven't really had the satisfying experience of
> getting a package to install lots of dependencies for me.  But I think
> I'm getting closer.  This is something I'm used to with software
> installation, but not Python.  Windows doesn't do this anyway, so you
> probably aren't clamouring for it.  I think it changes the larger
> ecosystem and encourages sharing; but the payoff isn't as immediate
> because there isn't as much sharing as there could be.

As I've hinted in previous messages, I have no interest whatsoever in
automatic location and download of packages - both for environmental
(firewall) reasons, and personal ones (security concerns, plus a
dislike of giving up control...) So, for me, getting a package to list
its dependencies is all I need - I'm still going to impose on myself
the requirement to manually download and install those dependencies.

This self-imposed manual process doesn't handle packages with
conflicting dependencies well, but that's never been a practical issue
for me so far (keeping my fingers crossed...)

> And last is formal metadata on a package, which is what makes plugins
> workable.  You'll only see this as people actually start using that
> metadata, so there's not much appeal there yet.

And that's the real killer benefit, to me, but it affects plugins much
more than standalone packages, so it isn't clear to me how relevant it
is to these discussions.

> The development installs are there for developers.  Software doesn't
> appear out of thin air!

Agreed, and I understand the benefits. But I don't have the experience
to understand the trade-offs and/or risks, so I have to be very
cautious in what I promise. (For example, if I code an "uninstall"
command, and someone runs it on a development install, does that
delete all of that person's working code? How can I make sure that
doesn't happen without breaking the "normal" uninstall functionality?
Given that I can't really test this, am I better just saying "not
supported if run against development installs"?)

> I'm doing a presentation on this stuff tomorrow, though, so hopefully
> I'll get closer today.  It'll feature lots of command-line transcripts
> you probably will find rather unexciting ;)

That comment would amuse my colleagues in the office - I'm the
"command line geek" round here :-)

Paul.