CPAN functionality for python - requirements

[Doug Hellmann]

On Tue, 27 Feb 2001, Bruce Sass wrote:
> On Mon, 26 Feb 2001, Doug Hellmann wrote:
> 
> > On Mon, 26 Feb 2001, Sean Reifschneider wrote:
> > > On Mon, Feb 26, 2001 at 06:45:33AM -0500, Doug Hellmann wrote:
> <...>
> > It still isn't clear to me why we would want to use more than one distribution
> > package format.  It appears that one argument is so that sysadmins can use
> > those tools to track what is installed on the system.  But isn't that part of
> > what this new thing (cyphon?) is supposed to do?  Do we want Python to require
> > an external package management system on each platform where this tool is
> > supported, or do we want a Python-esq tool which just works the way it is
> > supposed to, in the same way, everywhere Python is available?
> 
> I want users to be able to find and install Python stuff, preferably
> in a manner consistent with their native package management system.
> 
> > But I think we're arguing over a design before we've worked out requirements.
> > The basic tasks of this new tool, as I see them, are:
> >
> > 1. Locate Python packages you want on the net.
> > 2. Resolve dependencies between what you want, what you have, and what you
> >    need.
> 
> There is no way an archive network can resolve dependencies because

No, the network doesn't.  But the network by itself isn't going to do most of
these things.  The client app would need to resolve the dependencies.

If I tell the client to find and download PmwContribD, it needs to figure out
that I also need Pmw (and perhaps a minimum version) and either suggest that I
go ahead and install it or fail with a message telling me to install that first
depending on the command line arguments given.

> ther is no way for the archive network to know what is installed on a
> users machine (at least there had better not be!), or what packages
> are available to fulfil those dependencies.  The best that can
> be done is to tell the target system what is required by the pkg being
> looked at.  The dependencies get resolved when the native system
> decides on where it should fetch the other stuff from.

It sounds like the knows how to find a particular package, but getting it
installed is your problem.  We can make it smarter than that.

> Dependencies are a big problem.  I've spoken with a local LUG member
> who was complaining that Debian does not know about the stuff he gets from
> CPAN, which makes mixing CPAN and debianized CPAN stuff a pain.
> Debian's solution is a user level program to debianize arbitrary CPAN
> modules... I want the same solution for Python modules, except I want
> it done transparently.

I can see that as somewhat desirable, and it could be implemented, but do
you make the cyphon client download the right package format or convert
a standard distro into the right format?  Doesn't the user have to specify a
package type?

> > 3. Download the appropriate packages to give you what you want.
> > 4. Install those packages.
> 
> Doesn't that mean the program needs to know about all the distros, and
> all the version of those distros, and all the supported ways a user
> can tweak the system.  If you don't do that, and keep it up, then your
> installer is going to be breaking often (or always broken, depending
> on the user or distro).
> 
> > Do we agree these are the primary features?
> 
> Both 2. and 4. are the domain of the target system.  2. can be
> modified (or maybe just reworded).  4...

What I want is a single app I can use that will do all of those things.  I want
to type one command and have it find and install software on my system that
will allow me to use some Python package that I don't have yet.  That app would
have a client side and server side, but as a user I don't care about the server
side unless I want to specify a specific mirror to use by default.

> > I sense a consensus that the "install" part should be handled by distutils.  Is
> > that right?
> 
> As long as distutils does not presume to know how to do the
> installation.

How can it do the install if it doesn't know how?

It sounds like you do *not* want distutils to do the install, and that you
cyphon to download/create a platform-specific package to be installed.  Since we
*have* distutils as an ostensibly standard installation system for Python
modules, shouldn't we use it?

> > Other requirements we might lay down up front:
> >
> > 1. Should run on all platforms where Python runs.
> > 2. Must support mirrors on the server side.
> > 3. Need to include documentation along with source for packages.
> >
> > The features related to dependencies and downloads could be handled by a
> > platform-specific packaging system, but integrating with all of the different
> > options on all of the different platforms where Python runs increases the
> > complexity of the new tool.
> 
> Downloading is independent of the native package management system.

No, it isn't.  Even if *all* the tool does is download things I told it to find
and write them to the current directory, it has to know what format I like or
it may download (for example) debian packages for my Mac -- and then I won't
get useful results from the tool.

> Complexity is only increased if the new tool thinks it should handle
> platform/OS/distro specific issues.

It *must* handle those issues to be useful.

> > (How do we handle RPMs on MacOS?  How do we handle
> > HQX files on Win32?  What about for Pippy, where files as such might not even
> > be appropriate?  Does Jython have any special requirements?)  Do we deal with
> > this complexity by deciding on a per-platform basis what format to use, or do
> > we force the user to specify the format they want downloaded?
> 
> You don't.  You give the data to the target system and let it worry
> about the complexity of packaging.

What is the "target system," the OS?
 
> There is no reason for the archive network to know anything except
> where to send the data!

Right, but we do need a client so that users have a way to interact with the
network.

Wait -- maybe that's where the disconnect is!  I'm describing features of the
system from the client-side view, assuming the network would provide services
to support those features.  It sounds like you are describing features of the
network.  We should determine the requirements from the user perspective first,
and design something to meet those requirements.  Designs for the network
services will follow from the requirements.

> > If we're modeling what we're doing based on CPAN, maybe we should look at
> > that design.  What format does CPAN use when downloading Perl packages?  What
> > features does CPAN have that we want?  What does it not have that we want?
> 
> Pros:
> don't know (long strings of punctuation and symbols give me a headache ;).

Heh, tell me about it.

I can't remember now whether CPAN did installations or just downloaded stuff
and made you figure out how to install it.

> Cons:
> Dependencies problems when mixed with modules fetched by other methods.

Right, so we need a single way of determining, maintaining, and
resolving dependencies.

Doug