[Distutils] Reviving PEP 470 - Removing External Hosting support on PyPI

[Wichert Akkerman]

> On 31 Aug 2015, at 12:36, Paul Moore <p.f.moore at gmail.com> wrote:
> 
> On 31 August 2015 at 10:43, Wichert Akkerman <wichert at wiggy.net> wrote:
>> I just wanted to add a bit of context, since your statement seemed to reflect a slightly different reality than mine. You do bring up a good point though. I really like the apt-preferences approach. That allows you to define some rules to set which repository should be used. You can do things like always prefer a specific repository, or do that only for specific packages, with a default rule to use whichever repository has the latest version. Very, very useful.
> 
> There's been a few posts now about how the new system is or is not
> like Linux package management systems. As a Windows user, my view of
> Linux package management is pretty limited. To me it seems like the
> basic approach is, if a package is in the official repo, you can just
> do apt-get install (or yum install) and it works. If the package is
> elsewhere, you need to find out where (usually manually, as far as I
> can see) and then do a bit of config, and then the package is
> available just like standard ones. That's pretty much the same as the
> proposed solution for PyPI/pip.
> 
> If there's any additional functionality that Linux systems provide,
> could someone summarise it from an end user POV for me? (And maybe
> also point out why I'd never noticed it as a naive Linux user!) To me,
> that's a key to whether this PEP is missing something important
> relative to those systems.

Sure. My knowledge of rpm is 20 years out of date, so I am going to focus the deb/dpkg/apt world only. The whole packaging system is build around archives. The packaging tools themselves do not have anything hardcoded there, they pick up the archive from a configuration file (/etc/apt/sources.list). That file lists all archives. For example:

# Hetzner APT-Mirror
deb http://mirror.hetzner.de/ubuntu/packages trusty main restricted universe multiverse
deb-src http://mirror.hetzner.de/ubuntu/packages trusty main restricted universe multiverse

deb http://de.archive.ubuntu.com/ubuntu/ trusty multiverse

deb http://security.ubuntu.com/ubuntu trusty-security main restricted
deb-src http://security.ubuntu.com/ubuntu trusty-security main restricted

There are two things of note here: you can have multiple archive types (“deb” and “deb-src” in this case), and different URL types. Besides the standard http there is also a cdrom scheme which can mount cdroms, there is a tor scheme now, etc. These are pluggable and handled by a binaries in /usr/lib/apt/methods/ . When you do an install of a new computer the installer will add some default repositories there, generally a local mirror and the security archive. https://wiki.debian.org/SourcesList <https://wiki.debian.org/SourcesList> has some more detailed information (which is probably not relevant here).

There are some convenient tools available to register extra archives. For example add-apt-repository, which allows you to do this:

# add-apt-repository ppa:saltstack/salt
# add-apt-repository “deb http://nl.archive.ubuntu.com/ubuntu <http://nl.archive.ubuntu.com/ubuntu> trusty universe"

This will try to download and configure the correct GPG key to check archive signatures as well.

Each archive has an index which lists all its packages with metadata. You download these to your system (using “apt-get update” or a GUI), and the local copy is used by all other tools. That results in fast searches, and solvers having fast access to the complete database of available packages.

When installing a package your normally get the latest version, independent of which archive has that version. You can ask the system which versions are available:

# apt-cache policy salt-minion
salt-minion:
  Installed: 2015.5.3+ds-1trusty1
  Candidate: 2015.5.3+ds-1trusty1
  Version table:
 *** 2015.5.3+ds-1trusty1 0
        500 http://ppa.launchpad.net/saltstack/salt/ubuntu/ trusty/main amd64 Packages
        100 /var/lib/dpkg/status
     0.17.5+ds-1 0
        500 http://mirror.hetzner.de/ubuntu/packages/ trusty/universe amd64 Packages
        500 http://de.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

In this case there are two versions available: 2015.5.3+ds-1trusty1 is currently installed and came from a ppa, and 0.17.5+ds-1 which is available from two mirrors.

This can result in somewhat interesting behaviour when multiple archives have the same packages and are updated independently. To handle that you can define preferences to tell the system how you want to handle that. For example if you want salt to always be installed from the ppa you can define this preference:

Package: salt*
Pin: origin “LP-PPA-saltstack-salt”
Pin-Priority: 900

This makes sure packages from the salt ppa have a higher priority, so they are always preferred. You can also use this to make an  archive of backports available on your system, but only track a few specific packages from there. There are more options available there; see https://wiki.debian.org/AptPreferences <https://wiki.debian.org/AptPreferences> and https://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_tweaking_candidate_version <https://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_tweaking_candidate_version> for more information. 

Wichert.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150831/b62a36f5/attachment.html>