[Distutils] Handling the binary dependency management problem

[Nick Coghlan]

On 4 Dec 2013 01:31, "Oscar Benjamin" <oscar.j.benjamin at gmail.com> wrote:
>
> On 3 December 2013 13:53, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > On 3 December 2013 22:49, Oscar Benjamin <oscar.j.benjamin at gmail.com>
wrote:
> >
> > Hmm, I likely wouldn't build it into the core requirement system (that
> > all operates at the distribution level), but the latest metadata
> > updates split out a bunch of the optional stuff to extensions (see
> >
https://bitbucket.org/pypa/pypi-metadata-formats/src/default/standard-metadata-extensions.rst
).
> > What we're really after at this point is the ability to *detect*
> > conflicts if somebody tries to install incompatible builds into the
> > same virtual environment (e.g. you installed from custom index server
> > originally, but later you forget and install from PyPI).
> >
> > So perhaps we could have a "python.expects" extension, where we can
> > assert certain things about the metadata of other distributions in the
> > environment. So, say that numpy were to define a custom extension
> > where they can define the exported binary interfaces:
> >
> >     "extensions": {
> >         "numpy.compatibility": {
> >             "api_version": 1,
> >             "fortran_abi": "openblas-g77"
> >         }
> >     }
> [snip]
> >
> > I like the general idea of being able to detect conflicts through the
> > published metadata, but would like to use the extension mechanism to
> > avoid name conflicts.
>
> Helping to prevent borken installs in this way would definitely be an
> improvement. It would be a real shame though if PyPI would contain all
> the metadata needed to match up compatible binary wheels but pip would
> only use it to show error messages rather than to actually locate the
> wheel that the user wants.

One of the side benefits of the TUF end-to-end security proposal is that it
allows for more reliable local metadata caching. That means that even if a
mechanism like this was initially only used for conflict detection, it
could eventually also be used to choose between multiple possible
resolutions of a dependency.

Tangentially related, I'm considering adding an additional recommended URL
field to the "python.integrator" extension, which would be a pointer to a
custom index server operated by the integrator.

Cheers,
Nick.

>
>
> Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131204/b9cdd3c1/attachment.html>