[Distutils] New PEP : dependency specification

[Robert Collins]

On 7 November 2015 at 07:22, Daniel Holth <dholth at gmail.com> wrote:
> LGTM
>
> To clarify in this spec to specify a couple of requirements for the [foo]
> extra would you have to say
>
> [foo] requests
> [foo] sqlalchemy

Oh, so this spec describes how to specify a dependency, not how to
provide the metadata to e.g. setuptools. I may be misinterpreting your
question, but assuming you are referring to the setup.py interface...
it should be insertable in a compatible way with the existing
setuptools idioms.

e.g. in setuptools today you'd say

extras_require={'foo': ["requests", "sqlalchemy"]}

And I think you'd keep doing that.

What you might want to do though is say that for the foo extra that
one of the dependencies only applies sometimes. Today you do this
manual separation thing:

extras_require={
 'foo': ["requests", "sqlalchemy"],
 'foo:python_version<"2.7"': ["requests[security]"]}

A patch to setuptools to allow dependencies per this PEP would allow:

extras_require={
 'foo': ["requests", "sqlalchemy", "requests[security];python_version<'2.7'"]}

> Compare to requires.txt from setuptools which IIRC is a plain text file like
> so, with normal requirements not in a section, and extra or conditional
> requirements in sections named [extra_name;marker]:
>...

OTOH you might be talking about how we'd using this in an PKG_INFO /
egg-info / dist-info metadata serialisation format.

We haven't specified that yet - since there is strictly no new
capabilities over those existing on-disk formats, they are just
different, I think we have an ok status quo. It's not as great as
having a fully specified consistent thing end to end format though :(.

So taking a higher level view, we currently have three formats to describe this:
 - requires.txt, which can't specify markers at the granularity of a
dependency, only by having larger structure which also defines extras
and uses the empty extra to define marker based rules
 - PEP-345 PKG-INFO, which nothing outputs as PKG-INFO, but wheel
outputs as dist-info/METADATA - and is very nearly compatible with
this PEP (it just adds brackets around version specificiers). I think
adding those in as optional but not recommended elements, would permit
a single unified parser for METADATA specifiers and the ones in this
PEP.
 - requirements.txt files which I decided to use as the basis because
its been hammered on most IMO :).

I don't have a PEP yet to specify the *export* of distribution
metadata using this format. If we adopt 'what is in egg-info' as the
interop basis for now, then such a PEP can come later, but at the cost
of having a non-PEP defined format included by reference. If we adopt
what is in dist-info as the format, we get a define thing, but out of
date (because markers have evolved since PEP-345) - but we should be
able to do a very minimal update to 345 that solely references this
new PEP to update the dependency specification (if we do the brackets
thing above), ... which gives us sufficient coverage I think.

-Rob
-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud