[Python-Dev] Edits to Metadata 1.2 to add extras (optional dependencies)

[Nick Coghlan]

On Tue, Aug 28, 2012 at 6:29 AM, "Martin v. Löwis" <martin at v.loewis.de> wrote:
> Am 27.08.12 16:56, schrieb Daniel Holth:
>> Metadata 1.2 is nearly 8 years old and it's Accepted but not Final. Is
>> it better to continue editing it, or create a new PEP for Metadata
>> 1.3?
>
>
> You can't add new fields to the format after the fact, unless the format had
> provided for such additions (which it does not - there is
> no mention of custom fields anywhere, and no elaboration on how
> "unknown" fields should be processed).
>
> So if you want to add new fields, you need to create a new version
> of the metadata.

I agree with this point - the main reason the metadata PEP is still
lingering at Accepted rather than Final is the tangled relationship
between distutils and other projects that led to the complete
distutils feature freeze. Until distutils2 makes it into the standard
library as the packaging module, the standard library is going to be
stuck at v1.1 of the metadata format.

> Prepare for a ten-year period of acceptance - so it
> would be good to be sure that no further additions are desired within
> the next ten years before seeking approval for the PEP.

However, this point I really don't agree with. The packaging ecosystem
is currently evolving outside the standard library, but the
standardisation process for the data interchange formats still falls
under the authority of python-dev and the PEP process.

If there are things missing from v1.2 of the metadata spec, then
define v1.3 to address those known problems. Don't overengineer it in
an attempt to anticipate every possible need that might come in the
next decade. Tools outside the standard library are then free to adopt
the new standard, even while the stdlib itself continues to lag
behind.

When the packaging module is finally added (hopefully 3.4, even if
that means we have to temporarily cull the entire compiler
subpackage), it will handle the most recent accepted version of the
metadata format (as well as any previous versions). If more holes
reveal themselves in the next 18 months, then it's OK if v1.4 is
created when it becomes clear that it's necessary.

At the very least, something v1.3 should make explicit is that custom
metadata should NOT be put into the .dist-info/METADATA (PEP 376
location, PKG-INFO, in distutils terms) file. Instead, that data
should be placed in a *separate* file in the .dist-info directory.
Something that *may* be appropriate is a new field in METADATA that
explicitly calls out such custom metadata files by naming the PyPI
distribution that is the authority for the relevant format (e.g.
"Custom-Metadata: wheel" to indicate that 'wheel' defined metadata is
present)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia