[Distutils] RFC: PEP 566 - Metadata for Python Software Packages 1.3

[Dustin Ingram]

Hi all,

I'd appreciate your feedback on the following Metadata 1.3 PEP.

The goal here is not to provide a full specification for all fields as
in previous PEPs, but to:

* Motivate and describe the addition of the new fields or changes to
existing fields;
* Point to the Core Metadata Specifications reference document as the
canonical source for the specification.

Previous discussions:
* https://mail.python.org/pipermail/distutils-sig/2017-September/031465.html
* https://github.com/python/peps/issues/388

Thanks,
D.


PEP: 566
Title: Metadata for Python Software Packages 1.3
Version: $Revision$
Last-Modified: $Date$
Author: Dustin Ingram <di at di.codes>
Discussions-To: distutils-sig <distutils-sig at python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 1-Dec-2017
Python-Version: 3.x
Post-History:
Replaces: 345


Abstract
========

This PEP describes the changes between versions 1.2 and 1.3 of the core
metadata specification for Python packages. Version 1.2 is specified in PEP
345.

It also changes to the canonical source for field specifications to the `Core
Metadata Specification`_ reference document, which includes specifics of the
field names, and their semantics and usage.

Fields
======

The canonical source for the names and semantics of each of the supported
metadata fields is the `Core Metadata Specification`_ document.

Fields marked with "(Multiple use)" may be specified multiple times in a single
PKG-INFO file.  Other fields may only occur once in a PKG-INFO file.  Fields
marked with "(optional)" are not required to appear in a valid PKG-INFO file;
all other fields must be present.

New in Version 1.3
------------------

Description-Content-Type (optional)
:::::::::::::::::::::::::::::::::::

A string stating the markup syntax (if any) used in the distribution's
description, so that tools can intelligently render the description.

Historically, tools like PyPI assume that a package's description is formatted
in `reStructuredText (reST)
<http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html>`_, and
fall back on plain text if the description is not valid reST.

The introduction of this field allows PyPI to support additional types of
markup syntax, and not need to make this assumption.

The full specification for this field is defined in the `Core Metadata
Specification`_.


Provides-Extra (optional, multiple use)
:::::::::::::::::::::::::::::::::::::::

A string containing the name of an optional feature. Must be a valid Python
identifier. May be used to make a dependency conditional on whether the
optional feature has been requested.

This introduction of this field allows packge installation tools (such as
``pip``) to determine which extras are provided by a given package, and so that
package publication tools (such as ``twine``) can check for issues with
environment markers which use extras.

The full specification for this field is defined in the `Core Metadata
Specification`_.

Changed in Version 1.3
----------------------

Name
::::

The specification for the format of this field is now identical to the
distribution name specification defined in PEP 508.

Version Specifiers
==================

Version numbering requirements and the semantics for specifying comparisons
between versions are defined in PEP 440.

Environment markers
===================

An **environment marker** is a marker that can be added at the end of a
field after a semi-colon (";"), to add a condition about the execution
environment.

The environment marker format used to declare such a condition is defined in
the environment markers section of PEP 508.

JSON-compatible Metadata
========================

It may be necessary to store metadata in a data structure which does not
allow for multiple repeated keys, such as JSON.

The canonical method to transform metadata fields into such a data structure is
as follows:

#. The original key-value format should be read with
   ``email.parser.HeaderParser``;
#. All transformed keys should be reduced to lower case, but otherwise should
   retain all other characters;
#. The transformed value for any field marked with "(Multiple-use") should be a
   single list containing all the original values for the given key;
#. The ``Keywords`` field should be converted to a list by splitting the
   original value on whitespace characters;
#. The result should be stored as a string-keyed dictionary.

Summary of Differences From PEP 345
===================================

* Metadata-Version is now 1.3.

* Fields are now specified via the `Core Metadata Specification`_.

* Added two new fields: ``Description-Content-Type`` and ``Provides-Extra``

* Acceptable values for the ``Name`` field are now specified as per PEP 508.

* Added canonical method of transformation into JSON-compatible data structure.

References
==========

This document specifies version 1.3 of the metadata format.
Version 1.0 is specified in PEP 241.
Version 1.1 is specified in PEP 314.
Version 1.2 is specified in PEP 345.

.. _`Core Metadata Specification`:
   https://packaging.python.org/specifications/core-metadata/

Copyright
=========

This document has been placed in the public domain.

Acknowledgements
================

Thanks to Nick Colgan for contributing to this PEP.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 80
   End: