[Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

[M.-A. Lemburg]

On 19.02.2013 14:40, Nick Coghlan wrote:
> On Tue, Feb 19, 2013 at 11:23 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 19.02.2013 11:28, Nick Coghlan wrote:
>>> On Tue, Feb 19, 2013 at 7:37 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>> On 17.02.2013 11:11, Nick Coghlan wrote:
>>>> I'm not against modernizing the format, but given that version 1.2
>>>> has been out for around 8 years now, without much following,
>>>> I think we need to make the implementation bit a requirement
>>>> before accepting the PEP.
>>>
>>> It is being implemented in distlib, and the (short!) appendix to the
>>> PEP itself shows how to read the format using the standard library's
>>> email module.
>>
>> Hmm, what is distlib and where does it live ?
> 
> As part of the post-mortem of packaging's removal from Python 3.3,
> several subcomponents were identified as stable and useful. distlib is
> those subcomponents extracted into a separate repository by Vinay
> Sajip.
> 
> It will be proposed as the standard library infrastructure for
> building packaging related tools, while distutils will become purely a
> build system and have nothing to do with installing software directly
> (except perhaps on developer machines).

Shouldn't those details be mentioned in the PEP ?

>> The PEP only shows how to parse the RFC822-style format used by the
>> metadata. That's not what I was referring to.
>>
>> If a tools wants to support metadata 2.0, it has to support all
>> the complicated stuff as well, i.e. handle the requires fields,
>> the environment markers and version comparisons/sorting.
> 
> Which is what distutils2 can be used for now, and what distlib will
> provide without the unwanted build system infrastructure in
> distutils2.
>
>>> v2.0 is designed to fix many of the issues that prevented the adoption
>>> of v1.2, including tweaks to the standardised version scheme and the
>>> addition of a formal extension mechanism to avoid the ad hoc
>>> extensions that occurred with earlier metadata versions.
>>
>> Some more notes:
>>
>> * I find it confusing that we now have two version schemes,
>>   one defined in PEP 426 (hidden in the middle of the document)
>>   and one in PEP 386. It would be better to amend or replace
>>   PEP 386, since that's where you look for Python version strings.
> 
> You can't understand version specifiers without understanding the sort
> order defined for the version scheme, so documenting them separately
> is just a recipe for confusion.

PEP 386 defines both. The point here is that the version scheme
goes far beyond the metadata format and is complicated enough
to warrant a separate PEP.

> I plan to mark PEP 386 as Withdrawn when I accept this PEP, as the
> sorting scheme it defines is broken, and the distutils changes
> proposed in that PEP are never going to happen.

Hmm, Tarek is the author, so only he can withdraw the PEP, AFAIK.

>> * PEP 426 doesn't include any mention of the egg distribution format,
>>   even though it's the most popular distribution format at the moment.
>>   It should at least include the location of the metadata file
>>   in eggs (EGG-INFO/PKG-INFO) and egg installations
>>   (<eggdir>/EGG-INFO/PKG-INFO).
> 
> "Other tools involved in Python distribution may also use this format."
> 
> The egg format has never been, and never will be, officially endorsed
> by python-dev. The wheel format is the standard format for binary
> distribution, and PEP 376 defines the standard location for metadata
> on installed distributions.

See my other reply.

>>
>> Not sure whether related or not, I also think it would be a good idea
>> to make the metadata file available on PyPI for download (could be sent
>> there when registering the package release). The register command
>> only posts the data as 1.0 metadata, but includes fields from
>> metadata 1.1. PyPI itself only displays part of the data.
> 
> It's not related, but I plan to propose the adoption of TUF (with GPG
> based signatures) for PyPI's end-to-end security solution, and the
> conversion of the metadata files to JSON for distribution through
> TUF's metadata support. (Donald Stufft already wrote  PEP 426 <-> JSON
> bidirectional converter as part of an unrelated experiment)

Why convert the metadata format you are defining in PEP 426
to yet another format when it can be uploaded as file straight
to PyPI ?

TUF doesn't have anything to do with that, agreed ;-)

>> It would be useful to have the metadata readily available for
>> inspection on PyPI without having to download one of the
>> distribution files first.
> 
> Indeed, which is a large part of why TUF (aka The Update Framework:
> https://www.updateframework.com/) is such an interesting security
> solution.

The suggestion to have the metadata available on PyPI doesn't
have anything to do with security.

It's about being able to determine compatibility and select the
right distribution file for download. The metadata also helps in
creating dependency graphs, which are useful for a lot of things.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Feb 19 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/