[Distutils] Updated drafts of metadata 2.0 (PEP 426 and PEP 440)

[holger krekel]

On Fri, Jun 21, 2013 at 01:05 +1000, Nick Coghlan wrote:
> On 21 Jun 2013 00:36, "holger krekel" <holger at merlinux.eu> wrote:
> >
> >
> > I still think the testing part of "the interchange format
> > between software publication and integration tools" is underspecified.
> > The dependencies alone will not be sufficient to allow the running
> > of tests in many cases.  Or am i missing something?
> 
> If "setup.py test" works for a distribution today, it will still work under
> PEP 426. Anything more is deliberately deferred (as noted in the PEP).

I am a little confused.  Early in the PEP you say:

    The published metadata for distributions SHOULD allow integrators,
    with the aid of build and integration tools, to:
   
    ...
    * if supported by the distribution, run the distribution's automatic
      test suite on an installed instance of the distribution

but a few sentences later:

    The current iteration of the metadata relies on the distutils
    commands system to support other necessary integration and
    deployment activities:

    ...
    python setup.py test: run the distribution's test suite on a built
    (but not yet installed) distribution 

And i am not sure what exactly PEP426 defines wrt to testing now.
I saw the "test_requires" and "test_may_requires" fields and thought
they were part of an idea already how to configure running of tests.
But is there anything more in the current PEP426 that i missed?

To help with my confusion, could you maybe describe how "python setup.py
test" is to interoperate (if at all) with the new metadata format as is?

thanks,
holger


> Cheers,
> Nick.
> 
> >
> > best and thanks for your good work,
> >
> > holger
> >
> > On Thu, Jun 20, 2013 at 23:07 +1000, Nick Coghlan wrote:
> > > New versions of PEP 426 and PEP 440 are up on python.org:
> > >
> > > PEP 426 (metadata 2.0): http://www.python.org/dev/peps/pep-0426/
> > > PEP 440 (version spec): http://www.python.org/dev/peps/pep-0440/
> > >
> > > (as before, not including them inline due to sheer length)
> > >
> > > The bulk of the changes are in this commit:
> > > http://hg.python.org/peps/rev/de305af601fa
> > > (there are a few minor tweaks in subsequent commits to the PEPs repo)
> > >
> > > There have been several significant changes to the main metadata spec
> > > in PEP 426:
> > >
> > > * Added a basic "jsonschema" based description of the format
> > > * The "abbreviated metadata" notion is gone, replaced by "included
> > > documents". The metadata can now list the names of additional files
> > > included in the distinfo directory for the long description, the
> > > license and the change history. The contents are no longer embedded
> > > directly in the metadata (but will be extracted and made available by
> > > PyPI, with the markup format being determined from the file extension,
> > > just as it is by sites like GitHub)
> > > * The dependency system has been redesigned under the name "Semantic
> > > dependencies" (as they now allow a distribution to better say *why* a
> > > dependency is needed, rather than just saying "I need this" with
> > > almost no capacity to say why). There are now five kinds of
> > > dependencies (:meta:, :run:, :test:, :build: and :dev:) and they're
> > > integrated into the extras system so you can easily say you want to
> > > install some, none or all of them.
> > > * The "*" notation is added to extras to make it easier to say
> > > "install all dependencies", along with the "-extra_name" notation to
> > > exclude the dependencies for specific extras
> > > * The "-" notation is added to extras to make it easier to install
> > > *just* a distribution's dependencies, without installing the
> > > distribution itself.
> > > * "Install hooks" are now a distinct concept from the
> > > still-hypothetical "metabuild" system, and place more constraints on
> > > their expected handling (installation tools are also explicitly
> > > permitted to refuse to run them, but doing so is required to fail
> > > noisily rather than silently appearing to succeed)
> > >
> > > The most significant change to PEP 440 is to the handling of
> > > pre-releases: whether or not pre-releases should be accepted by
> > > default is now determined solely by whether or not there is a stable
> > > release that *also* satisfies the version specifier. Reference a
> > > pre-release (or not) now has no effect on whether pre-releases are
> > > considered viable candidates. Pre-releases are now accepted if:
> > > * they're already installed
> > > * there's no other available option
> > > * the installation tool is told specifically to consider them
> > >
> > > Other less significant changes to PEP 426 include:
> > >
> > > * Longer introduction giving more context for the nature and purpose
> > > of the metadata
> > > * Separated various other things out into appendices
> > > * Various tweaks to definitions (including the "Release" tweak from
> > > PEP 440, and switching "source archive" to refer to the original raw
> > > source, while using "sdist" for the Python specific format with the
> > > extra metadata)
> > > * Blanket permission for distribution related online services to
> > > impose additional restrictions to protect the integrity of the service
> > > (such as additional length limits beyond those stated in the PEP).
> > > * Explicitly require UTF-8 encoded JSON for serialisation
> > > * build_label renamed to source_label
> > > * version_url renamed to source_url
> > > * tightened up the validation rules for various fields (including RFC
> > > references where appropriate). Many of these "new" rules are things
> > > projects already comply with (because not complying doesn't work
> > > properly). Including them in the spec is about giving PyPI clear
> > > guidance to enforce them at point of upload, rather than leaving it to
> > > installation tools to try to sort out later.
> > > * a few more additions to the "Rejected Features" appendix (notably,
> > > my rationale for *not* requiring the install hooks to accept arbitrary
> > > keyword arguments)
> > >
> > > The other PEP 440 changes are also relatively minor:
> > >
> > > - what were previously called releases are now "final releases",
> > > freeing up "release" as a general term for any kind of release
> > > (developmental release, pre-release, final release, post release).
> > > - "source references" are now "direct references" and can also refer
> > > to prebuilt wheel files
> > > - automated tools (especially index servers) are given a lot of
> > > freedom to be opinionated about the appropriate uses for direct
> > > references
> > > - a few tweaks to the security guidelines for direct references
> > > - pytz/Olson database version translation is explicitly discussed (add
> > > a leading 0., convert the trailing letter to an incrementing number)
> > > - tighter rules for what characters are allowed in a source label
> > >
> > > The only major remaining open item is the addition of guidelines in
> > > Appendix A for converting legacy metadata to metadata 2.0. I consider
> > > the rest of the spec stable at this point, unless anyone picks up on a
> > > glaring hole in the latest draft that Daniel, Donald and I missed.
> > >
> > > Cheers,
> > > Nick.
> > >
> > > --
> > > Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> > > _______________________________________________
> > > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > > http://mail.python.org/mailman/listinfo/distutils-sig
> > >