[Python-Dev] Distutils ML wrap-up: setup.cfg new format

[David Lyon]

Tarek,

Are you claiming this as your own work and ideas?

Given:

http://mail.python.org/pipermail/distutils-sig/2009-August/012998.html

Regards

David


On Tue, 22 Sep 2009 15:21:06 +0200, Tarek Ziadé <ziade.tarek at gmail.com>
wrote:
> Hello
> 
> Here's a wrapup of the Distutils-SIG discussion
> we had on the "static metadata" topic.
> 
> I realize that it's a good thing to send in.
> python-dev such wrapup on distutils design
> decisions, to keep everyone informed and get
> more feedback when required.
> 
> I will try to do it for every upcoming changes
> that are not going in a PEP process (when it's not
> a 'big' change). The rate of such mails should
> not be very high. (around 1 mail in python-dev
> for +150 mails in distutils-SIG)
> 
> If you feel that what we are about to change in distutils
> is wrong, you can go ahead and help us by participating
> in Distutils-ML, so we keep one and only one media
> for these discussions.
> 
> The four sentences summary for people in a hurry:
> 
>     Getting metadata of a distribution that is not.
>     installed means running its setup.py. This means.
>     downloading the whole archive, and running.
>     third party code on your system.
> 
>     To avoid it, we are adding a [setup] section in.
>     setup.cfg where you can express the package metadata
>     statically.
> 
>     Conditional sections, specific to some system.
>     can be added to add some specific fields in [setup].
> 
>     At the end, you will be able to get metadata fields
>     without running any third-party code, and possibly
>     get only the distribution setup.cfg for this.
> 
> Now the long version.
> 
> 
> Rational
> ========
> 
> Today, if you want to list all the metadata (PEP 314) of a.
> distribution that is not installed, you need to use it's.
> setup.py CLI.
> 
> So, basically, you download it, and run::
> 
>     $ python setup.py --name
>     Foo
> 
>     $ python setup.py --version
>     1.2
> 
> Where `name` and `version` are metadata fields. That's OK but as.
> soon as the developers add more code in setup.py, this feature
> might break or (worse) might do unwanted things on the target.
> system.
> 
> Why should we run third-party code just to get its metadata ?
> 
> So we worked on a way to express those metadata statically,
> by pushing them in `setup.cfg`. That's not hard, since all.
> the metadata fields are static values.
> 
> Adding a setup section in setup.cfg
> ===================================
> 
> So the first thing we want to introduce is a [setup] section,
> that may contain any field from the metadata::
> 
>     [setup]
>     name: Foo
>     version: 1.2
> 
> Distutils will look for them, and will use them. Of course
> setup.py is still required, and in case an option that is.
> a metadata field is passed to setup(), it will override one.
> located in setup.cfg.
> 
> PEP 341 is coming up
> ====================
> 
> Remember the last Pycon Summit ? We said that we would
> introduce a new metadata field to describe requirements..
> That's basically what PEP 341 is about, and we are still.
> working on it.
> 
> Basically, we will be able to write things like::
> 
>     requires: Twisted == 8.2.0
> 
> What takes us so long is that adding requirements like
> this in the metadata requires more things:
> 
> - the ability to compare versions (this was described in.
>   PEP 386 but not finished yet)
> 
> - the ability to make these requirements vary depending on.
>   the target system
> 
> And the later makes our "setup.cfg" new [setup] section.
> obsolete as soon as this new metadata field is added in.
> Python.
> 
> So we need more that what I've shown in the previous section
> 
> Context-dependant sections
> ==========================
> 
> The second change we are going to introduce is context-dependant
> sections in setup.cfg.
> 
> A context dependant section is a section with a condition that is.
> used only if the execution environment meets its condition.
> 
> Here's an example::
> 
>     [setup]
>     name: Foo
>     version: 1.3
> 
>     [setup:sys_platform == 'win32']
>     requires: pywin32
>     requires: bar > 1.0
> 
>     [setup:os_machine == '64bits']
>     requires: some_package
> 
>     [setup:python_version == '2.4' or python_version == '2.5']
>     requires: some_package
> 
> 
> Every [setup:condition] section will be added in [setup]
> only if the condition is met.
> 
> The micro-language behind this is the simplest possible:
> it compares only strings, with usual string operators,
> and with the ability to combine expressions. It makes it also
> easy to understand to non-pythoneers (os packagers).
> 
> The pseudo-grammar is (I don't know how to write those but you'll
> get it hopefully)::
> 
>     comp: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'
>     comparison: expr (comp_op expr)*
>     expr: STRING
>     test: or_test
>     or_test: and_test ('or' and_test)*
>     and_test: not_test ('and' not_test)*
>     not_test: 'not' not_test | comparison
> 
> where STRING belongs to any of those:
> 
> - python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
> - os_name = os.name
> - sys_platform = sys.platform
> - os_sysname = os.uname()[0].
> - os_nodename = os.uname()[1]
> - os_release = os.uname()[2].
> - os_version = os.uname()[3]..
> - os_machine = os.uname()[4].
> - a free string, like '2.4', or 'win32'
> 
> Distutils will provide a function that is able to read the metadata
> of a distribution, given a setup.cfg file, for the target environment::
> 
>     >>> from distutils.util import local_metadata
>     >>> local_metadata('setup.cfg')
>     <DistributionMetadata instance>
> 
> Meaning that a Vanilla Python will be able to read the metadata
> of a package without running any code.
> 
> That's what distutils will use when setup.py is run.
> 
> That's what any package manager will be able to use to work.
> with distributions.
> 
> Limitations
> ===========
> 
> If a distribution is unable for obscure reasons (or does not
> wish) to set all metadata fields in setup.cfg, that's fine :.
> the fields will be set to UNKNOWN when `local_metadata` is called.
> 
> When setup.py is run, options passed to setup() will complete
> those.
> 
> When a package manager gets 'UNKOWN' values, it knows it might
> need to do further processing by running setup.py.
> 
> Why this change is good
> =====================
> 
> Once the requires metadata is added in PEP 341,
> being able to query the metadata for a distribution can be done
> without doing anything else than reading a static file and interpreting
> its conditions in a restricted and secured manner.
> 
> So there's no need to run third-party code anymore.
> 
> Possible use cases:
> 
> - setup.cfg is published on PyPI. Packages managers like easy_install
>   or pip will be able to list all the distributions needed and their
>   versions for installing a distribution without having to download
>   anything !
> 
> - OS packagers will be able to list the dependencies of a distribution
>   without having to browse Python code.
> 
> And, again, this new [setup] section is backward-compatible, and will
> be a good complement to the work done in PEP 376.
> 
> Thanks for reading so far. This code will be added in 2.7 and 3.2.
> 
> Tarek