[Python-Dev] Status of packaging in 3.3

[Donald Stufft]

On Friday, June 22, 2012 at 1:05 AM, Nick Coghlan wrote:
> 
> - I reject setup.cfg, as I believe ini-style configuration files are
> not appropriate for a metadata format that needs to include file
> listings and code fragments
> 
> - I reject bento.info (http://bento.info), as I think if we accept
> yet-another-custom-configuration-file-format into the standard library
> instead of just using YAML, we're even crazier than is already
> apparent
> 
> - I shall use "dist.yaml" as my proposed name for my "I wish I could
> define packages like this" format (and yes, that means adding yaml
> support to the standard library is part of the wish)
> 
> - many of the details below will be flawed, but I want to give a clear
> idea for how a concept like this might work in practice
> 
> - we need to define a clear set of build phases, and then design the
> dist metadata format accordingly. For example:
> - source
> - uses a "source" section in dist.yaml
> - "source/install" maps source files directly to desired
> install locations
> - essentially what the setup.cfg Resources section tries to do
> - used for pure Python code, documentation, etc
> - See below for example
> - "source/files" defines a list of extra files to be included
> - "source/exclude" defines the list of files to be excluded
> - "source/run" defines a Python fragment to be executed
> - serves a similar purpose to the "files" section in setup.cfg
> - creates a temporary directory (and sets it as the working directory)
> - dist.yaml is copied to the temporary directory
> - all files to be installed are copied to the temporary directory
> - all extra files are copied to the temporary directory
> - the Python fragment in "source/run" is executed (which can
> thus easily add more files)
> - if sdist archive creation is requested, entire contents of
> temporary directory are included
> - build
> - uses a "build" section in dist.yaml
> - "build/install" maps built files to desired install locations
> - like source/install, but for build artifacts
> - compiled C extensions, .pyc and .pyo files, etc would all go here
> - "build/run" defines a Python fragment to be executed
> - "build/files" defines the list of files to be included
> - "build/exclude" defines the list of files to be excluded
> - "build/requires" defines extra dependencies not needed at runtime
> - starting environment is a source directory that is either:
> - preexisting (e.g. to allow building in-place in the source tree)
> - created by running source first
> - created by unpacking an sdist archive
> - the Python fragment in "build/run" is executed to trigger the build
> - if the build succeeds (i.e. doesn't throw an exception)
> - create a temporary directory
> - copy dist.yaml
> - copy all specified files
> - this is the easiest way to exclude build artifacts from
> the distribution, while still keeping them around to enable
> incremental builds
> - if bdist_simple archive creation is requested, entire
> contents of temporary directory are included
> - other bdist formats (such as bdist_rpm) will have their own
> rules for getting from the bdist_simple format to the platform
> specific format
> - install
> - uses an "install" section in dist.yaml
> - "install/pre" defines a Python fragment to be executed
> before copying files
> - "install/post" defines a Python fragment to be executed
> after copying files
> - starting environment is a bdist_simple directory that is either:
> - preexisting (e.g. to allow creation by system packaging tools)
> - created by running build first
> - created by unpacking a bdist_simple archive
> - end result is a fully installed and usable piece of software
> - test
> - uses a "test" section in dist.yaml
> - "test/run" defines a Python fragment to be executed to start the tests
> - "test/requires" defines extra dependencies needed to run the
> test suite
> 
I dislike some of the (implementation) details, but in general I think this is a good direction 
to go in. Less trying to force tools to work together by hijacking setup.py or something
and more "this is a package, it contains the data you need to install, and how to install it, you
installation tool can use this data however it pleases to make sure it is installed." I feel like this
is (one of?) the missing piece of the puzzle to define a set of standards that _any_ package creation,
or installation tool can implement and gain interoperability.

I don't want to argue over implementation details as I think that is premature right now, so this
concept has a big +1 from me. RPM, deb, etc has a long history and a lot of shared knowledge
so looking at them and adapting it to work cross platform is likely to be huge win.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120622/e2110bc3/attachment.html>