[Python-Dev] Status of packaging in 3.3

[Paul Moore]

Can I take a step back and make a somewhat different point.

Developer requirements are very relevant, sure. But the most important
requirements are those of the end user. The person who simply wants to
*use* a distribution, couldn't care less how it was built, whether it
uses setuptools, or whatever.

End users should not need packaging tools on their machines.

At the moment, to install from source requires the tools the developer
chooses to use for his convenience (distribute/setuptools,
distutils2/packaging, bento) to be installed on the target machine.
And binary installers are only normally available for code that needs
a C extension, and in that case the developer's choice is still
visible in terms of the binary format provided.

I would argue that we should only put *end user* tools in the stdlib.
- A unified package format, suitable for binaries, but also for pure
Python code that wants to ship that way.
- Installation management tools (download, install, remove, list, and
dependency management) that handle the above package format
- Maybe support in the package format and/or installation tools for
managing "wrapper executables" for executable scripts in distributions

Development tools like distutils2, distribute/setuptools, bento would
*only* be needed on developer machines, and would be purely developer
choice. They would all interact with end users via the
stdlib-supported standard formats. They could live outside the stdlib,
and developers could use whichever tool suited them.

This is a radical idea in that it does not cater for the "zipped up
development directory as a distribution format" mental model that
current Python uses. That model could still work, but only if all the
tools generated a stdlib-supported build definition (which could
simply be a Python script that runs the various compile/copy commands,
plus some compiler support classes in the stdlib) in the same way that
lex/yacc generate C, and projects often distribute the generated C
along with the grammar files.

Legacy support in the form of distutils, converters from bdist_xxx
formats to the new binary format, and maybe pip-style "hide the
madness under a unified interface" tools could support this, either in
the stdlib or as 3rd party tools.

I realise this is probably too radical to happen, but at least, it
might put the debate into context if people try to remember that end
users, as well as package developers, are affected by this (and there
are a lot more end users than package developers...).

Paul.

PS I know that setuptools includes some end-user aspects -
multi-versioning, entry points and optional dependencies, for example.
Maybe these are needed - personally, I have never had a need for any
of these, so I'm not the best person to comment.