[Python-checkins] cpython: Close #19407: New installation & distribution guides
nick.coghlan
python-checkins at python.org
Thu Mar 13 13:14:06 CET 2014
http://hg.python.org/cpython/rev/d22ef969cb82
changeset: 89629:d22ef969cb82
user: Nick Coghlan <ncoghlan at gmail.com>
date: Thu Mar 13 22:13:45 2014 +1000
summary:
Close #19407: New installation & distribution guides
- based on pip and other PyPA tools
- includes references to the new Python Packaging User Guide
where appropriate (and the relevant section is at least
partially filled in)
- started new FAQ sections
- both guides aim to introduce users to basic open source
concepts if they aren't aware of them
- existing guides have been relocated (now linked from the
distutils docs) rather then removed, since there is
some needed material that has yet to be relocated to the
distutils docs as a reference for the legacy formats
files:
Doc/conf.py | 4 +-
Doc/contents.rst | 12 +-
Doc/distributing/index.rst | 143 +++++++++
Doc/distutils/index.rst | 6 +-
Doc/extending/index.rst | 42 ++-
Doc/install/index.rst | 6 +-
Doc/installing/index.rst | 211 ++++++++++++++
Doc/library/distutils.rst | 19 +-
Doc/library/ensurepip.rst | 2 +-
Doc/tools/sphinxext/indexcontent.html | 8 +-
Doc/tutorial/whatnow.rst | 4 +-
Doc/using/venv-create.inc | 5 +
Doc/using/windows.rst | 4 +
Doc/whatsnew/3.4.rst | 19 +
Misc/NEWS | 6 +
15 files changed, 459 insertions(+), 32 deletions(-)
diff --git a/Doc/conf.py b/Doc/conf.py
--- a/Doc/conf.py
+++ b/Doc/conf.py
@@ -120,11 +120,11 @@
latex_documents = [
('c-api/index', 'c-api.tex',
'The Python/C API', _stdauthor, 'manual'),
- ('distutils/index', 'distutils.tex',
+ ('distributing/index', 'distributing.tex',
'Distributing Python Modules', _stdauthor, 'manual'),
('extending/index', 'extending.tex',
'Extending and Embedding Python', _stdauthor, 'manual'),
- ('install/index', 'install.tex',
+ ('installing/index', 'installing.tex',
'Installing Python Modules', _stdauthor, 'manual'),
('library/index', 'library.tex',
'The Python Library Reference', _stdauthor, 'manual'),
diff --git a/Doc/contents.rst b/Doc/contents.rst
--- a/Doc/contents.rst
+++ b/Doc/contents.rst
@@ -11,8 +11,8 @@
library/index.rst
extending/index.rst
c-api/index.rst
- distutils/index.rst
- install/index.rst
+ distributing/index.rst
+ installing/index.rst
howto/index.rst
faq/index.rst
glossary.rst
@@ -21,3 +21,11 @@
bugs.rst
copyright.rst
license.rst
+
+.. include legacy packaging docs in build
+
+.. toctree::
+ :hidden:
+
+ distutils/index.rst
+ install/index.rst
diff --git a/Doc/distributing/index.rst b/Doc/distributing/index.rst
new file mode 100644
--- /dev/null
+++ b/Doc/distributing/index.rst
@@ -0,0 +1,143 @@
+.. _distributing-index:
+
+###############################
+ Distributing Python Modules
+###############################
+
+:Email: distutils-sig at python.org
+
+
+As a popular open source development project, Python has an active
+supporting community of contributors and users that also make their software
+available for other Python developers to use under open source license terms.
+
+This allows Python users to share and collaborate effectively, benefiting
+from the solutions others have already created to common (and sometimes
+even rare!) problems, as well as potentially contributing their own
+solutions to the common pool.
+
+This guide covers the distribution part of the process. For a guide to
+installing other Python projects, refer to the
+:ref:`installation guide <installing-index>`.
+
+.. note::
+
+ For corporate and other institutional users, be aware that many
+ organisations have their own policies around using and contributing to
+ open source software. Please take such policies into account when making
+ use of the distribution and installation tools provided with Python.
+
+
+Key terms
+=========
+
+* the `Python Package Index <https://pypi.python.org/pypi>`__ is a public
+ repository of open source licensed packages made available for use by
+ other Python users
+* the `Python Packaging Authority
+ <http://packaging.python.org/en/latest/future.html>`__ are the group of
+ developers and documentation authors responsible for the maintenance and
+ evolution of the standard packaging tools and the associated metadata and
+ file format standards. They maintain a variety of tools, documentation
+ and issue trackers on both `GitHub <https://github.com/pypa>` and
+ `BitBucket <https://bitbucket.org/pypa/>`__.
+* ``distutils`` is the original build and distribution system first added to
+ the Python standard library in 1998. While direct use of ``distutils`` is
+ being phased out, it still laid the foundation for the current packaging
+ and distribution infrastructure, and it not only remains part of the
+ standard library, but its name lives on in other ways (such as the name
+ of the mailing list used to coordinate Python packaging standards
+ development).
+
+
+Open source licensing and collaboration
+=======================================
+
+In most parts of the world, software is automatically covered by copyright.
+This means that other developers require explicit permission to copy, use,
+modify and redistribute the software.
+
+Open source licensing is a way of explicitly granting such permission in a
+relatively consistent way, allowing developers to share and collaborate
+efficiently by making common solutions to various problems freely available.
+This leaves many developers free to spend more time focusing on the problems
+that are relatively unique to their specific situation.
+
+The distribution tools provided with Python are designed to make it
+reasonably straightforward for developers to make their own contributions
+back to that common pool of software if they choose to do so.
+
+The same distribution tools can also be used to distribute software within
+an organisation, regardless of whether that software is published as open
+source software or not.
+
+
+Installing the tools
+====================
+
+The standard library does not include build tools that support modern
+Python packaging standards, as the core development team has found that it
+is important to have standard tools that work consistently, even on older
+versions of Python.
+
+The currently recommended build and distribution tools can be installed
+using ``pip``::
+
+ pip install setuptools wheel twine
+
+
+Reading the guide
+=================
+
+The Python Packaging User Guide covers the various key steps and elements
+involved in creating a project
+
+* `Project structure`_
+* `Building and packaging the project`_
+* `Uploading the project to the Python Package Index`_
+
+.. _Project structure: \
+ http://packaging.python.org/en/latest/tutorial.html#creating-your-own-project
+.. _Building and packaging the project: \
+ http://packaging.python.org/en/latest/tutorial.html#building-packaging-your-project
+.. _Uploading the project to the Python Package Index: \
+ http://packaging.python.org/en/latest/tutorial.html#building-uploading-your-project-to-pypi
+
+
+How do I...?
+============
+
+These are quick answers or links for some common tasks.
+
+... choose a name for my project?
+---------------------------------
+
+This isn't an easy topic, but here are a few tips:
+
+* check the Python Package Index to see if the name is already in use
+* check popular hosting sites like GitHub, BitBucket, etc to see if there
+ is already a project with that name
+* check what comes up in a web search for the name you're considering
+* avoid particularly common words, especially ones with multiple meanings,
+ as they can make it difficult for users to find your software when
+ searching for it
+
+
+... create and distribute binary extensions?
+--------------------------------------------
+
+This is actually quite a complex topic, with a variety of alternatives
+available depending on exactly what you're aiming to achieve. See the
+Python Packaging User Guide for more information and recommendations.
+
+.. seealso::
+
+ `Python Packaging User Guide: Binary Extensions
+ <http://packaging.python.org/en/latest/extensions.html>`__
+
+.. other topics:
+
+ Once the Development & Deployment part of PPUG is fleshed out, some of
+ those sections should be linked from new questions here (most notably,
+ we should have a question about avoiding depending on PyPI that links to
+ http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches)
diff --git a/Doc/distutils/index.rst b/Doc/distutils/index.rst
--- a/Doc/distutils/index.rst
+++ b/Doc/distutils/index.rst
@@ -1,8 +1,8 @@
.. _distutils-index:
-###############################
- Distributing Python Modules
-###############################
+##############################################
+ Distributing Python Modules (Legacy version)
+##############################################
:Authors: Greg Ward, Anthony Baxter
:Email: distutils-sig at python.org
diff --git a/Doc/extending/index.rst b/Doc/extending/index.rst
--- a/Doc/extending/index.rst
+++ b/Doc/extending/index.rst
@@ -21,14 +21,31 @@
For a detailed description of the whole Python/C API, see the separate
:ref:`c-api-index`.
-.. note::
- This guide only covers the basic tools for creating extensions provided
- as part of this version of CPython. Third party tools may offer simpler
- alternatives. Refer to the `binary extensions section
- <https://python-packaging-user-guide.readthedocs.org/en/latest/extensions.html>`__
- in the Python Packaging User Guide for more information.
+Recommended third party tools
+=============================
+This guide only covers the basic tools for creating extensions provided
+as part of this version of CPython. Third party tools like Cython,
+``cffi``, SWIG and Numba offer both simpler and more sophisticated
+approaches to creating C and C++ extensions for Python.
+
+.. seealso::
+
+ `Python Packaging User Guide: Binary Extensions <https://packaging.python.org/en/latest/extensions.html>`_
+ The Python Packaging User Guide not only covers several available
+ tools that simplify the creation of binary extensions, but also
+ discusses the various reasons why creating an extension module may be
+ desirable in the first place.
+
+
+Creating extensions without third party tools
+=============================================
+
+This section of the guide covers creating C and C++ extensions without
+assistance from third party tools. It is intended primarily for creators
+of those tools, rather than being a recommended way to create your own
+C extensions.
.. toctree::
:maxdepth: 2
@@ -38,4 +55,17 @@
newtypes.rst
building.rst
windows.rst
+
+Embedding the CPython runtime in a larger application
+=====================================================
+
+Sometimes, rather than creating an extension that runs inside the Python
+interpreter as the main application, it is desirable to instead embed
+the CPython runtime inside a larger application. This section covers
+some of the details involved in doing that successfully.
+
+.. toctree::
+ :maxdepth: 2
+ :numbered:
+
embedding.rst
diff --git a/Doc/install/index.rst b/Doc/install/index.rst
--- a/Doc/install/index.rst
+++ b/Doc/install/index.rst
@@ -2,9 +2,9 @@
.. _install-index:
-*****************************
- Installing Python Modules
-*****************************
+********************************************
+ Installing Python Modules (Legacy version)
+********************************************
:Author: Greg Ward
diff --git a/Doc/installing/index.rst b/Doc/installing/index.rst
new file mode 100644
--- /dev/null
+++ b/Doc/installing/index.rst
@@ -0,0 +1,211 @@
+.. highlightlang:: none
+
+.. _installing-index:
+
+*****************************
+ Installing Python Modules
+*****************************
+
+:Email: distutils-sig at python.org
+
+As a popular open source development project, Python has an active
+supporting community of contributors and users that also make their software
+available for other Python developers to use under open source license terms.
+
+This allows Python users to share and collaborate effectively, benefiting
+from the solutions others have already created to common (and sometimes
+even rare!) problems, as well as potentially contributing their own
+solutions to the common pool.
+
+This guide covers the installation part of the process. For a guide to
+creating and sharing your own Python projects, refer to the
+:ref:`distribution guide <distributing-index>`.
+
+.. note::
+
+ For corporate and other institutional users, be aware that many
+ organisations have their own policies around using and contributing to
+ open source software. Please take such policies into account when making
+ use of the distribution and installation tools provided with Python.
+
+
+Key terms
+=========
+
+* ``pip`` is the preferred installer program. Starting with Python 3.4, it
+ is included by default with the Python binary installers.
+ installed into virtual environments
+* a virtual environment is a semi-isolated Python environment that allows
+ packages to be installed for use by a particular application, rather than
+ being installed system wide
+* ``pyvenv`` is the standard tool for creating virtual environments, and has
+ been part of Python since Python 3.3. Starting with Python 3.4, it
+ defaults to installing ``pip`` into all created virtual environments
+* the `Python Package Index <https://pypi.python.org/pypi>`__ is a public
+ repository of open source licensed packages made available for use by
+ other Python users
+* the `Python Packaging Authority
+ <http://packaging.python.org/en/latest/future.html>`__ are the group of
+ developers and documentation authors responsible for the maintenance and
+ evolution of the standard packaging tools and the associated metadata and
+ file format standards. They maintain a variety of tools, documentation
+ and issue trackers on both `GitHub <https://github.com/pypa>` and
+ `BitBucket <https://bitbucket.org/pypa/>`__.
+* ``distutils`` is the original build and distribution system first added to
+ the Python standard library in 1998. While direct use of ``distutils`` is
+ being phased out, it still laid the foundation for the current packaging
+ and distribution infrastructure, and it not only remains part of the
+ standard library, but its name lives on in other ways (such as the name
+ of the mailing list used to coordinate Python packaging standards
+ development).
+
+
+Basic usage
+===========
+
+The standard packaging tools are all designed to be used from the command
+line. For Windows users, the examples below assume that the option to
+adjust the system PATH environment variable was selected when installing
+Python. For Linux users, the command to install into the system version of
+Python 3 is likely to be ``pip3`` rather than ``pip``.
+
+The following command will install the latest version of a module and its
+dependencies from the Python Package Index::
+
+ pip install SomePackage
+
+It's also possible to specify an exact or minimum version directly on the
+command line::
+
+ pip install SomePackage==1.0.4 # specific version
+ pip install 'SomePackage>=1.0.4' # minimum version
+
+Normally, if a suitable module is already installed, attempting to install
+it again will have no effect. Upgrading existing modules must be requested
+explicitly::
+
+ pip install --upgrade SomePackage
+
+More information and resources regarding ``pip`` and its capabilities can be
+found in the `Python Packaging User Guide <http://packaging.python.org>`__.
+
+``pyvenv`` has its own documentation at :ref:`scripts-pyvenv`. Installing
+into an active virtual environment uses the commands shown above.
+
+.. seealso::
+
+ `Python Packaging User Guide: Installing Python packages
+ <http://packaging.python.org/en/latest/tutorial.html#installing-python-packages>`__
+
+
+How do I ...?
+=============
+
+These are quick answers or links for some common tasks.
+
+... install ``pip`` in versions of Python prior to Python 3.4?
+--------------------------------------------------------------
+
+Python only started bundling ``pip`` with Python 3.4. For earlier versions,
+``pip`` needs to be "bootstrapped" as described in the Python Packaging
+User Guide.
+
+.. seealso::
+
+ `Python Packaging User Guide: Installing the Tools
+ <http://packaging.python.org/en/latest/tutorial.html#installing-the-tools>`__
+
+
+.. installing-per-user-installation:
+
+... install packages just for the current user?
+-----------------------------------------------
+
+Passing the ``--user`` option to ``pip install`` will install a package
+just for the current user, rather than for all users of the system.
+
+
+... install scientific Python packages?
+---------------------------------------
+
+A number of scientific Python packages have complex binary dependencies, and
+aren't currently easy to install using ``pip`` directly. At this point in
+time, it will often be easier for users to install these packages by
+`other means
+<http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>`__
+rather than attempting to install them with ``pip``.
+
+.. seealso::
+
+ `Python Packaging User Guide: Installing Scientific Packages
+ <http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>`__
+
+
+... work with multiple versions of Python installed in parallel?
+----------------------------------------------------------------
+
+On Linux, Mac OS X and other POSIX systems, use the versioned Python commands
+in combination with the ``-m`` switch to run the appropriate copy of
+``pip``::
+
+ python2 -m pip install SomePackage # default Python 2
+ python2.7 -m pip install SomePackage # specifically Python 2.7
+ python3 -m pip install SomePackage # default Python 3
+ python3.4 -m pip install SomePackage # specifically Python 3.4
+
+(appropriately versioned ``pip`` commands may also be available)
+
+On Windows, use the ``py`` Python launcher in combination with the ``-m``
+switch::
+
+ py -2 -m pip install SomePackage # default Python 2
+ py -2.7 -m pip install SomePackage # specifically Python 2.7
+ py -3 -m pip install SomePackage # default Python 3
+ py -3.4 -m pip install SomePackage # specifically Python 3.4
+
+.. other questions:
+
+ Once the Development & Deployment part of PPUG is fleshed out, some of
+ those sections should be linked from new questions here (most notably,
+ we should have a question about avoiding depending on PyPI that links to
+ http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches)
+
+
+Common installation issues
+==========================
+
+Installing into the system Python on Linux
+------------------------------------------
+
+On Linux systems, a Python installation will typically be included as part
+of the distribution. Installing into this Python installation requires
+root access to the system, and may interfere with the operation of the
+system package manager and other components of the system if a component
+is unexpectedly upgraded using ``pip``.
+
+On such systems, it is often better to use a virtual environment or a
+per-user installation when installing packages with ``pip``.
+
+
+Installing binary extensions
+----------------------------
+
+Python has typically relied heavily on source based distribution, with end
+users being expected to compile extension modules from source as part of
+the installation process.
+
+With the introduction of support for the binary ``wheel`` format, and the
+ability to publish wheels for at least Windows and Mac OS X through the
+Python Package Index, this problem is expected to diminish over time,
+as users are more regularly able to install pre-built extensions rather
+than needing to build them themselves.
+
+Some of the solutions for installing `scientific software
+<http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>`__
+that is not yet available as pre-built ``wheel`` files may also help with
+obtaining other binary extensions without needing to build them locally.
+
+.. seealso::
+
+ `Python Packaging User Guide: Binary Extensions
+ <http://packaging.python.org/en/latest/extensions.html>`__
diff --git a/Doc/library/distutils.rst b/Doc/library/distutils.rst
--- a/Doc/library/distutils.rst
+++ b/Doc/library/distutils.rst
@@ -12,14 +12,15 @@
100%-pure Python, or may be extension modules written in C, or may be
collections of Python packages which include modules coded in both Python and C.
+Most Python users will *not* want to use this module directly, but instead
+use the cross-version tools maintained by the Python Packaging Authority.
+Refer to the `Python Packaging User Guide <http://packaging.python.org>`_
+for more information.
-User documentation and API reference are provided in another document:
+For the benefits of packaging tool authors and users seeking a deeper
+understanding of the details of the current packaging and distribution
+system, the legacy :mod:`distutils` based user documentation and API
+reference remain available:
-.. seealso::
-
- :ref:`distutils-index`
- The manual for developers and packagers of Python modules. This describes
- how to prepare :mod:`distutils`\ -based packages so that they may be
- easily installed into an existing Python installation. It also contains
- instructions for end-users wanting to install a distutils-based package,
- :ref:`install-index`.
+* :ref:`install-index`
+* :ref:`distutils-index`
diff --git a/Doc/library/ensurepip.rst b/Doc/library/ensurepip.rst
--- a/Doc/library/ensurepip.rst
+++ b/Doc/library/ensurepip.rst
@@ -28,7 +28,7 @@
.. seealso::
- :ref:`install-index`
+ :ref:`installing-index`
The end user guide for installing Python packages
:pep:`453`: Explicit bootstrapping of pip in Python installations
diff --git a/Doc/tools/sphinxext/indexcontent.html b/Doc/tools/sphinxext/indexcontent.html
--- a/Doc/tools/sphinxext/indexcontent.html
+++ b/Doc/tools/sphinxext/indexcontent.html
@@ -16,14 +16,14 @@
<p class="biglink"><a class="biglink" href="{{ pathto("howto/index") }}">Python HOWTOs</a><br/>
<span class="linkdescr">in-depth documents on specific topics</span></p>
</td><td width="50%">
+ <p class="biglink"><a class="biglink" href="{{ pathto("installing/index") }}">Installing Python Modules</a><br/>
+ <span class="linkdescr">installing from the Python Package Index & other sources</span></p>
+ <p class="biglink"><a class="biglink" href="{{ pathto("distributing/index") }}">Distributing Python Modules</a><br/>
+ <span class="linkdescr">publishing modules for installation by others</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("extending/index") }}">Extending and Embedding</a><br/>
<span class="linkdescr">tutorial for C/C++ programmers</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("c-api/index") }}">Python/C API</a><br/>
<span class="linkdescr">reference for C/C++ programmers</span></p>
- <p class="biglink"><a class="biglink" href="{{ pathto("install/index") }}">Installing Python Modules</a><br/>
- <span class="linkdescr">information for installers & sys-admins</span></p>
- <p class="biglink"><a class="biglink" href="{{ pathto("distutils/index") }}">Distributing Python Modules</a><br/>
- <span class="linkdescr">sharing modules with others</span></p>
<p class="biglink"><a class="biglink" href="{{ pathto("faq/index") }}">FAQs</a><br/>
<span class="linkdescr">frequently asked questions (with answers!)</span></p>
</td></tr>
diff --git a/Doc/tutorial/whatnow.rst b/Doc/tutorial/whatnow.rst
--- a/Doc/tutorial/whatnow.rst
+++ b/Doc/tutorial/whatnow.rst
@@ -21,8 +21,8 @@
and many other tasks. Skimming through the Library Reference will give you an
idea of what's available.
-* :ref:`install-index` explains how to install external modules written by other
- Python users.
+* :ref:`installing-index` explains how to install additional modules written
+ by other Python users.
* :ref:`reference-index`: A detailed explanation of Python's syntax and
semantics. It's heavy reading, but is useful as a complete guide to the
diff --git a/Doc/using/venv-create.inc b/Doc/using/venv-create.inc
--- a/Doc/using/venv-create.inc
+++ b/Doc/using/venv-create.inc
@@ -11,6 +11,11 @@
Windows). It also creates an (initially empty) ``lib/pythonX.Y/site-packages``
subdirectory (on Windows, this is ``Lib\site-packages``).
+.. seealso::
+
+ `Python Packaging User Guide: Creating and using virtual environments
+ <http://packaging.python.org/en/latest/tutorial.html#creating-and-using-virtual-environments>`__
+
.. highlight:: none
On Windows, you may have to invoke the ``pyvenv`` script as follows, if you
diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst
--- a/Doc/using/windows.rst
+++ b/Doc/using/windows.rst
@@ -11,6 +11,10 @@
This document aims to give an overview of Windows-specific behaviour you should
know about when using Python on Microsoft Windows.
+.. XXX (ncoghlan)
+
+ This looks rather stale to me...
+
Installing Python
=================
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -180,6 +180,9 @@
PEP 453: Explicit Bootstrapping of PIP in Python Installations
--------------------------------------------------------------
+Bootstrapping pip by default
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard
cross-platform mechanism to bootstrap the pip installer into Python
installations and virtual environments.
@@ -207,6 +210,22 @@
__ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors
+
+Documentation changes
+~~~~~~~~~~~~~~~~~~~~~
+
+As part of this change, the :ref:`installing-index` and
+:ref:`distributing-index` sections of the documentation have been
+completely redesigned as short getting started and FAQ documents. Most
+packaging documentation has now been moved out to the Python Packaging
+Authority maintained `Python Packaging User Guide
+<http://packaging.python.org>`__ and the documentation of the individual
+projects.
+
+However, as this migration is currently still incomplete, the legacy
+versions of those guides remaining available as :ref:`install-index`
+and :ref:`distutils-index`.
+
.. note::
To avoid conflicts between parallel Python 2 and Python 3 installations,
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -59,6 +59,12 @@
- Issue #20765: Add missing documentation for PurePath.with_name() and
PurePath.with_suffix().
+- Issue #19407: New package installation and distribution guides based on
+ the Python Packaging Authority tools. Existing guides have been retained
+ as legacy links from the distutils docs, as they still contain some
+ required reference material for tool developers that isn't recorded
+ anywhere else.
+
Tests
-----
--
Repository URL: http://hg.python.org/cpython
More information about the Python-checkins
mailing list