[Python-checkins] peps: PEP 441: Updates from Paul Moore.

berker.peksag python-checkins at python.org
Fri Feb 20 14:51:25 CET 2015 

https://hg.python.org/peps/rev/6dfeec7203f1
changeset:   5699:6dfeec7203f1
user:        Berker Peksag <berker.peksag at gmail.com>
date:        Fri Feb 20 15:51:59 2015 +0200
summary:
  PEP 441: Updates from Paul Moore.

files:
  pep-0441.txt |  265 ++++++++++++++++++++++++++++++--------
  1 files changed, 210 insertions(+), 55 deletions(-)


diff --git a/pep-0441.txt b/pep-0441.txt
--- a/pep-0441.txt
+++ b/pep-0441.txt
@@ -2,12 +2,14 @@
 Title: Improving Python ZIP Application Support
 Version: $Revision$
 Last-Modified: $Date$
-Author: Daniel Holth <dholth at gmail.com>
+Author: Daniel Holth <dholth at gmail.com>,
+        Paul Moore <p.f.moore at gmail.com>
+Discussions-To: https://mail.python.org/pipermail/python-dev/2015-February/138277.html
 Status: Draft
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 30 March 2013
-Post-History: 30 March 2013, 1 April 2013
+Post-History: 30 March 2013, 1 April 2013, 16 February 2015
 
 Improving Python ZIP Application Support
 ========================================
@@ -15,116 +17,269 @@
 Python has had the ability to execute directories or ZIP-format
 archives as scripts since version 2.6 [1]_.  When invoked with a zip
 file or directory as its first argument the interpreter adds that
-directory to sys.path and executes the __main__ module.  These
+directory to sys.path and executes the ``__main__`` module.  These
 archives provide a great way to publish software that needs to be
 distributed as a single file script but is complex enough to need to
 be written as a collection of modules.
 
-This feature is not as popular as it should be mainly because no one’s
-heard of it as it wasn’t promoted as part of Python 2.6 [2]_, but also
-because Windows users don’t have a file extension (other than .py) to
-associate with the launcher.
+This feature is not as popular as it should be mainly because it was
+not promoted as part of Python 2.6 [2]_, so that it is relatively
+unknown, but also because the Windows installer does not register a
+file extension (other than ``.py``) for this format of file, to associate
+with the launcher.
 
 This PEP proposes to fix these problems by re-publicising the feature,
-defining the .pyz and .pyzw extensions as “Python ZIP Applications”
-and “Windowed Python ZIP Applications”, and providing some simple
+defining the ``.pyz`` and ``.pyzw`` extensions as "Python ZIP Applications"
+and "Windowed Python ZIP Applications", and providing some simple
 tooling to manage the format.
 
 A New Python ZIP Application Extension
 ======================================
 
-The Python 3.4 installer will associate .pyz and .pyzw “Python ZIP
-Applications” with the platform launcher so they can be executed.  A
-.pyz archive is a console application and a .pyzw archive is a
+The terminology "Python Zip Application" will be the formal term used
+for a zip-format archive that contains Python code in a form that can
+be directly executed by Python (specifically, it must have a
+``__main__.py`` file in the root directory of the archive).  The
+extension ``.pyz`` will be formally associated with such files.
+
+The Python 3.5 installer will associate ``.pyz`` and ``.pyzw`` "Python
+Zip Applications" with the platform launcher so they can be executed.
+A ``.pyz`` archive is a console application and a ``.pyzw`` archive is a
 windowed application, indicating whether the console should appear
 when running the app.
 
-Why not use .zip or .py? Users expect a .zip file would be opened with
-an archive tool, and users expect .py to contain text.  Both would be
-confusing for this use case.
+On Unix, it would be ideal if the ``.pyz`` extension and the name
+"Python Zip Application" were registered (in the mime types database?).
+However, such an association is out of scope for this PEP.
 
-For UNIX users, .pyz applications should be prefixed with a #! line
+Python Zip applications can be prefixed with a ``#!`` line
 pointing to the correct Python interpreter and an optional
 explanation::
 
     #!/usr/bin/env python3
-    # This is a Python application stored in a ZIP archive, created with
-    # pyzaa.
+    #  Python application packed with zipapp module
     (binary contents of archive)
 
+On Unix, this allows the OS to run the file with the correct
+interpreter, via the standard "shebang" support.  On Windows, the
+Python launcher implements shebang support.
+
+However, it is always possible to execute a ``.pyz`` application by
+supplying the filename to the Python interpreter directly.
+
 As background, ZIP archives are defined with a footer containing
 relative offsets from the end of the file.  They remain valid when
 concatenated to the end of any other file.  This feature is completely
 standard and is how self-extracting ZIP archives and the bdist_wininst
 installer format work.
 
-Minimal Tooling: The pyzaa Module
-=================================
+Minimal Tooling: The zipapp Module
+==================================
 
-This PEP also proposes including a simple application for working with
-these archives: The Python Zip Application Archiver “pyzaa” (rhymes
-with “huzzah” or “pizza”).  “pyzaa” can archive these files, compile
-bytecode, and can write the __main__ module if it is not present.
+This PEP also proposes including a module for working with these
+archives.  The module will contain functions for working with Python
+zip application archives, and a command line interface (via ``python
+-m zipapp``) for their creation and manipulation.
 
-Usage
------
+More complete tools for managing Python Zip Applications are
+encouraged as 3rd party applications on PyPI.  Currently, pyyzer [5]_
+and pex [6]_ are two tools known to exist.
 
-``python -m pyzaa (pack | compile)``
+Module Interface
+----------------
 
-``python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] [-w]
-[-p interpreter] directory``::
+The zipapp module will provide the following functions::
 
-    ZIP the contents of directory as directory.pyz or [-w]
-    directory.pyzw. Adds the executable flag to the archive.
+    pack(target, directory, interpreter=None, main=None)
 
-    ``-c`` compile .pyc files and add them to the archive
+Writes an application archive called *target*, containing the
+contents of *directory*.  If *interpreter* is specified, it will
+be written to the start of the archive as a shebang line and the file
+will be made executable (if no interpreter is specified, the shebang
+line will be omitted).  If the directory contains no ``__main__.py``
+file, the function will construct a ``__main__.py`` which calls the
+function specified in the *main* argument (which should be in the
+form ``'pkg.mod:fn'``).
 
-    ``-p interpreter`` include #!interpreter as the first line of the archive
+It is an error to specify *main* if the directory contains a
+``__main__.py``, or to omit *main* when there is no ``__main__.py``
+(as that will result in an archive which has no main function and so
+cannot be executed).
 
-    ``-o path/name`` archive is written to path/name.pyz[w] instead of
-    dirname. The extension is added if not specified.
+``get_interpreter(archive)``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-    ``-m module.submodule:callable`` __main__.py is written as “import
-    module.submodule; module.submodule.callable()”
+Returns the interpreter specified in the shebang line of the *archive*.
+If there is no shebang, the function returns ``None``.
 
-    pyzaa pack will warn if the directory contains C extensions or if
-    it doesn’t contain __main__.py.
+``set_interpreter(archive, new_archive, interpreter=None)``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-``python -m pyzaa compile arcname.pyz[w]``::
+Modifies the *archive*'s shebang line to contain the specified
+interpreter, and writes the updated archive to *new_archive*.  If the
+*interpreter* is ``None``, removes the shebang line.
 
-    The Python files in arcname.pyz[w] are compiled and appended to the
-    ZIP file.
+Command Line Usage
+------------------
 
-A standard ZIP utility or Python’s zipfile module can unpack the
-archives.
+The zipapp module can be run with the python ``-m`` flag.  The command
+line interface is as follows::
+
+    python -m zipapp [options] directory
+
+        Create an archive from the contents of the given directory. By
+        default, an archive will be created with the same name as the
+        source directory, with a .pyz extension.
+
+        The following options can be specified:
+
+        -o archive / --output archive
+
+            The destination archive will have the specified name.
+
+        -p interpreter / --python interpreter
+
+            The given interpreter will be written to the shebang line
+            of the archive. If this option is not given, the archive
+            will have no shebang line.
+
+        -m pkg.mod:fn / --main pkg.mod:fn
+
+            The source directory must not have a __main__.py file. The
+            archiver will write a __main__.py file into the target
+            which calls fn from the module pkg.mod.
+
+The behaviour of the command line interface matches that of
+``zipapp.pack()``.
+
+As noted, the archives are standard zip files, and so can be unpacked
+using any standard ZIP utility or Python's zipfile module.
 
 FAQ
 ---
 
-Q. Are you sure a standard ZIP utility can handle #! at the beginning?
+Are you sure a standard ZIP utility can handle ``#!`` at the beginning?
+    Absolutely.  The zipfile specification allows for arbitrary data to
+    be prepended to a zipfile.  This feature is commonly used by
+    "self-extracting zip" programs.  If your archive program can't
+    handle this, it is a bug in your archive program.
 
-A. Absolutely. If it doesn't, it is a bug in your archive program.
+Isn't zipapp just a very thin wrapper over the zipfile module?
+    Yes.  If you prefer to build your own Python zip application
+    archives using other tools, they will work just as well.  The
+    zipapp module is a convenience, nothing more.
 
-Q. Isn’t pyzaa just a very thin wrapper over zipfile and compileall?
+Why not use just use a ``.zip`` or ``.py`` extension?
+    Users expect a ``.zip`` file to be opened with an archive tool, and
+    expect a ``.py`` file to contain readable text.  Both would be
+    confusing for this use case.
 
-A. Yes.
+How does this compete with existing package formats?
+    The sdist, bdist and wheel formats are designed for packaging of
+    modules to be installed into an existing Python installation.
+    They are not intended to be used without installing.  The
+    executable zip format is specifically designed for standalone use,
+    without needing to be installed.  They are in effect a multi-file
+    version of a standalone Python script.
 
-Q. How does this compete with existing sdist/bdist formats?
+Rejected Proposals
+==================
 
-A. There is some overlap, but .pyz files are especially interesting
-as a way to distribute an installer.  They may also prove useful as a
-way to deliver applications when users shouldn’t be asked to perform
-virtualenv + “pip install”.
+Convenience Values for Shebang Lines
+------------------------------------
+
+Is it worth having "convenience" forms for any of the common
+interpreter values? For example, ``-p 3`` meaning the same as ``-p
+"/usr/bin/env python3"``.  It would save a lot of typing for the
+common cases, as well as giving cross-platform options for people who
+don't want or need to understand the intricacies of shebang handling
+on "other" platforms.
+
+Downsides are that it's not obvious how to translate the
+abbreviations.  For example, should "3" mean "/usr/bin/env python3",
+"/usr/bin/python3", "python3", or something else?  Also, there is no
+obvious short form for the key case of "/usr/bin/env python" (any
+available version of Python), which could easily result in scripts
+being written with overly-restrictive shebang lines.
+
+Overall, this seems like there are more problems than benefits, and as
+a result has been dropped from consideration.
+
+Registering ``.pyz`` as a Media Type
+--------------------------------
+
+It was suggested [3]_ that the ``.pyz`` extension should be registered
+in the Unix database of extensions.  While it makes sense to do this
+as an equivalent of the Windows installer registering the extension,
+the ``.py`` extension is not listed in the media types database [4]_.
+It doesn't seem reasonable to register ``.pyz`` without ``.py``, so
+this idea has been omitted from this PEP.  An interested party could
+arrange for *both* ``.py`` and ``.pyz`` to be registered at a future
+date.
+
+Default Interpreter
+-------------------
+
+The initial draft of this PEP proposed using ``/usr/bin/env python``
+as the default interpreter.  Unix users have problems with this
+behaviour, as the default for the python command on many distributions
+is Python 2, and it is felt that this PEP should prefer Python 3 by
+default.  However, using a command of ``python3`` can result in
+unexpected behaviour for Windows users, where the default behaviour of
+the launcher for the command ``python`` is commonly customised by users,
+but the behaviour of ``python3`` may not be modified to match.
+
+As a result, the principle "in the face of ambiguity, refuse to guess"
+has been invoked, and archives have no shebang line unless explicitly
+requested.  On Windows, the archives will still be run (with the
+default Python) by the launcher, and on Unix, the archives can be run
+by explicitly invoking the desired Python interpreter.
+
+Command Line Tool to Manage Shebang Lines
+-----------------------------------------
+
+It is conceivable that users would want to modify the shebang line for
+an existing archive, or even just display the current shebang line.
+This is tricky to do so with existing tools (zip programs typically
+ignore prepended data totally, and text editors can have trouble
+editing files containing binary data).
+
+The zipapp module provides functions to handle the shebang line, but
+does not include a command line interface to that functionality.  This
+is because it is not clear how to provide one without the resulting
+interface being over-complex and potentially confusing.  Changing the
+shebang line is expected to be an uncommon requirement.
+
+Reference Implementation
+========================
+
+A reference implementation is at http://bugs.python.org/issue23491.
 
 References
 ==========
 
-.. [1]  “Allow interpreter to execute a zip file”
+.. [1] Allow interpreter to execute a zip file
    (http://bugs.python.org/issue1739468)
 
-.. [2] “Feature is not documented”
+.. [2] Feature is not documented
    (http://bugs.python.org/issue17359)
 
+.. [3] Discussion of adding a .pyz mime type on python-dev
+   (https://mail.python.org/pipermail/python-dev/2015-February/138338.html)
+
+.. [4] Register of media types
+   (http://www.iana.org/assignments/media-types/media-types.xhtml)
+
+.. [5] pyzzer - A tool for creating Python-executable archives
+   (https://pypi.python.org/pypi/pyzzer)
+
+.. [6] pex - The PEX packaging toolchain
+   (https://pypi.python.org/pypi/pex)
+
+The discussion of this PEP took place on the python-dev mailing list,
+in the thread starting at
+https://mail.python.org/pipermail/python-dev/2015-February/138277.html
+
 Copyright
 =========
 

-- 
Repository URL: https://hg.python.org/peps

More information about the Python-checkins mailing list