[Python-checkins] r55045 - peps/trunk/pep-0366.txt

[nick.coghlan]

Author: nick.coghlan
Date: Tue May  1 12:59:12 2007
New Revision: 55045

Added:
   peps/trunk/pep-0366.txt   (contents, props changed)
Log:
Add PEP 366 proposing a low cost fix to the PEP 328/338 incompatibility

Added: peps/trunk/pep-0366.txt
==============================================================================
--- (empty file)
+++ peps/trunk/pep-0366.txt	Tue May  1 12:59:12 2007
@@ -0,0 +1,187 @@
+PEP: 338
+Title: Executing modules as scripts
+Version: $Revision:$
+Last-Modified: $Date:$
+Author: Nick Coghlan <ncoghlan at gmail.com>
+Status: Final
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 1-May-2007
+Python-Version: 2.6
+Post-History: 1-May-2007
+
+
+Abstract
+========
+
+This PEP proposes a backwards compatible mechanism that permits
+the use of explicit relative imports from executable modules within
+packages. Such imports currently fail due to an awkward interaction
+between PEP 328 and PEP 338 - this behaviour is the subject of at
+least one open SF bug report (#1510172)[1].
+
+With the proposed mechanism, relative imports will work automatically
+if the module is executed using the ``-m`` switch. A small amount of
+boilerplate will be needed in the module itself to allow the relative
+imports to work when the file is executed by name.
+
+
+Import Statements and the Main Module
+=====================================
+
+(This section is taken from the final revision of PEP 338)
+
+The release of 2.5b1 showed a surprising  (although obvious in
+retrospect) interaction between PEP 338 and PEP 328 - explicit
+relative imports don't work from a main module. This is due to
+the fact that relative imports rely on ``__name__`` to determine
+the current module's position in the package hierarchy. In a main
+module, the value of ``__name__`` is always ``'__main__'``, so
+explicit relative imports will always fail (as they only work for
+a module inside a package).
+
+Investigation into why implicit relative imports *appear* to work when
+a main module is executed directly but fail when executed using ``-m``
+showed that such imports are actually always treated as absolute
+imports. Because of the way direct execution works, the package
+containing the executed module is added to sys.path, so its sibling
+modules are actually imported as top level modules. This can easily
+lead to multiple copies of the sibling modules in the application if
+implicit relative imports are used in modules that may be directly
+executed (e.g. test modules or utility scripts).
+
+For the 2.5 release, the recommendation is to always use absolute
+imports in any module that is intended to be used as a main module.
+The ``-m`` switch already provides a benefit here, as it inserts the
+current directory into ``sys.path``, instead of the directory contain the
+main module. This means that it is possible to run a module from
+inside a package using ``-m`` so long as the current directory contains
+the top level directory for the package. Absolute imports will work
+correctly even if the package isn't installed anywhere else on
+sys.path. If the module is executed directly and uses absolute imports
+to retrieve its sibling modules, then the top level package directory
+needs to be installed somewhere on sys.path (since the current directory
+won't be added automatically).
+
+Here's an example file layout::
+
+    devel/
+        pkg/
+            __init__.py
+            moduleA.py
+            moduleB.py
+            test/
+                __init__.py
+                test_A.py
+                test_B.py
+
+So long as the current directory is ``devel``, or ``devel`` is already
+on ``sys.path`` and the test modules use absolute imports (such as
+``import pkg.moduleA`` to retrieve the module under test, PEP 338
+allows the tests to be run as::
+
+    python -m pkg.test.test_A
+    python -m pkg.test.test_B
+
+Rationale for Change
+====================
+
+In rejecting PEP 3122 (which proposed a higher impact solution to this
+problem), Guido has indicated that he still isn't particularly keen on
+the idea of executing modules inside packages as scripts [2]. Despite
+these misgivings he has previously approved the addition of the ``-m``
+switch in Python 2.4, and the ``runpy`` module based enhancements
+described in PEP 338 for Python 2.5.
+
+The philosophy that motivated those previous additions (i.e. access to
+utility or testing scripts without needing to worry about name clashes in
+either the OS executable namespace or the top level Python namespace) is
+also the motivation behind fixing what I see as a bug in the current
+implementation.
+
+This PEP is intended to provide a solution which permits explicit
+relative imports from main modules, without incurring any significant
+costs during interpreter startup or normal module import.
+
+
+Proposed Solution
+=================
+
+The heart of the proposed solution is a new module attribute
+``__package_name__``. This attribute will be defined only in
+the main module (i.e. modules where ``__name__ == "__main__"``).
+
+For a directly executed main module, this attribute will be set
+to the empty string. For a module executed using
+``runpy.run_module()`` with the ``run_name`` parameter set to
+``"__main__"``, the attribute will be set to
+``mod_name.rpartition('.')[0]`` (i.e., everything up to
+but not including the last period).
+
+In the import machinery there is an error handling path which
+deals with the case where an explicit relative reference attempts
+to go higher than the top level in the package hierarchy. This
+error path would be changed to fall back on the ``__package_name__``
+attribute for explicit relative imports when the importing module
+is called ``"__main__"``.
+
+With this change, explicit relative imports will work automatically
+from a script executed with the ``-m`` switch. To allow direct
+execution of the module, the following boilerplate would be needed at
+the top of the script::
+
+  if __name__ == "__main__" and not __package_name__:
+      __package_name__ = "<expected_pkg_name>"
+
+Note that this boilerplate has the same disadvantage as the use of
+absolute imports of sibling modules - if the script is moved to a
+different package or subpackage, the boilerplate will need to be
+updated manually.
+
+With this feature in place, the test scripts in the package above
+would be able to change their import lines to something along the
+lines of ``import ..moduleA``. The scripts could then be
+executed unmodified even if the name of the package was changed.
+
+(Rev 47142 in SVN implemented an early variant of this proposal
+which stored the main module's real module name in the
+'__module_name__' attribute. It was reverted due to the fact
+that 2.5 was already in beta by that time.)
+
+
+Alternative Proposals
+=====================
+
+PEP 3122 proposed addressing this problem by changing the way
+the main module is identified. That's a huge compatibility cost
+to incur to fix something that is a pretty minor bug in the overall
+scheme of things.
+
+The advantage of the proposal in this PEP is that its only impact on
+normal code is the tiny amount of time needed at startup to set the extra
+attribute in the main module. The changes to the import machinery are all
+in an existing error handling path, so normal imports don't incur any
+performance penalty at all.
+
+
+References
+==========
+
+.. [1] Absolute/relative import not working?
+   (http://www.python.org/sf/1510172)
+
+.. [2] Guido's rejection of PEP 3122
+   (http://mail.python.org/pipermail/python-3000/2007-April/006793.html)
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   End: