[Python-checkins] peps: PEP 406 (ImportEngine API): defer this until 3.4 at the earliest, don't propose
nick.coghlan
python-checkins at python.org
Sun Dec 4 03:05:11 CET 2011
http://hg.python.org/peps/rev/e2ccffa4d3b5
changeset: 4009:e2ccffa4d3b5
user: Nick Coghlan <ncoghlan at gmail.com>
date: Sun Dec 04 12:05:01 2011 +1000
summary:
PEP 406 (ImportEngine API): defer this until 3.4 at the earliest, don't propose altering the PEP 302 APIs
files:
pep-0406.txt | 127 +++++++++++++++++++++++---------------
1 files changed, 77 insertions(+), 50 deletions(-)
diff --git a/pep-0406.txt b/pep-0406.txt
--- a/pep-0406.txt
+++ b/pep-0406.txt
@@ -3,11 +3,12 @@
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan at gmail.com>, Greg Slodkowicz <jergosh at gmail.com>
-Status: Draft
+Status: Deferred
Type: Standards Track
Content-Type: text/x-rst
Created: 4-Jul-2011
-Post-History: 31-Jul-2011, 13-Nov-2011
+Python-Version: 3.4
+Post-History: 31-Jul-2011, 13-Nov-2011, 4-Dec-2011
Abstract
========
@@ -18,17 +19,27 @@
an alternative to completely replacing the built-in implementation of the
import statement, by overriding the ``__import__()`` function. To work with
the builtin import functionality and importing via import engine objects,
-module importers and loaders must accept an optional ``engine`` parameter. In
-that sense, this PEP constitutes a revision of finder and loader interfaces
-described in PEP 302 [1]_. However, the standard import process will not
-supply the additional argument, so this proposal remains fully backwards
-compatible.
+this PEP proposes a context management based approach to temporarily replacing
+the global import state.
The PEP also proposes inclusion of a ``GlobalImportEngine`` subclass and a
globally accessible instance of that class, which "writes through" to the
-process global state and invokes importers and loaders without the additional
-``engine`` argument. This provides a backwards compatible bridge between the
-proposed encapsulated API and the legacy process global state.
+process global state. This provides a backwards compatible bridge between the
+proposed encapsulated API and the legacy process global state, and allows
+straightforward support for related state updates (e.g. selectively
+invalidating path cache entries when ``sys.path`` is modified).
+
+
+PEP Deferral
+============
+
+The import system is already seeing substantial changes in Python 3.3, to
+natively handle packages split across multiple directories (PEP 382) and
+(potentially) to make the import semantics in the main module better match
+those in other modules (PEP 395).
+
+Accordingly, the proposal in this PEP will not be seriously considered until
+Python 3.4 at the earliest.
Rationale
@@ -58,6 +69,10 @@
*additional* process global state, in order to correctly update package paths
as ``sys.path`` is modified.
+Finally, providing a coherent object for all this state makes it feasible to
+also provide context management features that allow the import state to be
+temporarily substituted.
+
Proposal
========
@@ -68,11 +83,10 @@
desired and also an ``import_module()`` method, equivalent to
``importlib.import_module()`` [3]_.
-Since the new style finders and loaders should also have the option to
-modify the global import state, we introduce a ``GlobalImportState``
-class with an interface identical to ``ImportEngine`` but taking
-advantage of the global state. This can be easily implemented using
-class properties.
+Since there are global import state invariants that are assumed and should be
+maintained, we introduce a ``GlobalImportState`` class with an interface
+identical to ``ImportEngine`` but directly accessing the current global import
+state. This can be easily implemented using class properties.
Specification
@@ -121,6 +135,14 @@
methods like ``ImportEngine`` but writes through to the global state
in ``sys``.
+To support various namespace package mechanisms, when ``sys.path`` is altered,
+tools like ``pkgutil.extend_path`` should be used to also modify other parts
+of the import state (in this case, package ``__path__`` attributes). The path
+importer cache should also be invalidated when a variety of changes are made.
+
+The ``ImportEngine`` API will provide convenience methods that automatically
+make related import state updates as part of a single operation.
+
Global variables
~~~~~~~~~~~~~~~~
@@ -133,24 +155,26 @@
a copy of the process global import state.
-Necessary changes to finder/loader interfaces:
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+No changes to finder/loader interfaces
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-``find_module (cls, fullname, path=None, engine=None)``
+Rather than attempting to update the PEP 302 APIs to accept additional state,
+this PEP proposes that ``ImportEngine`` support the content management
+protocol (similar to the context substitution mechanisms in the ``decimal``
+module).
-``load_module (cls, fullname, path=None, engine=None)``
+The context management mechanism for ``ImportEngine`` would:
-The only difference between engine compatible and PEP 302 compatible
-finders/loaders is the presence of an additional ``engine`` parameter.
-This is intended to specify an ImportEngine instance or subclass thereof.
-This parameter is optional so that engine compatible finders and
-loaders can be made backwards compatible with PEP 302 calling conventions by
-falling back on ``engine.sysengine`` with the following simple pattern::
+* On entry:
+ * Acquire the import lock
+ * Substitute the global import state with the import engine's own state
+* On exit:
+ * Restore the previous global import state
+ * Release the import lock
- def find_module(cls, fullname, path=None, engine=None):
- if not engine:
- engine = importlib.engine.sysengine
- ...
+The precise API for this is TBD (but will probably use a distinct context
+management object, along the lines of that created by
+``decimal.localcontext``).
Open Issues
@@ -185,35 +209,38 @@
cache - it's only loading them directly which causes problems.
-Nested imports
-~~~~~~~~~~~~~~
+Scope of substitution
+~~~~~~~~~~~~~~~~~~~~~
-The reference implementation currently applies only to the outermost import.
-Any imports by the module being imported will be handled using the standard
-import machinery.
+Related to the previous open issue is the question of what state to substitute
+when using the context management API. It is currently the case that replacing
+``sys.modules`` can be unreliable due to cached references and there's the
+underlying fact that having independent copies of some modules is simply
+impossible due to platform limitations.
-One way to handle this is to place the burden on the implementation of module
-loaders to set ``module.__dict__["__import__"] = engine.__import__`` before
-running the module's code. The ``importlib`` design facilities this by
-allowing the change to be made in one place (``_LoaderBasics._load_module``).
+As part of this PEP, it will be necessary to document explicitly:
-
-Scope of API updates
-~~~~~~~~~~~~~~~~~~~~
-
-The reference implementation focuses on finding and loading modules. There
-may be other PEP 302 APIs that should also be updated to accept an optional
-``engine`` parameter.
+* Which parts of the global import state can be substituted (and declare code
+ which caches references to that state without dealing with the substitution
+ case buggy)
+* Which parts must be modified in-place (and hence are not substituted by the
+ ``ImportEngine`` context management API, or otherwise scoped to
+ ``ImportEngine`` instances)
Reference Implementation
========================
-A reference implementation [4]_ based on Brett Cannon's importlib has been
-developed by Greg Slodkowicz as part of the 2011 Google Summer of Code. Note
-that the current implementation avoids modifying existing code, and hence
-duplicates a lot of things unnecessarily. An actual implementation would just
-modify any such affected code in place.
+A reference implementation [4]_ for an earlier draft of this PEP, based on
+Brett Cannon's importlib has been developed by Greg Slodkowicz as part of the
+2011 Google Summer of Code. Note that the current implementation avoids
+modifying existing code, and hence duplicates a lot of things unnecessarily.
+An actual implementation would just modify any such affected code in place.
+
+That earlier draft of the PEP proposed change the PEP 302 APIs to support passing
+in an optional engine instance. This had the (serious) downside of not correctly
+affecting further imports from the imported module, hence the change to the
+context management based proposal for substituting the global state.
References
--
Repository URL: http://hg.python.org/peps
More information about the Python-checkins
mailing list