[Python-checkins] peps (merge default -> default): merged
martin.v.loewis
python-checkins at python.org
Mon Jun 18 16:28:38 CEST 2012
http://hg.python.org/peps/rev/0cde8b31a8a2
changeset: 4464:0cde8b31a8a2
parent: 4463:80ce8e5dd630
parent: 4462:634c46edc47a
user: Martin v. Löwis <martin at v.loewis.de>
date: Mon Jun 18 16:28:24 2012 +0200
summary:
merged
files:
pep-0362.txt | 650 +++++++++++++++++++++++---------------
pep-0398.txt | 4 +-
pep-0405.txt | 18 +-
pep-0420.txt | 2 +-
pep-0421.txt | 2 +-
pep-0422.txt | 354 +++++++++++++++++++++
6 files changed, 762 insertions(+), 268 deletions(-)
diff --git a/pep-0362.txt b/pep-0362.txt
--- a/pep-0362.txt
+++ b/pep-0362.txt
@@ -2,332 +2,468 @@
Title: Function Signature Object
Version: $Revision$
Last-Modified: $Date$
-Author: Brett Cannon <brett at python.org>, Jiwon Seo <seojiwon at gmail.com>
+Author: Brett Cannon <brett at python.org>, Jiwon Seo <seojiwon at gmail.com>,
+ Yury Selivanov <yselivanov at sprymix.com>, Larry Hastings <larry at hastings.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 21-Aug-2006
-Python-Version: 2.6
-Post-History: 05-Sep-2007
+Python-Version: 3.3
+Post-History: 04-Jun-2012
Abstract
========
Python has always supported powerful introspection capabilities,
-including that for functions and methods (for the rest of this PEP the
-word "function" refers to both functions and methods). Taking a
-function object, you can fully reconstruct the function's signature.
-Unfortunately it is a little unruly having to look at all the
-different attributes to pull together complete information for a
-function's signature.
+including introspecting functions and methods (for the rest of
+this PEP, "function" refers to both functions and methods). By
+examining a function object you can fully reconstruct the function's
+signature. Unfortunately this information is stored in an inconvenient
+manner, and is spread across a half-dozen deeply nested attributes.
-This PEP proposes an object representation for function signatures.
-This should help facilitate introspection on functions for various
-uses. The introspection information contains all possible information
-about the parameters in a signature (including Python 3.0 features).
+This PEP proposes a new representation for function signatures.
+The new representation contains all necessary information about a function
+and its parameters, and makes introspection easy and straightforward.
-This object, though, is not meant to replace existing ways of
-introspection on a function's signature. The current solutions are
-there to make Python's execution work in an efficient manner. The
-proposed object representation is only meant to help make application
-code have an easier time to query a function on its signature.
-
-
-Purpose
-=======
-
-An object representation of a function's call signature should provide
-an easy way to introspect what a function expects as arguments. It
-does not need to be a "live" representation, though; the signature can
-be inferred once and stored without changes to the signature object
-representation affecting the function it represents (but this is an
-`Open Issues`_).
-
-Indirection of signature introspection can also occur. If a
-decorator took a decorated function's signature object and set it on
-the decorating function then introspection could be redirected to what
-is actually expected instead of the typical ``*args, **kwargs``
-signature of decorating functions.
+However, this object does not replace the existing function
+metadata, which is used by Python itself to execute those
+functions. The new metadata object is intended solely to make
+function introspection easier for Python programmers.
Signature Object
================
-The overall signature of an object is represented by the Signature
-object. This object is to store a `Parameter object`_ for each
-parameter in the signature. It is also to store any information
-about the function itself that is pertinent to the signature.
+A Signature object represents the call signature of a function and
+its return annotation. For each parameter accepted by the function
+it stores a `Parameter object`_ in its ``parameters`` collection.
-A Signature object has the following structure attributes:
+A Signature object has the following public attributes and methods:
-* name : str
- Name of the function. This is not fully qualified because
- function objects for methods do not know the class they are
- contained within. This makes functions and methods
- indistinguishable from one another when passed to decorators,
- preventing proper creation of a fully qualified name.
-* var_args : str
- Name of the variable positional parameter (i.e., ``*args``), if
- present, or the empty string.
-* var_kw_args : str
- Name of the variable keyword parameter (i.e., ``**kwargs``), if
- present, or the empty string.
-* var_annotations: dict(str, object)
- Dict that contains the annotations for the variable parameters.
- The keys are of the variable parameter with values of the
- annotation. If an annotation does not exist for a variable
- parameter then the key does not exist in the dict.
* return_annotation : object
- If present, the attribute is set to the annotation for the return
- type of the function.
-* parameters : list(Parameter)
- List of the parameters of the function as represented by
- Parameter objects in the order of its definition (keyword-only
- arguments are in the order listed by ``code.co_varnames``).
-* bind(\*args, \*\*kwargs) -> dict(str, object)
- Create a mapping from arguments to parameters. The keys are the
- names of the parameter that an argument maps to with the value
- being the value the parameter would have if this function was
- called with the given arguments.
+ The annotation for the return type of the function if specified.
+ If the function has no annotation for its return type, this
+ attribute is not set.
+* parameters : OrderedDict
+ An ordered mapping of parameters' names to the corresponding
+ Parameter objects (keyword-only arguments are in the same order
+ as listed in ``code.co_varnames``).
+* bind(\*args, \*\*kwargs) -> BoundArguments
+ Creates a mapping from positional and keyword arguments to
+ parameters. Raises a ``TypeError`` if the passed arguments do
+ not match the signature.
+* bind_partial(\*args, \*\*kwargs) -> BoundArguments
+ Works the same way as ``bind()``, but allows the omission
+ of some required arguments (mimics ``functools.partial``
+ behavior.) Raises a ``TypeError`` if the passed arguments do
+ not match the signature.
+* format(...) -> str
+ Formats the Signature object to a string. Optional arguments allow
+ for custom render functions for parameter names,
+ annotations and default values, along with custom separators.
-Signature objects also have the following methods:
+Signature implements the ``__str__`` method, which fallbacks to the
+``Signature.format()`` call.
-* __getitem__(self, key : str) -> Parameter
- Returns the Parameter object for the named parameter.
-* __iter__(self)
- Returns an iterator that returns Parameter objects in their
- sequential order based on their 'position' attribute.
+It's possible to test Signatures for equality. Two signatures
+are equal when they have equal parameters and return annotations.
-The Signature object is stored in the ``__signature__`` attribute of
-a function. When it is to be created is discussed in
-`Open Issues`_.
+Changes to the Signature object, or to any of its data members,
+do not affect the function itself.
Parameter Object
================
-A function's signature is made up of several parameters. Python's
-different kinds of parameters is quite large and rich and continues to
-grow. Parameter objects represent any possible parameter.
-
-Originally the plan was to represent parameters using a list of
-parameter names on the Signature object along with various dicts keyed
-on parameter names to disseminate the various pieces of information
-one can know about a parameter. But the decision was made to
-incorporate all information about a parameter in a single object so
-as to make extending the information easier. This was originally put
-forth by Talin and the preferred form of Guido (as discussed at the
-2006 Google Sprint).
+Python's expressive syntax means functions can accept many different
+kinds of parameters with many subtle semantic differences. We
+propose a rich Parameter object designed to represent any possible
+function parameter.
The structure of the Parameter object is:
-* name : (str | tuple(str))
- The name of the parameter as a string if it is not a tuple. If
- the argument is a tuple then a tuple of strings is used.
-* position : int
- The position of the parameter within the signature of the
- function (zero-indexed). For keyword-only parameters the position
- value is arbitrary while not conflicting with positional
- parameters. The suggestion of setting the attribute to None or -1
- to represent keyword-only parameters was rejected to prevent
- variable type usage and as a possible point of errors,
- respectively.
-* default_value : object
- The default value for the parameter, if present, else the
- attribute does not exist.
-* keyword_only : bool
- True if the parameter is keyword-only, else False.
-* annotation
- Set to the annotation for the parameter. If ``has_annotation`` is
- False then the attribute does not exist to prevent accidental use.
+* name : str
+ The name of the parameter as a string.
+
+* default : object
+ The default value for the parameter, if specified. If the
+ parameter has no default value, this attribute is not set.
+
+* annotation : object
+ The annotation for the parameter if specified. If the
+ parameter has no annotation, this attribute is not set.
+
+* kind : str
+ Describes how argument values are bound to the parameter.
+ Possible values:
+
+ * ``Parameter.POSITIONAL_ONLY`` - value must be supplied
+ as a positional argument.
+
+ Python has no explicit syntax for defining positional-only
+ parameters, but many builtin and extension module functions
+ (especially those that accept only one or two parameters)
+ accept them.
+
+ * ``Parameter.POSITIONAL_OR_KEYWORD`` - value may be
+ supplied as either a keyword or positional argument
+ (this is the standard binding behaviour for functions
+ implemented in Python.)
+
+ * ``Parameter.KEYWORD_ONLY`` - value must be supplied
+ as a keyword argument. Keyword only parameters are those
+ which appear after a "*" or "\*args" entry in a Python
+ function definition.
+
+ * ``Parameter.VAR_POSITIONAL`` - a tuple of positional
+ arguments that aren't bound to any other parameter.
+ This corresponds to a "\*args" parameter in a Python
+ function definition.
+
+ * ``Parameter.VAR_KEYWORD`` - a dict of keyword arguments
+ that aren't bound to any other parameter. This corresponds
+ to a "\*\*kwds" parameter in a Python function definition.
+
+* implemented : bool
+ True if the parameter is implemented for use. Some platforms
+ implement functions but can't support specific parameters
+ (e.g. "mode" for ``os.mkdir``). Passing in an unimplemented
+ parameter may result in the parameter being ignored,
+ or in NotImplementedError being raised. It is intended that
+ all conditions where ``implemented`` may be False be
+ thoroughly documented.
+
+Two parameters are equal when all their attributes are equal.
+
+
+BoundArguments Object
+=====================
+
+Result of a ``Signature.bind`` call. Holds the mapping of arguments
+to the function's parameters.
+
+Has the following public attributes:
+
+* arguments : OrderedDict
+ An ordered, mutable mapping of parameters' names to arguments' values.
+ Does not contain arguments' default values.
+* args : tuple
+ Tuple of positional arguments values. Dynamically computed from
+ the 'arguments' attribute.
+* kwargs : dict
+ Dict of keyword arguments values. Dynamically computed from
+ the 'arguments' attribute.
+
+The ``arguments`` attribute should be used in conjunction with
+``Signature.parameters`` for any arguments processing purposes.
+
+``args`` and ``kwargs`` properties can be used to invoke functions:
+::
+
+ def test(a, *, b):
+ ...
+
+ sig = signature(test)
+ ba = sig.bind(10, b=20)
+ test(*ba.args, **ba.kwargs)
Implementation
==============
-An implementation can be found in Python's sandbox [#impl]_.
-There is a function named ``signature()`` which
-returns the value stored on the ``__signature__`` attribute if it
-exists, else it creates the Signature object for the
-function and sets ``__signature__``. For methods this is stored
-directly on the im_func function object since that is what decorators
-work with.
+The implementation adds a new function ``signature()`` to the ``inspect``
+module. The function is the preferred way of getting a ``Signature`` for
+a callable object.
+
+The function implements the following algorithm:
+
+ - If the object is not callable - raise a TypeError
+
+ - If the object has a ``__signature__`` attribute and if it
+ is not ``None`` - return a deepcopy of it
+
+ - If it is ``None`` and the object is an instance of
+ ``BuiltinFunction``, raise a ``ValueError``
+
+ - If it has a ``__wrapped__`` attribute, return
+ ``signature(object.__wrapped__)``
+
+ - If the object is a an instance of ``FunctionType`` construct
+ and return a new ``Signature`` for it
+
+ - If the object is a method or a classmethod, construct and return
+ a new ``Signature`` object, with its first parameter (usually
+ ``self`` or ``cls``) removed
+
+ - If the object is a staticmethod, construct and return
+ a new ``Signature`` object
+
+ - If the object is an instance of ``functools.partial``, construct
+ a new ``Signature`` from its ``partial.func`` attribute, and
+ account for already bound ``partial.args`` and ``partial.kwargs``
+
+ - If the object is a class or metaclass:
+
+ - If the object's type has a ``__call__`` method defined in
+ its MRO, return a Signature for it
+
+ - If the object has a ``__new__`` method defined in its class,
+ return a Signature object for it
+
+ - If the object has a ``__init__`` method defined in its class,
+ return a Signature object for it
+
+ - Return ``signature(object.__call__)``
+
+Note, that the ``Signature`` object is created in a lazy manner, and
+is not automatically cached. If, however, the Signature object was
+explicitly cached by the user, ``signature()`` returns a new deepcopy
+of it on each invocation.
+
+An implementation for Python 3.3 can be found at [#impl]_.
+The python issue tracking the patch is [#issue]_.
+
+
+Design Considerations
+=====================
+
+No implicit caching of Signature objects
+----------------------------------------
+
+The first PEP design had a provision for implicit caching of ``Signature``
+objects in the ``inspect.signature()`` function. However, this has the
+following downsides:
+
+ * If the ``Signature`` object is cached then any changes to the function
+ it describes will not be reflected in it. However, If the caching is
+ needed, it can be always done manually and explicitly
+
+ * It is better to reserve the ``__signature__`` attribute for the cases
+ when there is a need to explicitly set to a ``Signature`` object that
+ is different from the actual one
Examples
========
+Visualizing Callable Objects' Signature
+---------------------------------------
+
+Let's define some classes and functions:
+
+::
+
+ from inspect import signature
+ from functools import partial, wraps
+
+
+ class FooMeta(type):
+ def __new__(mcls, name, bases, dct, *, bar:bool=False):
+ return super().__new__(mcls, name, bases, dct)
+
+ def __init__(cls, name, bases, dct, **kwargs):
+ return super().__init__(name, bases, dct)
+
+
+ class Foo(metaclass=FooMeta):
+ def __init__(self, spam:int=42):
+ self.spam = spam
+
+ def __call__(self, a, b, *, c) -> tuple:
+ return a, b, c
+
+
+ def shared_vars(*shared_args):
+ """Decorator factory that defines shared variables that are
+ passed to every invocation of the function"""
+
+ def decorator(f):
+ @wraps(f)
+ def wrapper(*args, **kwds):
+ full_args = shared_args + args
+ return f(*full_args, **kwds)
+ # Override signature
+ sig = wrapper.__signature__ = signature(f)
+ for __ in shared_args:
+ sig.parameters.popitem(last=False)
+ return wrapper
+ return decorator
+
+
+ @shared_vars({})
+ def example(_state, a, b, c):
+ return _state, a, b, c
+
+
+ def format_signature(obj):
+ return str(signature(obj))
+
+
+Now, in the python REPL:
+
+::
+
+ >>> format_signature(FooMeta)
+ '(name, bases, dct, *, bar:bool=False)'
+
+ >>> format_signature(Foo)
+ '(spam:int=42)'
+
+ >>> format_signature(Foo.__call__)
+ '(self, a, b, *, c) -> tuple'
+
+ >>> format_signature(Foo().__call__)
+ '(a, b, *, c) -> tuple'
+
+ >>> format_signature(partial(Foo().__call__, 1, c=3))
+ '(b, *, c=3) -> tuple'
+
+ >>> format_signature(partial(partial(Foo().__call__, 1, c=3), 2, c=20))
+ '(*, c=20) -> tuple'
+
+ >>> format_signature(example)
+ '(a, b, c)'
+
+ >>> format_signature(partial(example, 1, 2))
+ '(c)'
+
+ >>> format_signature(partial(partial(example, 1, b=2), c=3))
+ '(b=2, c=3)'
+
+
Annotation Checker
------------------
::
- def quack_check(fxn):
- """Decorator to verify arguments and return value quack as they should.
+ import inspect
+ import functools
- Positional arguments.
- >>> @quack_check
- ... def one_arg(x:int): pass
- ...
- >>> one_arg(42)
- >>> one_arg('a')
- Traceback (most recent call last):
- ...
- TypeError: 'a' does not quack like a <type 'int'>
+ def checktypes(func):
+ '''Decorator to verify arguments and return types
+ Example:
- *args
- >>> @quack_check
- ... def var_args(*args:int): pass
- ...
- >>> var_args(*[1,2,3])
- >>> var_args(*[1,'b',3])
- Traceback (most recent call last):
- ...
- TypeError: *args contains a a value that does not quack like a <type 'int'>
+ >>> @checktypes
+ ... def test(a:int, b:str) -> int:
+ ... return int(a * b)
- **kwargs
- >>> @quack_check
- ... def var_kw_args(**kwargs:int): pass
- ...
- >>> var_kw_args(**{'a': 1})
- >>> var_kw_args(**{'a': 'A'})
- Traceback (most recent call last):
- ...
- TypeError: **kwargs contains a value that does not quack like a <type 'int'>
+ >>> test(10, '1')
+ 1111111111
- Return annotations.
- >>> @quack_check
- ... def returned(x) -> int: return x
- ...
- >>> returned(42)
- 42
- >>> returned('a')
- Traceback (most recent call last):
- ...
- TypeError: the return value 'a' does not quack like a <type 'int'>
+ >>> test(10, 1)
+ Traceback (most recent call last):
+ ...
+ ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int'
+ '''
- """
- # Get the signature; only needs to be calculated once.
- sig = Signature(fxn)
- def check(*args, **kwargs):
- # Find out the variable -> value bindings.
- bindings = sig.bind(*args, **kwargs)
- # Check *args for the proper quack.
+ sig = inspect.signature(func)
+
+ types = {}
+ for param in sig.parameters.values():
+ # Iterate through function's parameters and build the list of
+ # arguments types
try:
- duck = sig.var_annotations[sig.var_args]
- except KeyError:
+ type_ = param.annotation
+ except AttributeError:
+ continue
+ else:
+ if not inspect.isclass(type_):
+ # Not a type, skip it
+ continue
+
+ types[param.name] = type_
+
+ # If the argument has a type specified, let's check that its
+ # default value (if present) conforms with the type.
+ try:
+ default = param.default
+ except AttributeError:
+ continue
+ else:
+ if not isinstance(default, type_):
+ raise ValueError("{func}: wrong type of a default value for {arg!r}". \
+ format(func=func.__qualname__, arg=param.name))
+
+ def check_type(sig, arg_name, arg_type, arg_value):
+ # Internal function that encapsulates arguments type checking
+ if not isinstance(arg_value, arg_type):
+ raise ValueError("{func}: wrong type of {arg!r} argument, " \
+ "{exp!r} expected, got {got!r}". \
+ format(func=func.__qualname__, arg=arg_name,
+ exp=arg_type.__name__, got=type(arg_value).__name__))
+
+ @functools.wraps(func)
+ def wrapper(*args, **kwargs):
+ # Let's bind the arguments
+ ba = sig.bind(*args, **kwargs)
+ for arg_name, arg in ba.arguments.items():
+ # And iterate through the bound arguments
+ try:
+ type_ = types[arg_name]
+ except KeyError:
+ continue
+ else:
+ # OK, we have a type for the argument, lets get the corresponding
+ # parameter description from the signature object
+ param = sig.parameters[arg_name]
+ if param.kind == param.VAR_POSITIONAL:
+ # If this parameter is a variable-argument parameter,
+ # then we need to check each of its values
+ for value in arg:
+ check_type(sig, arg_name, type_, value)
+ elif param.kind == param.VAR_KEYWORD:
+ # If this parameter is a variable-keyword-argument parameter:
+ for subname, value in arg.items():
+ check_type(sig, arg_name + ':' + subname, type_, value)
+ else:
+ # And, finally, if this parameter a regular one:
+ check_type(sig, arg_name, type_, arg)
+
+ result = func(*ba.args, **ba.kwargs)
+ # The last bit - let's check that the result is correct
+ try:
+ return_type = sig.return_annotation
+ except AttributeError:
+ # Looks like we don't have any restriction on the return type
pass
else:
- # Check every value in *args.
- for value in bindings[sig.var_args]:
- if not isinstance(value, duck):
- raise TypeError("*%s contains a a value that does not "
- "quack like a %r" %
- (sig.var_args, duck))
- # Remove it from the bindings so as to not check it again.
- del bindings[sig.var_args]
- # **kwargs.
- try:
- duck = sig.var_annotations[sig.var_kw_args]
- except (KeyError, AttributeError):
- pass
- else:
- # Check every value in **kwargs.
- for value in bindings[sig.var_kw_args].values():
- if not isinstance(value, duck):
- raise TypeError("**%s contains a value that does not "
- "quack like a %r" %
- (sig.var_kw_args, duck))
- # Remove from bindings so as to not check again.
- del bindings[sig.var_kw_args]
- # For each remaining variable ...
- for var, value in bindings.items():
- # See if an annotation was set.
- try:
- duck = sig[var].annotation
- except AttributeError:
- continue
- # Check that the value quacks like it should.
- if not isinstance(value, duck):
- raise TypeError('%r does not quack like a %s' % (value, duck))
- else:
- # All the ducks quack fine; let the call proceed.
- returned = fxn(*args, **kwargs)
- # Check the return value.
- try:
- if not isinstance(returned, sig.return_annotation):
- raise TypeError('the return value %r does not quack like '
- 'a %r' % (returned,
- sig.return_annotation))
- except AttributeError:
- pass
- return returned
- # Full-featured version would set function metadata.
- return check
+ if isinstance(return_type, type) and not isinstance(result, return_type):
+ raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \
+ format(func=func.__qualname__, exp=return_type.__name__,
+ got=type(result).__name__))
+ return result
+ return wrapper
-Open Issues
-===========
-When to construct the Signature object?
----------------------------------------
+Render Function Signature to HTML
+---------------------------------
-The Signature object can either be created in an eager or lazy
-fashion. In the eager situation, the object can be created during
-creation of the function object. In the lazy situation, one would
-pass a function object to a function and that would generate the
-Signature object and store it to ``__signature__`` if
-needed, and then return the value of ``__signature__``.
+::
+ import inspect
-Should ``Signature.bind`` return Parameter objects as keys?
------------------------------------------------------------
+ def format_to_html(func):
+ sig = inspect.signature(func)
-Instead of returning a dict with keys consisting of the name of the
-parameters, would it be more useful to instead use Parameter
-objects? The name of the argument can easily be retrieved from the
-key (and the name would be used as the hash for a Parameter object).
+ html = sig.format(token_params_separator='<span class="t-comma">,</span>',
+ token_colon='<span class="t-colon">:</span>',
+ token_eq='<span class="t-eq">=</span>',
+ token_return_annotation='<span class="t-ra">-></span>',
+ token_left_paren='<span class="t-lp">(</span>',
+ token_right_paren='<span class="t-lp">)</span>',
+ token_kwonly_separator='<span class="t-ast">*</span>',
+ format_name=lambda name: '<span class="name">'+name+'</span>')
-
-Have ``var_args`` and ``_var_kw_args`` default to ``None``?
-------------------------------------------------------------
-
-It has been suggested by Fred Drake that these two attributes have a
-value of ``None`` instead of empty strings when they do not exist.
-The answer to this question will influence what the defaults are for
-other attributes as well.
-
-
-Deprecate ``inspect.getargspec()`` and ``.formatargspec()``?
--------------------------------------------------------------
-
-Since the Signature object replicates the use of ``getargspec()``
-from the ``inspect`` module it might make sense to deprecate it in
-2.6. ``formatargspec()`` could also go if Signature objects gained a
-__str__ representation.
-
-Issue with that is types such as ``int``, when used as annotations,
-do not lend themselves for output (e.g., ``"<type 'int'>"`` is the
-string represenation for ``int``). The repr representation of types
-would need to change in order to make this reasonable.
-
-
-Have the objects be "live"?
----------------------------
-
-Jim Jewett pointed out that Signature and Parameter objects could be
-"live". That would mean requesting information would be done on the
-fly instead of caching it on the objects. It would also allow for
-mutating the function if the Signature or Parameter objects were
-mutated.
+ return '<span class="py-func">{}</span>'.format(html)
References
==========
-.. [#impl] pep362 directory in Python's sandbox
- (http://svn.python.org/view/sandbox/trunk/pep362/)
+.. [#impl] pep362 branch (https://bitbucket.org/1st1/cpython/overview)
+.. [#issue] issue 15008 (http://bugs.python.org/issue15008)
Copyright
@@ -335,7 +471,6 @@
This document has been placed in the public domain.
-
..
Local Variables:
diff --git a/pep-0398.txt b/pep-0398.txt
--- a/pep-0398.txt
+++ b/pep-0398.txt
@@ -70,6 +70,7 @@
* PEP 417: Including mock in the Standard Library
* PEP 418: Add monotonic time, performance counter, and process time functions
* PEP 420: Implicit Namespace Packages
+* PEP 421: Adding sys.implementation
* PEP 3118: Revising the buffer protocol (protocol semantics finalised)
* PEP 3144: IP Address manipulation library
* PEP 3151: Reworking the OS and IO exception hierarchy
@@ -87,8 +88,6 @@
* PEP 362: Function Signature Object
* PEP 397: Python launcher for Windows
-* PEP 421: Adding sys.implementation
-* PEP 3143: Standard daemon process library
* PEP 3154: Pickle protocol version 4
(Note that these are not accepted yet and even if they are, they might
@@ -105,6 +104,7 @@
Deferred to post-3.3:
* PEP 395: Qualified Names for Modules
+* PEP 3143: Standard daemon process library
* Breaking out standard library and docs in separate repos
Copyright
diff --git a/pep-0405.txt b/pep-0405.txt
--- a/pep-0405.txt
+++ b/pep-0405.txt
@@ -4,7 +4,7 @@
Last-Modified: $Date$
Author: Carl Meyer <carl at oddbird.net>
BDFL-Delegate: Nick Coghlan
-Status: Accepted
+Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 13-Jun-2011
@@ -285,15 +285,15 @@
Current virtualenv handles include files in this way:
-On POSIX systems where the installed Python's include files are found
-in ``${base_prefix}/include/pythonX.X``, virtualenv creates
-``${venv}/include/`` and symlink ``${base_prefix}/include/pythonX.X``
+On POSIX systems where the installed Python's include files are found in
+``${base_prefix}/include/pythonX.X``, virtualenv creates
+``${venv}/include/`` and symlinks ``${base_prefix}/include/pythonX.X``
to ``${venv}/include/pythonX.X``. On Windows, where Python's include
files are found in ``{{ sys.prefix }}/Include`` and symlinks are not
reliably available, virtualenv copies ``{{ sys.prefix }}/Include`` to
``${venv}/Include``. This ensures that extension modules built and
-installed within the virtualenv will always find the Python header
-files they need in the expected location relative to ``sys.prefix``.
+installed within the virtualenv will always find the Python header files
+they need in the expected location relative to ``sys.prefix``.
This solution is not ideal when an extension module installs its own
header files, as the default installation location for those header
@@ -467,10 +467,10 @@
site-packages directories.
The most notable case is probably `setuptools`_ and its fork
-`distribute`_, which mostly use ``distutils``and ``sysconfig`` APIs,
+`distribute`_, which mostly use ``distutils`` and ``sysconfig`` APIs,
but do use ``sys.prefix`` directly to build up a list of site
-directories for pre-flight checking where ``pth`` files can usefully
-be placed.
+directories for pre-flight checking where ``pth`` files can usefully be
+placed.
Otherwise, a `Google Code Search`_ turns up what appears to be a
roughly even mix of usage between packages using ``sys.prefix`` to
diff --git a/pep-0420.txt b/pep-0420.txt
--- a/pep-0420.txt
+++ b/pep-0420.txt
@@ -3,7 +3,7 @@
Version: $Revision$
Last-Modified: $Date$
Author: Eric V. Smith <eric at trueblade.com>
-Status: Accepted
+Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 19-Apr-2012
diff --git a/pep-0421.txt b/pep-0421.txt
--- a/pep-0421.txt
+++ b/pep-0421.txt
@@ -4,7 +4,7 @@
Last-Modified: $Date$
Author: Eric Snow <ericsnowcurrently at gmail.com>
BDFL-Delegate: Barry Warsaw
-Status: Accepted
+Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 26-April-2012
diff --git a/pep-0422.txt b/pep-0422.txt
new file mode 100644
--- /dev/null
+++ b/pep-0422.txt
@@ -0,0 +1,354 @@
+PEP: 422
+Title: Simple class initialisation hook
+Version: $Revision$
+Last-Modified: $Date$
+Author: Nick Coghlan <ncoghlan at gmail.com>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 5-Jun-2012
+Python-Version: 3.4
+Post-History: 5-Jun-2012
+
+
+Abstract
+========
+
+In Python 2, the body of a class definition could modify the way a class
+was created (or simply arrange to run other code after the class was created)
+by setting the ``__metaclass__`` attribute in the class body. While doing
+this implicitly from called code required the use of an implementation detail
+(specifically, ``sys._getframes()``), it could also be done explicitly in a
+fully supported fashion (for example, by passing ``locals()`` to an
+function that calculated a suitable ``__metaclass__`` value)
+
+There is currently no corresponding mechanism in Python 3 that allows the
+code executed in the class body to directly influence how the class object
+is created. Instead, the class creation process is fully defined by the
+class header, before the class body even begins executing.
+
+This PEP proposes a mechanism that will once again allow the body of a
+class definition to more directly influence the way a class is created
+(albeit in a more constrained fashion), as well as replacing some current
+uses of metaclasses with a simpler, easier to understand alternative.
+
+
+Background
+==========
+
+For an already created class ``cls``, the term "metaclass" has a clear
+meaning: it is the value of ``type(cls)``.
+
+*During* class creation, it has another meaning: it is also used to refer to
+the metaclass hint that may be provided as part of the class definition.
+While in many cases these two meanings end up referring to one and the same
+object, there are two situations where that is not the case:
+
+* If the metaclass hint refers to a subclass of ``type``, then it is
+ considered as a candidate metaclass along with the metaclasses of all of
+ the parents of the class being defined. If a more appropriate metaclass is
+ found amongst the candidates, then it will be used instead of the one
+ given in the metaclass hint.
+* Otherwise, an explicit metaclass hint is assumed to be a factory function
+ and is called directly to create the class object. In this case, the final
+ metaclass will be determined by the factory function definition. In the
+ typical case (where the factory functions just calls ``type``, or, in
+ Python 3.3 or later, ``types.new_class``) the actual metaclass is then
+ determined based on the parent classes.
+
+It is notable that only the actual metaclass is inherited - a factory
+function used as a metaclass hook sees only the class currently being
+defined, and is not invoked for any subclasses.
+
+In Python 3, the metaclass hint is provided using the ``metaclass=Meta``
+keyword syntax in the class header. This allows the ``__prepare__`` method
+on the metaclass to be used to create the ``locals()`` namespace used during
+execution of the class body (for example, specifying the use of
+``collections.OrderedDict`` instead of a regular ``dict``).
+
+In Python 2, there was no ``__prepare__`` method (that API was added for
+Python 3 by PEP 3115). Instead, a class body could set the ``__metaclass__``
+attribute, and the class creation process would extract that value from the
+class namespace to use as the metaclass hint. There is `published code`_ that
+makes use of this feature.
+
+Another new feature in Python 3 is the zero-argument form of the ``super()``
+builtin, introduced by PEP 3135. This feature uses an implicit ``__class__``
+reference to the class being defined to replace the "by name" references
+required in Python 2. Just as code invoked during execution of a Python 2
+metaclass could not call methods that referenced the class by name (as the
+name had not yet been bound in the containing scope), similarly, Python 3
+metaclasses cannot call methods that rely on the implicit ``__class__``
+reference (as it is not populated until after the metaclass has returned
+control to the class creation machiner).
+
+
+Proposal
+========
+
+This PEP proposes that a mechanism be added to Python 3 that meets the
+following criteria:
+
+1. Restores the ability for class namespaces to have some influence on the
+ class creation process (above and beyond populating the namespace itself),
+ but potentially without the full flexibility of the Python 2 style
+ ``__metaclass__`` hook
+2. Integrates nicely with class inheritance structures (including mixins and
+ multiple inheritance)
+3. Integrates nicely with the implicit ``__class__`` reference and
+ zero-argument ``super()`` syntax introduced by PEP 3135
+4. Can be added to an existing base class without a significant risk of
+ introducing backwards compatibility problems
+
+One mechanism that can achieve this goal is to add a new class
+initialisation hook, modelled directly on the existing instance
+initialisation hook, but with the signature constrained to match that
+of an ordinary class decorator.
+
+Specifically, it is proposed that class definitions be able to provide a
+class initialisation hook as follows::
+
+ class Example:
+ @classmethod
+ def __init_class__(cls):
+ # This is invoked after the class is created, but before any
+ # explicit decorators are called
+ # The usual super() mechanisms are used to correctly support
+ # multiple inheritance. The decorator style invocation helps
+ # ensure that invoking the parent class is as simple as possible.
+
+If present on the created object, this new hook will be called by the class
+creation machinery *after* the ``__class__`` reference has been initialised.
+For ``types.new_class()``, it will be called as the last step before
+returning the created class object.
+
+If a metaclass wishes to block class initialisation for some reason, it
+must arrange for ``cls.__init_class__`` to trigger ``AttributeError``.
+
+This general proposal is not a new idea (it was first suggested for
+inclusion in the language definition `more than 10 years ago`_, and a
+similar mechanism has long been supported by `Zope's ExtensionClass`_),
+but I believe the situation has changed sufficiently in recent years that
+the idea is worth reconsidering.
+
+
+Key Benefits
+============
+
+
+Replaces many use cases for dynamic setting of ``__metaclass__``
+-----------------------------------------------------------------
+
+For use cases that don't involve completely replacing the defined class,
+Python 2 code that dynamically set ``__metaclass__`` can now dynamically
+set ``__init_class__`` instead. For more advanced use cases, introduction of
+an explicit metaclass (possibly made available as a required base class) will
+still be necessary in order to support Python 3.
+
+
+Easier inheritance of definition time behaviour
+-----------------------------------------------
+
+Understanding Python's metaclasses requires a deep understanding of
+the type system and the class construction process. This is legitimately
+seen as challenging, due to the need to keep multiple moving parts (the code,
+the metaclass hint, the actual metaclass, the class object, instances of the
+class object) clearly distinct in your mind. Even when you know the rules,
+it's still easy to make a mistake if you're not being extremely careful.
+An earlier version of this PEP actually included such a mistake: it
+stated "instance of type" for a constraint that is actually "subclass of
+type".
+
+Understanding the proposed class initialisation hook only requires
+understanding decorators and ordinary method inheritance, which isn't
+quite as daunting a task. The new hook provides a more gradual path
+towards understanding all of the phases involved in the class definition
+process.
+
+
+Reduced chance of metaclass conflicts
+-------------------------------------
+
+One of the big issues that makes library authors reluctant to use metaclasses
+(even when they would be appropriate) is the risk of metaclass conflicts.
+These occur whenever two unrelated metaclasses are used by the desired
+parents of a class definition. This risk also makes it very difficult to
+*add* a metaclass to a class that has previously been published without one.
+
+By contrast, adding an ``__init_class__`` method to an existing type poses
+a similar level of risk to adding an ``__init__`` method: technically, there
+is a risk of breaking poorly implemented subclasses, but when that occurs,
+it is recognised as a bug in the subclass rather than the library author
+breaching backwards compatibility guarantees. In fact, due to the constrained
+signature of ``__init_class__``, the risk in this case is actually even
+lower than in the case of ``__init__``.
+
+
+Integrates cleanly with \PEP 3135
+---------------------------------
+
+Unlike code that runs as part of the metaclass, code that runs as part of
+the new hook will be able to freely invoke class methods that rely on the
+implicit ``__class__`` reference introduced by PEP 3135, including methods
+that use the zero argument form of ``super()``.
+
+
+Alternatives
+============
+
+
+The Python 3 Status Quo
+-----------------------
+
+The Python 3 status quo already offers a great deal of flexibility. For
+changes which only affect a single class definition and which can be
+specified at the time the code is written, then class decorators can be
+used to modify a class explicitly. Class decorators largely ignore class
+inheritance and can make full use of methods that rely on the ``__class__``
+reference being populated.
+
+Using a custom metaclass provides the same level of power as it did in
+Python 2. However, it's notable that, unlike class decorators, a metaclass
+cannot call any methods that rely on the ``__class__`` reference, as that
+reference is not populated until after the metaclass constructor returns
+control to the class creation code.
+
+One major use case for metaclasses actually closely resembles the use of
+class decorators. It occurs whenever a metaclass has an implementation that
+uses the following pattern::
+
+ class Metaclass(type):
+ def __new__(meta, *args, **kwds):
+ cls = super(Metaclass, meta).__new__(meta, *args, **kwds)
+ # Do something with cls
+ return cls
+
+The key difference between this pattern and a class decorator is that it
+is automatically inherited by subclasses. However, it also comes with a
+major disadvantage: Python does not allow you to inherit from classes with
+unrelated metaclasses.
+
+Thus, the status quo requires that developers choose between the following
+two alternatives:
+
+* Use a class decorator, meaning that behaviour is not inherited and must be
+ requested explicitly on every subclass
+* Use a metaclass, meaning that behaviour is inherited, but metaclass
+ conflicts may make integration with other libraries and frameworks more
+ difficult than it otherwise would be
+
+If this PEP is ultimately rejected, then this is the existing design that
+will remain in place by default.
+
+
+Restoring the Python 2 metaclass hook
+-------------------------------------
+
+One simple alternative would be to restore support for a Python 2 style
+``metaclass`` hook in the class body. This would be checked after the class
+body was executed, potentially overwriting the metaclass hint provided in the
+class header.
+
+The main attraction of such an approach is that it would simplify porting
+Python 2 applications that make use of this hook (especially those that do
+so dynamically).
+
+However, this approach does nothing to simplify the process of adding
+*inherited* class definition time behaviour, nor does it interoperate
+cleanly with the PEP 3135 ``__class__`` and ``super()`` semantics (as with
+any metaclass based solution, the ``__metaclass__`` hook would have to run
+before the ``__class__`` reference has been populated.
+
+
+Dynamic class decorators
+------------------------
+
+The original version of this PEP was called "Dynamic class decorators" and
+focused solely on a significantly more complicated proposal than that
+presented in the current version.
+
+As with the current version, it proposed that a new step be added to the
+class creation process, after the metaclass invocation to construct the
+class instance and before the application of lexical decorators. However,
+instead of a simple process of calling a single class method that relies
+on normal inheritance mechanisms, it proposed a far more complicated
+procedure that walked the class MRO looking for decorators stored in
+iterable ``__decorators__`` attributes.
+
+Using the current version of the PEP, the scheme originally proposed could
+be implemented as::
+
+ class DynamicDecorators:
+ @classmethod
+ def __init_class__(cls):
+ super(DynamicDecorators, cls).__init_class__()
+ for entry in reversed(cls.mro()):
+ decorators = entry.__dict__.get("__decorators__", ())
+ for deco in reversed(decorators):
+ cls = deco(cls)
+
+Any subclasses of this type would automatically have the contents of any
+``__decorators__`` attributes processed and invoked.
+
+The mechanism in the current PEP is considered superior, as many issues
+to do with ordering and the same decorator being invoked multiple times
+just go away, as that kind of thing is taken care of through the use of an
+ordinary class method invocation.
+
+
+Automatic metaclass derivation
+------------------------------
+
+When no appropriate metaclass is found, it's theoretically possible to
+automatically derive a metaclass for a new type based on the metaclass hint
+and the metaclasses of the bases.
+
+While adding such a mechanism would reduce the risk of spurious metaclass
+conflicts, it would do nothing to improve integration with PEP 3135, would
+not help with porting Python 2 code that set ``__metaclass__`` dynamically
+and would not provide a more straightforward inherited mechanism for invoking
+additional operations after the class invocation is complete.
+
+In addition, there would still be a risk of metaclass conflicts in cases
+where the base metaclasses were not written with multiple inheritance in
+mind. In such situations, there's a chance of introducing latent defects
+if one or more metaclasses are not invoked correctly.
+
+
+Calling the new hook from ``type.__init__``
+-------------------------------------------
+
+Calling the new hook automatically from ``type.__init__``, would achieve most
+of the goals of this PEP. However, using that approach would mean that
+``__init_class__`` implementations would be unable to call any methods that
+relied on the ``__class__`` reference (or used the zero-argument form of
+``super()``), and could not make use of those features themselves.
+
+
+References
+==========
+
+.. _published code:
+ http://mail.python.org/pipermail/python-dev/2012-June/119878.html
+
+.. _more than 10 years ago:
+ http://mail.python.org/pipermail/python-dev/2001-November/018651.html
+
+.. _Zope's ExtensionClass:
+ http://docs.zope.org/zope_secrets/extensionclass.html
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+..
+ Local Variables:
+ mode: indented-text
+ indent-tabs-mode: nil
+ sentence-end-double-space: t
+ fill-column: 70
+ coding: utf-8
+ End:
+
--
Repository URL: http://hg.python.org/peps
More information about the Python-checkins
mailing list