[Python-checkins] peps: Source layout tweaks and minor clarifications
ronald.oussoren
python-checkins at python.org
Wed Jul 22 09:14:26 CEST 2015
https://hg.python.org/peps/rev/f5f06f202b82
changeset: 5907:f5f06f202b82
user: Ronald Oussoren <ronaldoussoren at mac.com>
date: Wed Jul 22 09:14:17 2015 +0200
summary:
Source layout tweaks and minor clarifications
files:
pep-0447.txt | 117 +++++++++++++++++++++++++++++----------
1 files changed, 87 insertions(+), 30 deletions(-)
diff --git a/pep-0447.txt b/pep-0447.txt
--- a/pep-0447.txt
+++ b/pep-0447.txt
@@ -7,7 +7,7 @@
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Jun-2013
-Post-History: 2-Jul-2013, 15-Jul-2013, 29-Jul-2013
+Post-History: 2-Jul-2013, 15-Jul-2013, 29-Jul-2013, 22-Jul-2015
Abstract
@@ -16,7 +16,8 @@
Currently ``object.__getattribute__`` and ``super.__getattribute__`` peek
in the ``__dict__`` of classes on the MRO for a class when looking for
an attribute. This PEP adds an optional ``__getdescriptor__`` method to
-a metaclass that can be used to override this behavior.
+a metaclass that replaces this behavior and gives more control over attribute
+lookup, especially when using a `super`_ object.
That is, the MRO walking loop in ``_PyType_Lookup`` and
``super.__getattribute__`` gets changed from::
@@ -39,6 +40,15 @@
return NotFound
+The default implemention of ``__getdescriptor__`` looks in the class
+dictionary::
+
+ class type:
+ def __getdescriptor__(cls, name):
+ try:
+ return cls.__dict__[name]
+ except KeyError:
+ raise AttributeError(name) from None
Rationale
=========
@@ -63,13 +73,23 @@
an example of which is `PyObjC`_. PyObjC creates a Python class for every
class in the Objective-C runtime, and looks up methods in the Objective-C
runtime when they are used. This works fine for normal access, but doesn't
-work for access with ``super`` objects. Because of this PyObjC currently
-includes a custom ``super`` that must be used with its classes.
+work for access with `super`_ objects. Because of this PyObjC currently
+includes a custom `super`_ that must be used with its classes, as well as
+completely reimplementing `PyObject_GenericGetAttr`_ for normal attribute
+access.
-The API in this PEP makes it possible to remove the custom ``super`` and
+The API in this PEP makes it possible to remove the custom `super`_ and
simplifies the implementation because the custom lookup behavior can be
added in a central location.
+.. note::
+
+ `PyObjC`_ cannot precalculate the contents of the class ``__dict__``
+ because Objective-C classes can grow new methods at runtime. Furthermore
+ Objective-C classes tend to contain a lot of methods while most Python
+ code will only use a small subset of them, this makes precalculating
+ unnecessarily expensive.
+
The superclass attribute lookup hook
====================================
@@ -88,12 +108,12 @@
Aside: Attribute resolution algorithm in Python
-----------------------------------------------
-The attribute resolution proces as implemented by ``object.__getattribute__`` (or
-PyObject_GenericGetAttr`` in CPython's implementation) is fairly straightforward,
-but not entirely so without reading C code.
+The attribute resolution proces as implemented by ``object.__getattribute__``
+(or PyObject_GenericGetAttr`` in CPython's implementation) is fairly
+straightforward, but not entirely so without reading C code.
-The current CPython implementation of object.__getattribute__ is basicly equivalent
-to the following (pseudo-) Python code (excluding some house keeping and speed tricks)::
+The current CPython implementation of object.__getattribute__ is basicly
+equivalent to the following (pseudo-) Python code (excluding some house keeping and speed tricks)::
def _PyType_Lookup(tp, name):
@@ -179,8 +199,8 @@
This PEP should change the dict lookup at the lines starting at "# PEP 447" with
-a method call to perform the actual lookup, making is possible to affect that lookup
-both for normal attribute access and access through the `super proxy`_.
+a method call to perform the actual lookup, making is possible to affect that
+lookup both for normal attribute access and access through the `super proxy`_.
Note that specific classes can already completely override the default
behaviour by implementing their own ``__getattribute__`` slot (with or without
@@ -232,18 +252,19 @@
obj = SillyObject()
assert obj.m() == "fortytwo"
-As mentioned earlier in this PEP a more realistic use case of this functionallity
-is a ``__getdescriptor__`` method that dynamicly populates the class ``__dict__``
-based on attribute access, primarily when it is not possible to reliably keep the
-class dict in sync with its source, for example because the source used to populate
-``__dict__`` is dynamic as well and does not have triggers that can be used to detect
-changes to that source.
+As mentioned earlier in this PEP a more realistic use case of this
+functionallity is a ``__getdescriptor__`` method that dynamicly populates the
+class ``__dict__`` based on attribute access, primarily when it is not
+possible to reliably keep the class dict in sync with its source, for example
+because the source used to populate ``__dict__`` is dynamic as well and does
+not have triggers that can be used to detect changes to that source.
-An example of that are the class bridges in PyObjC: the class bridge is a Python
-object (class) that represents an Objective-C class and conceptually has a Python
-method for every Objective-C method in the Objective-C class. As with Python it is
-possible to add new methods to an Objective-C class, or replace existing ones, and
-there are no callbacks that can be used to detect this.
+An example of that are the class bridges in PyObjC: the class bridge is a
+Python object (class) that represents an Objective-C class and conceptually
+has a Python method for every Objective-C method in the Objective-C class.
+As with Python it is possible to add new methods to an Objective-C class, or
+replace existing ones, and there are no callbacks that can be used to detect
+this.
In C code
---------
@@ -290,16 +311,20 @@
with a metaclass that uses a custom ``__getdescriptor__`` method. This section
lists those changes.
+The items listed below are only affected by custom ``__getdescriptor__``
+methods, the default implementation for ``object`` won't cause problems
+because that still only uses the class ``__dict__`` and won't cause visible
+changes to the visible behaviour of the ``object.__getattribute__``.
+
* ``dir`` might not show all attributes
- As with a custom ``__getattribute__`` method ``dir()`` might not see all
+ As with a custom ``__getattribute__`` method `dir()`_ might not see all
(instance) attributes when using the ``__getdescriptor__()`` method to
dynamicly resolve attributes.
The solution for that is quite simple: classes using ``__getdescriptor__``
- should also implement ``__dir__`` if they want full support for the builtin
- ``dir`` function.
-
+ should also implement `__dir__()`_ if they want full support for the builtin
+ `dir()`_ function.
* ``inspect.getattr_static`` might not show all attributes
@@ -316,13 +341,26 @@
**TODO**: I haven't fully worked out what the impact of this is, and if there
are mitigations for those using either updates to these functions, or
- additional methods that users should implement to be fully compatible with these
- functions.
+ additional methods that users should implement to be fully compatible with
+ these functions.
+
+ One possible mitigation is to have a custom ``__getattribute__`` for these
+ classes that fills ``__dict__`` before returning and and defers to the
+ default implementation for other attributes.
+
+* Direct introspection of the class ``__dict__``
+
+ Any code that directly access the class ``__dict__`` for introspection
+ can be affected by a custom ``__getdescriptor__`` method.
Performance impact
------------------
+**WARNING**: The benchmark results in this section are old, and will be updated
+when I've ported the patch to the current trunk. I don't expect significant
+changes to the results in this section.
+
The pybench output below compares an implementation of this PEP with the
regular source tree, both based on changeset a5681f50bae2, run on an idle
machine an Core i7 processor running Centos 6.4.
@@ -567,11 +605,22 @@
This would mean that using ``tp_getattro`` instead of peeking the class
dictionaries changes the semantics of the `super class`_.
+Alternate placement of the new method
+.....................................
+
+This PEP proposes to add ``__getdescriptor__`` as a method on the metaclass.
+An alternative would be to add it as a class method on the class itself
+(simular to how ``__new__`` is a `staticmethod`_ of the class and not a method
+of the metaclass).
+
+The two are functionally equivalent, and there's something to be said about
+not requiring the use of a meta class.
+
References
==========
-* `Issue 18181`_ contains a prototype implementation
+* `Issue 18181`_ contains an out of date prototype implementation
Copyright
=========
@@ -584,6 +633,14 @@
.. _`super proxy`: http://docs.python.org/3/library/functions.html#super
+.. _`super`: http://docs.python.org/3/library/functions.html#super
+
+.. _`dir()`: http://docs.python.org/3/library/functions.html#dir
+
+.. _`staticmethod`: http://docs.python.org/3/library/functions.html#staticmethod
+
+.. _`__dir__()`: https://docs.python.org/3/reference/datamodel.html#object.__dir__
+
.. _`NotImplemented`: http://docs.python.org/3/library/constants.html#NotImplemented
.. _`PyObject_GenericGetAttr`: http://docs.python.org/3/c-api/object.html#PyObject_GenericGetAttr
--
Repository URL: https://hg.python.org/peps
More information about the Python-checkins
mailing list