[Python-checkins] bpo-43987: Add "Annotations Best Practices" HOWTO doc. (#25746)
larryhastings
webhook-mailer at python.org
Sun May 2 00:19:36 EDT 2021
https://github.com/python/cpython/commit/49b26fa517165f991c35a4afcbef1fcb26836bec
commit: 49b26fa517165f991c35a4afcbef1fcb26836bec
branch: master
author: larryhastings <larry at hastings.org>
committer: larryhastings <larry at hastings.org>
date: 2021-05-01T21:19:24-07:00
summary:
bpo-43987: Add "Annotations Best Practices" HOWTO doc. (#25746)
Add "Annotations Best Practices" HOWTO doc.
files:
A Doc/howto/annotations.rst
A Misc/NEWS.d/next/Documentation/2021-04-30-04-27-02.bpo-43987.1DftVa.rst
M Doc/glossary.rst
M Doc/howto/index.rst
M Doc/library/inspect.rst
M Doc/reference/datamodel.rst
M Doc/whatsnew/3.10.rst
diff --git a/Doc/glossary.rst b/Doc/glossary.rst
index 0661c8283290ce..29c68ed72c6d70 100644
--- a/Doc/glossary.rst
+++ b/Doc/glossary.rst
@@ -57,6 +57,8 @@ Glossary
See :term:`variable annotation`, :term:`function annotation`, :pep:`484`
and :pep:`526`, which describe this functionality.
+ Also see :ref:`annotations-howto`
+ for best practices on working with annotations.
argument
A value passed to a :term:`function` (or :term:`method`) when calling the
@@ -455,6 +457,8 @@ Glossary
See :term:`variable annotation` and :pep:`484`,
which describe this functionality.
+ Also see :ref:`annotations-howto`
+ for best practices on working with annotations.
__future__
A pseudo-module which programmers can use to enable new language features
@@ -1211,6 +1215,8 @@ Glossary
See :term:`function annotation`, :pep:`484`
and :pep:`526`, which describe this functionality.
+ Also see :ref:`annotations-howto`
+ for best practices on working with annotations.
virtual environment
A cooperatively isolated runtime environment that allows Python users
diff --git a/Doc/howto/annotations.rst b/Doc/howto/annotations.rst
new file mode 100644
index 00000000000000..3e61103e99c9a6
--- /dev/null
+++ b/Doc/howto/annotations.rst
@@ -0,0 +1,226 @@
+.. _annotations-howto:
+
+**************************
+Annotations Best Practices
+**************************
+
+:author: Larry Hastings
+
+.. topic:: Abstract
+
+ This document is designed to encapsulate the best practices
+ for working with annotations dicts. If you write Python code
+ that examines ``__annotations__`` on Python objects, we
+ encourage you to follow the guidelines described below.
+
+ The document is organized into four sections:
+ best practices for accessing the annotations of an object
+ in Python versions 3.10 and newer,
+ best practices for accessing the annotations of an object
+ in Python versions 3.9 and older,
+ other best practices
+ for ``__annotations__`` that apply to any Python version,
+ and
+ quirks of ``__annotations__``.
+
+ Note that this document is specifically about working with
+ ``__annotations__``, not uses *for* annotations.
+ If you're looking for information on how to use "type hints"
+ in your code, please see the :mod:`typing` module.
+
+
+Accessing The Annotations Dict Of An Object In Python 3.10 And Newer
+====================================================================
+
+ Python 3.10 adds a new function to the standard library:
+ :func:`inspect.get_annotations`. In Python versions 3.10
+ and newer, calling this function is the best practice for
+ accessing the annotations dict of any object that supports
+ annotations. This function can also "un-stringize"
+ stringized annotations for you.
+
+ If for some reason :func:`inspect.get_annotations` isn't
+ viable for your use case, you may access the
+ ``__annotations__`` data member manually. Best practice
+ for this changed in Python 3.10 as well: as of Python 3.10,
+ ``o.__annotations__`` is guaranteed to *always* work
+ on Python functions, classes, and modules. If you're
+ certain the object you're examining is one of these three
+ *specific* objects, you may simply use ``o.__annotations__``
+ to get at the object's annotations dict.
+
+ However, other types of callables--for example,
+ callables created by :func:`functools.partial`--may
+ not have an ``__annotations__`` attribute defined. When
+ accessing the ``__annotations__`` of a possibly unknown
+ object, best practice in Python versions 3.10 and
+ newer is to call :func:`getattr` with three arguments,
+ for example ``getattr(o, '__annotations__', None)``.
+
+
+Accessing The Annotations Dict Of An Object In Python 3.9 And Older
+===================================================================
+
+ In Python 3.9 and older, accessing the annotations dict
+ of an object is much more complicated than in newer versions.
+ The problem is a design flaw in these older versions of Python,
+ specifically to do with class annotations.
+
+ Best practice for accessing the annotations dict of other
+ objects--functions, other callables, and modules--is the same
+ as best practice for 3.10, assuming you aren't calling
+ :func:`inspect.get_annotations`: you should use three-argument
+ :func:`getattr` to access the object's ``__annotations__``
+ attribute.
+
+ Unfortunately, this isn't best practice for classes. The problem
+ is that, since ``__annotations__`` is optional on classes, and
+ because classes can inherit attributes from their base classes,
+ accessing the ``__annotations__`` attribute of a class may
+ inadvertently return the annotations dict of a *base class.*
+ As an example::
+
+ class Base:
+ a: int = 3
+ b: str = 'abc'
+
+ class Derived(Base):
+ pass
+
+ print(Derived.__annotations__)
+
+ This will print the annotations dict from ``Base``, not
+ ``Derived``.
+
+ Your code will have to have a separate code path if the object
+ you're examining is a class (``isinstance(o, type)``).
+ In that case, best practice relies on an implementation detail
+ of Python 3.9 and before: if a class has annotations defined,
+ they are stored in the class's ``__dict__`` dictionary. Since
+ the class may or may not have annotations defined, best practice
+ is to call the ``get`` method on the class dict.
+
+ To put it all together, here is some sample code that safely
+ accesses the ``__annotations__`` attribute on an arbitrary
+ object in Python 3.9 and before::
+
+ if isinstance(o, type):
+ ann = o.__dict__.get('__annotations__', None)
+ else:
+ ann = getattr(o, '__annotations__', None)
+
+ After running this code, ``ann`` should be either a
+ dictionary or ``None``. You're encouraged to double-check
+ the type of ``ann`` using :func:`isinstance` before further
+ examination.
+
+ Note that some exotic or malformed type objects may not have
+ a ``__dict__`` attribute, so for extra safety you may also wish
+ to use :func:`getattr` to access ``__dict__``.
+
+
+Manually Un-Stringizing Stringized Annotations
+==============================================
+
+ In situations where some annotations may be "stringized",
+ and you wish to evaluate those strings to produce the
+ Python values they represent, it really is best to
+ call :func:`inspect.get_annotations` to do this work
+ for you.
+
+ If you're using Python 3.9 or older, or if for some reason
+ you can't use :func:`inspect.get_annotations`, you'll need
+ to duplicate its logic. You're encouraged to examine the
+ implementation of :func:`inspect.get_annotations` in the
+ current Python version and follow a similar approach.
+
+ In a nutshell, if you wish to evaluate a stringized annotation
+ on an arbitrary object ``o``:
+
+ * If ``o`` is a module, use ``o.__dict__`` as the
+ ``globals`` when calling :func:`eval`.
+ * If ``o`` is a class, use ``sys.modules[o.__module__].__dict__``
+ as the ``globals``, and ``dict(vars(o))`` as the ``locals``,
+ when calling :func:`eval`.
+ * If ``o`` is a wrapped callable using :func:`functools.update_wrapper`,
+ :func:`functools.wraps`, or :func:`functools.partial`, iteratively
+ unwrap it by accessing either ``o.__wrapped__`` or ``o.func`` as
+ appropriate, until you have found the root unwrapped function.
+ * If ``o`` is a callable (but not a class), use
+ ``o.__globals__`` as the globals when calling :func:`eval`.
+
+ However, not all string values used as annotations can
+ be successfully turned into Python values by :func:`eval`.
+ String values could theoretically contain any valid string,
+ and in practice there are valid use cases for type hints that
+ require annotating with string values that specifically
+ *can't* be evaluated. For example:
+
+ * :pep:`604` union types using `|`, before support for this
+ was added to Python 3.10.
+ * Definitions that aren't needed at runtime, only imported
+ when :const:`typing.TYPE_CHECKING` is true.
+
+ If :func:`eval` attempts to evaluate such values, it will
+ fail and raise an exception. So, when designing a library
+ API that works with annotations, it's recommended to only
+ attempt to evaluate string values when explicitly requested
+ to by the caller.
+
+
+Best Practices For ``__annotations__`` In Any Python Version
+============================================================
+
+ * You should avoid assigning to the ``__annotations__`` member
+ of objects directly. Let Python manage setting ``__annotations__``.
+
+ * If you do assign directly to the ``__annotations__`` member
+ of an object, you should always set it to a ``dict`` object.
+
+ * If you directly access the ``__annotations__`` member
+ of an object, you should ensure that it's a
+ dictionary before attempting to examine its contents.
+
+ * You should avoid modifying ``__annotations__`` dicts.
+
+ * You should avoid deleting the ``__annotations__`` attribute
+ of an object.
+
+
+``__annotations__`` Quirks
+==========================
+
+ In all versions of Python 3, function
+ objects lazy-create an annotations dict if no annotations
+ are defined on that object. You can delete the ``__annotations__``
+ attribute using ``del fn.__annotations__``, but if you then
+ access ``fn.__annotations__`` the object will create a new empty dict
+ that it will store and return as its annotations. Deleting the
+ annotations on a function before it has lazily created its annotations
+ dict will throw an ``AttributeError``; using ``del fn.__annotations__``
+ twice in a row is guaranteed to always throw an ``AttributeError``.
+
+ Everything in the above paragraph also applies to class and module
+ objects in Python 3.10 and newer.
+
+ In all versions of Python 3, you can set ``__annotations__``
+ on a function object to ``None``. However, subsequently
+ accessing the annotations on that object using ``fn.__annotations__``
+ will lazy-create an empty dictionary as per the first paragraph of
+ this section. This is *not* true of modules and classes, in any Python
+ version; those objects permit setting ``__annotations__`` to any
+ Python value, and will retain whatever value is set.
+
+ If Python stringizes your annotations for you
+ (using ``from __future__ import annotations``), and you
+ specify a string as an annotation, the string will
+ itself be quoted. In effect the annotation is quoted
+ *twice.* For example::
+
+ from __future__ import annotations
+ def foo(a: "str"): pass
+
+ print(foo.__annotations__)
+
+ This prints ``{'a': "'str'"}``. This shouldn't really be considered
+ a "quirk"; it's mentioned here simply because it might be surprising.
diff --git a/Doc/howto/index.rst b/Doc/howto/index.rst
index e0dacd224d82ef..eae8f143ee206f 100644
--- a/Doc/howto/index.rst
+++ b/Doc/howto/index.rst
@@ -30,4 +30,5 @@ Currently, the HOWTOs are:
ipaddress.rst
clinic.rst
instrumentation.rst
+ annotations.rst
diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst
index 56c2f76708d43b..b9e8be1234e259 100644
--- a/Doc/library/inspect.rst
+++ b/Doc/library/inspect.rst
@@ -1149,6 +1149,9 @@ Classes and functions
with the result of calling :func:`eval()` on those values:
* If eval_str is true, :func:`eval()` is called on values of type ``str``.
+ (Note that ``get_annotations`` doesn't catch exceptions; if :func:`eval()`
+ raises an exception, it will unwind the stack past the ``get_annotations``
+ call.)
* If eval_str is false (the default), values of type ``str`` are unchanged.
``globals`` and ``locals`` are passed in to :func:`eval()`; see the documentation
@@ -1164,6 +1167,10 @@ Classes and functions
although if ``obj`` is a wrapped function (using
``functools.update_wrapper()``) it is first unwrapped.
+ Calling ``get_annotations`` is best practice for accessing the
+ annotations dict of any object. See :ref:`annotations-howto` for
+ more information on annotations best practices.
+
.. versionadded:: 3.10
diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
index 3a812eb21471a0..eefdc3d5100b56 100644
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -553,7 +553,10 @@ Callable types
| | the dict are the parameter | |
| | names, and ``'return'`` for | |
| | the return annotation, if | |
- | | provided. | |
+ | | provided. For more | |
+ | | information on working with | |
+ | | this attribute, see | |
+ | | :ref:`annotations-howto`. | |
+-------------------------+-------------------------------+-----------+
| :attr:`__kwdefaults__` | A dict containing defaults | Writable |
| | for keyword-only parameters. | |
@@ -748,16 +751,29 @@ Modules
single: __annotations__ (module attribute)
pair: module; namespace
- Predefined (writable) attributes: :attr:`__name__` is the module's name;
- :attr:`__doc__` is the module's documentation string, or ``None`` if
- unavailable; :attr:`__annotations__` (optional) is a dictionary containing
- :term:`variable annotations <variable annotation>` collected during module
- body execution; :attr:`__file__` is the pathname of the file from which the
- module was loaded, if it was loaded from a file. The :attr:`__file__`
- attribute may be missing for certain types of modules, such as C modules
- that are statically linked into the interpreter; for extension modules
- loaded dynamically from a shared library, it is the pathname of the shared
- library file.
+ Predefined (writable) attributes:
+
+ :attr:`__name__`
+ The module's name.
+
+ :attr:`__doc__`
+ The module's documentation string, or ``None`` if
+ unavailable.
+
+ :attr:`__file__`
+ The pathname of the file from which the
+ module was loaded, if it was loaded from a file.
+ The :attr:`__file__`
+ attribute may be missing for certain types of modules, such as C modules
+ that are statically linked into the interpreter. For extension modules
+ loaded dynamically from a shared library, it's the pathname of the shared
+ library file.
+
+ :attr:`__annotations__`
+ A dictionary containing
+ :term:`variable annotations <variable annotation>` collected during
+ module body execution. For best practices on working
+ with :attr:`__annotations__`, please see :ref:`annotations-howto`.
.. index:: single: __dict__ (module attribute)
@@ -821,14 +837,30 @@ Custom classes
single: __doc__ (class attribute)
single: __annotations__ (class attribute)
- Special attributes: :attr:`~definition.__name__` is the class name; :attr:`__module__` is
- the module name in which the class was defined; :attr:`~object.__dict__` is the
- dictionary containing the class's namespace; :attr:`~class.__bases__` is a
- tuple containing the base classes, in the order of their occurrence in the
- base class list; :attr:`__doc__` is the class's documentation string,
- or ``None`` if undefined; :attr:`__annotations__` (optional) is a dictionary
- containing :term:`variable annotations <variable annotation>` collected during
- class body execution.
+ Special attributes:
+
+ :attr:`~definition.__name__`
+ The class name.
+
+ :attr:`__module__`
+ The name of the module in which the class was defined.
+
+ :attr:`~object.__dict__`
+ The dictionary containing the class's namespace.
+
+ :attr:`~class.__bases__`
+ A tuple containing the base classes, in the order of
+ their occurrence in the base class list.
+
+ :attr:`__doc__`
+ The class's documentation string, or ``None`` if undefined.
+
+ :attr:`__annotations__`
+ A dictionary containing
+ :term:`variable annotations <variable annotation>`
+ collected during class body execution. For best practices on
+ working with :attr:`__annotations__`, please see
+ :ref:`annotations-howto`.
Class instances
.. index::
diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst
index a59e2e511154b2..679522bdfe71e8 100644
--- a/Doc/whatsnew/3.10.rst
+++ b/Doc/whatsnew/3.10.rst
@@ -807,7 +807,9 @@ Other Language Changes
* Class and module objects now lazy-create empty annotations dicts on demand.
The annotations dicts are stored in the object’s ``__dict__`` for
- backwards compatibility.
+ backwards compatibility. This improves the best practices for working
+ with ``__annotations__``; for more information, please see
+ :ref:`annotations-howto`.
(Contributed by Larry Hastings in :issue:`43901`.)
New Modules
@@ -996,7 +998,9 @@ defined on an object. It works around the quirks of accessing the annotations
on various types of objects, and makes very few assumptions about the object
it examines. :func:`inspect.get_annotations` can also correctly un-stringize
stringized annotations. :func:`inspect.get_annotations` is now considered
-best practice for accessing the annotations dict defined on any Python object.
+best practice for accessing the annotations dict defined on any Python object;
+for more information on best practices for working with annotations, please see
+:ref:`annotations-howto`.
Relatedly, :func:`inspect.signature`,
:func:`inspect.Signature.from_callable`, and ``inspect.Signature.from_function``
now call :func:`inspect.get_annotations` to retrieve annotations. This means
diff --git a/Misc/NEWS.d/next/Documentation/2021-04-30-04-27-02.bpo-43987.1DftVa.rst b/Misc/NEWS.d/next/Documentation/2021-04-30-04-27-02.bpo-43987.1DftVa.rst
new file mode 100644
index 00000000000000..158259e3ab378e
--- /dev/null
+++ b/Misc/NEWS.d/next/Documentation/2021-04-30-04-27-02.bpo-43987.1DftVa.rst
@@ -0,0 +1 @@
+Add "Annotations Best Practices" document as a new HOWTO.
More information about the Python-checkins
mailing list