[Python-checkins] bpo-41559: Documentation for PEP 612 (GH-24000)
gvanrossum
webhook-mailer at python.org
Fri Jan 1 19:45:59 EST 2021
https://github.com/python/cpython/commit/11276cd9c49faea66ce7760f26a238d1edbf6421
commit: 11276cd9c49faea66ce7760f26a238d1edbf6421
branch: master
author: Ken Jin <28750310+Fidget-Spinner at users.noreply.github.com>
committer: gvanrossum <gvanrossum at gmail.com>
date: 2021-01-01T16:45:50-08:00
summary:
bpo-41559: Documentation for PEP 612 (GH-24000)
files:
M Doc/library/stdtypes.rst
M Doc/library/typing.rst
M Doc/whatsnew/3.10.rst
M Lib/typing.py
diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
index 2869378bbdaf0..2331849c02e98 100644
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -4959,6 +4959,11 @@ All parameterized generics implement special read-only attributes.
(~T,)
+ .. note::
+ A ``GenericAlias`` object with :class:`typing.ParamSpec` parameters may not
+ have correct ``__parameters__`` after substitution because
+ :class:`typing.ParamSpec` is intended primarily for static type checking.
+
.. seealso::
* :pep:`585` -- "Type Hinting Generics In Standard Collections"
diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
index b07bb8943d16f..d74f8bcc27a20 100644
--- a/Doc/library/typing.rst
+++ b/Doc/library/typing.rst
@@ -18,7 +18,8 @@
--------------
This module provides runtime support for type hints as specified by
-:pep:`484`, :pep:`526`, :pep:`544`, :pep:`586`, :pep:`589`, :pep:`591`, and :pep:`613`.
+:pep:`484`, :pep:`526`, :pep:`544`, :pep:`586`, :pep:`589`, :pep:`591`,
+:pep:`612` and :pep:`613`.
The most fundamental support consists of the types :data:`Any`, :data:`Union`,
:data:`Tuple`, :data:`Callable`, :class:`TypeVar`, and
:class:`Generic`. For full specification please see :pep:`484`. For
@@ -171,6 +172,22 @@ It is possible to declare the return type of a callable without specifying
the call signature by substituting a literal ellipsis
for the list of arguments in the type hint: ``Callable[..., ReturnType]``.
+Callables which take other callables as arguments may indicate that their
+parameter types are dependent on each other using :class:`ParamSpec`.
+Additionally, if that callable adds or removes arguments from other
+callables, the :data:`Concatenate` operator may be used. They
+take the form ``Callable[ParamSpecVariable, ReturnType]`` and
+``Callable[Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable], ReturnType]``
+respectively.
+
+.. versionchanged:: 3.10
+ ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`.
+ See :pep:`612` for more information.
+
+.. seealso::
+ The documentation for :class:`ParamSpec` and :class:`Concatenate` provide
+ examples of usage in ``Callable``.
+
.. _generics:
Generics
@@ -316,6 +333,43 @@ User defined generic type aliases are also supported. Examples::
.. versionchanged:: 3.7
:class:`Generic` no longer has a custom metaclass.
+User-defined generics for parameter expressions are also supported via parameter
+specification variables in the form ``Generic[P]``. The behavior is consistent
+with type variables' described above as parameter specification variables are
+treated by the typing module as a specialized type variable. The one exception
+to this is that a list of types can be used to substitute a :class:`ParamSpec`::
+
+ >>> from typing import Generic, ParamSpec, TypeVar
+
+ >>> T = TypeVar('T')
+ >>> P = ParamSpec('P')
+
+ >>> class Z(Generic[T, P]): ...
+ ...
+ >>> Z[int, [dict, float]]
+ __main__.Z[int, (<class 'dict'>, <class 'float'>)]
+
+
+Furthermore, a generic with only one parameter specification variable will accept
+parameter lists in the forms ``X[[Type1, Type2, ...]]`` and also
+``X[Type1, Type2, ...]`` for aesthetic reasons. Internally, the latter is converted
+to the former and are thus equivalent::
+
+ >>> class X(Generic[P]): ...
+ ...
+ >>> X[int, str]
+ __main__.X[(<class 'int'>, <class 'str'>)]
+ >>> X[[int, str]]
+ __main__.X[(<class 'int'>, <class 'str'>)]
+
+Do note that generics with :class:`ParamSpec` may not have correct
+``__parameters__`` after substitution in some cases because they
+are intended primarily for static type checking.
+
+.. versionchanged:: 3.10
+ :class:`Generic` can now be parameterized over parameter expressions.
+ See :class:`ParamSpec` and :pep:`612` for more details.
+
A user-defined generic class can have ABCs as base classes without a metaclass
conflict. Generic metaclasses are not supported. The outcome of parameterizing
generics is cached, and most types in the typing module are hashable and
@@ -602,10 +656,80 @@ These can be used as types in annotations using ``[]``, each having a unique syn
``Callable[..., Any]``, and in turn to
:class:`collections.abc.Callable`.
+ Callables which take other callables as arguments may indicate that their
+ parameter types are dependent on each other using :class:`ParamSpec`.
+ Additionally, if that callable adds or removes arguments from other
+ callables, the :data:`Concatenate` operator may be used. They
+ take the form ``Callable[ParamSpecVariable, ReturnType]`` and
+ ``Callable[Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable], ReturnType]``
+ respectively.
+
.. deprecated:: 3.9
:class:`collections.abc.Callable` now supports ``[]``. See :pep:`585` and
:ref:`types-genericalias`.
+ .. versionchanged:: 3.10
+ ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`.
+ See :pep:`612` for more information.
+
+ .. seealso::
+ The documentation for :class:`ParamSpec` and :class:`Concatenate` provide
+ examples of usage with ``Callable``.
+
+.. data:: Concatenate
+
+ Used with :data:`Callable` and :class:`ParamSpec` to type annotate a higher
+ order callable which adds, removes, or transforms parameters of another
+ callable. Usage is in the form
+ ``Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable]``. ``Concatenate``
+ is currently only valid when used as the first argument to a :data:`Callable`.
+ The last parameter to ``Concatenate`` must be a :class:`ParamSpec`.
+
+ For example, to annotate a decorator ``with_lock`` which provides a
+ :class:`threading.Lock` to the decorated function, ``Concatenate`` can be
+ used to indicate that ``with_lock`` expects a callable which takes in a
+ ``Lock`` as the first argument, and returns a callable with a different type
+ signature. In this case, the :class:`ParamSpec` indicates that the returned
+ callable's parameter types are dependent on the parameter types of the
+ callable being passed in::
+
+ from collections.abc import Callable
+ from threading import Lock
+ from typing import Any, Concatenate, ParamSpec
+
+ P = ParamSpec('P')
+ R = ParamSpec('R')
+
+ # Use this lock to ensure that only one thread is executing a function
+ # at any time.
+ my_lock = Lock()
+
+ def with_lock(f: Callable[Concatenate[Lock, P], R]) -> Callable[P, R]:
+ '''A type-safe decorator which provides a lock.'''
+ global my_lock
+ def inner(*args: P.args, **kwargs: P.kwargs) -> T:
+ # Provide the lock as the first argument.
+ return f(my_lock, *args, **kwargs)
+ return inner
+
+ @with_lock
+ def sum_threadsafe(lock: Lock, numbers: list[float]) -> float:
+ '''Add a list of numbers together in a thread-safe manner.'''
+ with lock:
+ return sum(numbers)
+
+ # We don't need to pass in the lock ourselves thanks to the decorator.
+ sum_threadsafe([1.1, 2.2, 3.3])
+
+.. versionadded:: 3.10
+
+.. seealso::
+
+ * :pep:`612` -- Parameter Specification Variables (the PEP which introduced
+ ``ParamSpec`` and ``Concatenate``).
+ * :class:`ParamSpec` and :class:`Callable`.
+
+
.. class:: Type(Generic[CT_co])
A variable annotated with ``C`` may accept a value of type ``C``. In
@@ -876,6 +1000,84 @@ These are not used in annotations. They are building blocks for creating generic
for the type variable must be a subclass of the boundary type,
see :pep:`484`.
+.. class:: ParamSpec(name, *, bound=None, covariant=False, contravariant=False)
+
+ Parameter specification variable. A specialized version of
+ :class:`type variables <TypeVar>`.
+
+ Usage::
+
+ P = ParamSpec('P')
+
+ Parameter specification variables exist primarily for the benefit of static
+ type checkers. They are used to forward the parameter types of one
+ callable to another callable -- a pattern commonly found in higher order
+ functions and decorators. They are only valid when used in ``Concatenate``,
+ or as the first argument to ``Callable``, or as parameters for user-defined
+ Generics. See :class:`Generic` for more information on generic types.
+
+ For example, to add basic logging to a function, one can create a decorator
+ ``add_logging`` to log function calls. The parameter specification variable
+ tells the type checker that the callable passed into the decorator and the
+ new callable returned by it have inter-dependent type parameters::
+
+ from collections.abc import Callable
+ from typing import TypeVar, ParamSpec
+ import logging
+
+ T = TypeVar('T')
+ P = ParamSpec('P')
+
+ def add_logging(f: Callable[P, T]) -> Callable[P, T]:
+ '''A type-safe decorator to add logging to a function.'''
+ def inner(*args: P.args, **kwargs: P.kwargs) -> T:
+ logging.info(f'{f.__name__} was called')
+ return f(*args, **kwargs)
+ return inner
+
+ @add_logging
+ def add_two(x: float, y: float) -> float:
+ '''Add two numbers together.'''
+ return x + y
+
+ Without ``ParamSpec``, the simplest way to annotate this previously was to
+ use a :class:`TypeVar` with bound ``Callable[..., Any]``. However this
+ causes two problems:
+
+ 1. The type checker can't type check the ``inner`` function because
+ ``*args`` and ``**kwargs`` have to be typed :data:`Any`.
+ 2. :func:`~cast` may be required in the body of the ``add_logging``
+ decorator when returning the ``inner`` function, or the static type
+ checker must be told to ignore the ``return inner``.
+
+ .. attribute:: args
+ .. attribute:: kwargs
+
+ Since ``ParamSpec`` captures both positional and keyword parameters,
+ ``P.args`` and ``P.kwargs`` can be used to split a ``ParamSpec`` into its
+ components. ``P.args`` represents the tuple of positional parameters in a
+ given call and should only be used to annotate ``*args``. ``P.kwargs``
+ represents the mapping of keyword parameters to their values in a given call,
+ and should be only be used to annotate ``**kwargs`` or ``**kwds``. Both
+ attributes require the annotated parameter to be in scope.
+
+ Parameter specification variables created with ``covariant=True`` or
+ ``contravariant=True`` can be used to declare covariant or contravariant
+ generic types. The ``bound`` argument is also accepted, similar to
+ :class:`TypeVar`. However the actual semantics of these keywords are yet to
+ be decided.
+
+ .. versionadded:: 3.10
+
+ .. note::
+ Only parameter specification variables defined in global scope can
+ be pickled.
+
+ .. seealso::
+ * :pep:`612` -- Parameter Specification Variables (the PEP which introduced
+ ``ParamSpec`` and ``Concatenate``).
+ * :class:`Callable` and :class:`Concatenate`.
+
.. data:: AnyStr
``AnyStr`` is a type variable defined as
diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst
index aa547ff46481b..be529d75e13c0 100644
--- a/Doc/whatsnew/3.10.rst
+++ b/Doc/whatsnew/3.10.rst
@@ -144,6 +144,28 @@ See :pep:`604` for more details.
(Contributed by Maggie Moss and Philippe Prados in :issue:`41428`.)
+PEP 612: Parameter Specification Variables
+------------------------------------------
+
+Two new options to improve the information provided to static type checkers for
+:pep:`484`\ 's ``Callable`` have been added to the :mod:`typing` module.
+
+The first is the parameter specification variable. They are used to forward the
+parameter types of one callable to another callable -- a pattern commonly
+found in higher order functions and decorators. Examples of usage can be found
+in :class:`typing.ParamSpec`. Previously, there was no easy way to type annotate
+dependency of parameter types in such a precise manner.
+
+The second option is the new ``Concatenate`` operator. It's used in conjunction
+with parameter specification variables to type annotate a higher order callable
+which adds or removes parameters of another callable. Examples of usage can
+be found in :class:`typing.Concatenate`.
+
+See :class:`typing.Callable`, :class:`typing.ParamSpec`,
+:class:`typing.Concatenate` and :pep:`612` for more details.
+
+(Contributed by Ken Jin in :issue:`41559`.)
+
Other Language Changes
======================
diff --git a/Lib/typing.py b/Lib/typing.py
index b140b0e669626..88d0d623a421f 100644
--- a/Lib/typing.py
+++ b/Lib/typing.py
@@ -544,8 +544,9 @@ def TypeAlias(self, parameters):
@_SpecialForm
def Concatenate(self, parameters):
- """Used in conjunction with ParamSpec and Callable to represent a higher
- order function which adds, removes or transforms parameters of a Callable.
+ """Used in conjunction with ``ParamSpec`` and ``Callable`` to represent a
+ higher order function which adds, removes or transforms parameters of a
+ callable.
For example::
@@ -735,11 +736,11 @@ class ParamSpec(_Final, _Immutable, _TypeVarLike, _root=True):
Parameter specification variables exist primarily for the benefit of static
type checkers. They are used to forward the parameter types of one
- Callable to another Callable, a pattern commonly found in higher order
- functions and decorators. They are only valid when used in Concatenate, or
- as the first argument to Callable, or as parameters for user-defined Generics.
- See class Generic for more information on generic types. An example for
- annotating a decorator::
+ callable to another callable, a pattern commonly found in higher order
+ functions and decorators. They are only valid when used in ``Concatenate``,
+ or s the first argument to ``Callable``, or as parameters for user-defined
+ Generics. See class Generic for more information on generic types. An
+ example for annotating a decorator::
T = TypeVar('T')
P = ParamSpec('P')
More information about the Python-checkins
mailing list