[Python-checkins] bpo-36974: expand call protocol documentation (GH-13844)
Miss Islington (bot)
webhook-mailer at python.org
Tue Nov 12 08:08:05 EST 2019
https://github.com/python/cpython/commit/9a13a388f202268dd7b771638adbec132449b98b
commit: 9a13a388f202268dd7b771638adbec132449b98b
branch: master
author: Jeroen Demeyer <jeroen.k.demeyer at gmail.com>
committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com>
date: 2019-11-12T05:08:00-08:00
summary:
bpo-36974: expand call protocol documentation (GH-13844)
CC @encukou
I'm also adding Petr Viktorin as contributor for vectorcall in the "what's new" section.
https://bugs.python.org/issue36974
Automerge-Triggered-By: @encukou
Automerge-Triggered-By: @encukou
files:
A Doc/c-api/call.rst
M Doc/c-api/abstract.rst
M Doc/c-api/exceptions.rst
M Doc/c-api/object.rst
M Doc/c-api/structures.rst
M Doc/c-api/type.rst
M Doc/c-api/typeobj.rst
M Doc/whatsnew/3.8.rst
diff --git a/Doc/c-api/abstract.rst b/Doc/c-api/abstract.rst
index 0edd1d5f6240a..1823f9d70c79f 100644
--- a/Doc/c-api/abstract.rst
+++ b/Doc/c-api/abstract.rst
@@ -18,6 +18,7 @@ but whose items have not been set to some non-\ ``NULL`` value yet.
.. toctree::
object.rst
+ call.rst
number.rst
sequence.rst
mapping.rst
diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst
new file mode 100644
index 0000000000000..0833531b1d5ee
--- /dev/null
+++ b/Doc/c-api/call.rst
@@ -0,0 +1,411 @@
+.. highlight:: c
+
+.. _call:
+
+Call Protocol
+=============
+
+CPython supports two different calling protocols:
+*tp_call* and vectorcall.
+
+The *tp_call* Protocol
+----------------------
+
+Instances of classes that set :c:member:`~PyTypeObject.tp_call` are callable.
+The signature of the slot is::
+
+ PyObject *tp_call(PyObject *callable, PyObject *args, PyObject *kwargs);
+
+A call is made using a tuple for the positional arguments
+and a dict for the keyword arguments, similarly to
+``callable(*args, **kwargs)`` in Python code.
+*args* must be non-NULL (use an empty tuple if there are no arguments)
+but *kwargs* may be *NULL* if there are no keyword arguments.
+
+This convention is not only used by *tp_call*:
+:c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_init`
+also pass arguments this way.
+
+To call an object, use :c:func:`PyObject_Call` or other
+:ref:`call API <capi-call>`.
+
+
+.. _vectorcall:
+
+The Vectorcall Protocol
+-----------------------
+
+.. versionadded:: 3.8
+
+The vectorcall protocol was introduced in :pep:`590` as an additional protocol
+for making calls more efficient.
+
+.. warning::
+
+ The vectorcall API is provisional and expected to become public in
+ Python 3.9, with a different names and, possibly, changed semantics.
+ If you use the it, plan for updating your code for Python 3.9.
+
+As rule of thumb, CPython will prefer the vectorcall for internal calls
+if the callable supports it. However, this is not a hard rule.
+Additionally, some third-party extensions use *tp_call* directly
+(rather than using :c:func:`PyObject_Call`).
+Therefore, a class supporting vectorcall must also implement
+:c:member:`~PyTypeObject.tp_call`.
+Moreover, the callable must behave the same
+regardless of which protocol is used.
+The recommended way to achieve this is by setting
+:c:member:`~PyTypeObject.tp_call` to :c:func:`PyVectorcall_Call`.
+This bears repeating:
+
+.. warning::
+
+ A class supporting vectorcall **must** also implement
+ :c:member:`~PyTypeObject.tp_call` with the same semantics.
+
+A class should not implement vectorcall if that would be slower
+than *tp_call*. For example, if the callee needs to convert
+the arguments to an args tuple and kwargs dict anyway, then there is no point
+in implementing vectorcall.
+
+Classes can implement the vectorcall protocol by enabling the
+:const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag and setting
+:c:member:`~PyTypeObject.tp_vectorcall_offset` to the offset inside the
+object structure where a *vectorcallfunc* appears.
+This is a pointer to a function with the following signature:
+
+.. c:type:: PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)
+
+- *callable* is the object being called.
+- *args* is a C array consisting of the positional arguments followed by the
+ values of the keyword arguments.
+ This can be *NULL* if there are no arguments.
+- *nargsf* is the number of positional arguments plus possibly the
+ :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag.
+ To get the actual number of positional arguments from *nargsf*,
+ use :c:func:`PyVectorcall_NARGS`.
+- *kwnames* is a tuple containing the names of the keyword arguments;
+ in other words, the keys of the kwargs dict.
+ These names must be strings (instances of ``str`` or a subclass)
+ and they must be unique.
+ If there are no keyword arguments, then *kwnames* can instead be *NULL*.
+
+.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET
+
+ If this flag is set in a vectorcall *nargsf* argument, the callee is allowed
+ to temporarily change ``args[-1]``. In other words, *args* points to
+ argument 1 (not 0) in the allocated vector.
+ The callee must restore the value of ``args[-1]`` before returning.
+
+ For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that
+ ``args[0]`` may be changed.
+
+ Whenever they can do so cheaply (without additional allocation), callers
+ are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`.
+ Doing so will allow callables such as bound methods to make their onward
+ calls (which include a prepended *self* argument) very efficiently.
+
+To call an object that implements vectorcall, use a :ref:`call API <capi-call>`
+function as with any other callable.
+:c:func:`_PyObject_Vectorcall` will usually be most efficient.
+
+
+Recursion Control
+.................
+
+When using *tp_call*, callees do not need to worry about
+:ref:`recursion <recursion>`: CPython uses
+:c:func:`Py_EnterRecursiveCall` and :c:func:`Py_LeaveRecursiveCall`
+for calls made using *tp_call*.
+
+For efficiency, this is not the case for calls done using vectorcall:
+the callee should use *Py_EnterRecursiveCall* and *Py_LeaveRecursiveCall*
+if needed.
+
+
+Vectorcall Support API
+......................
+
+.. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf)
+
+ Given a vectorcall *nargsf* argument, return the actual number of
+ arguments.
+ Currently equivalent to::
+
+ (Py_ssize_t)(nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET)
+
+ However, the function ``PyVectorcall_NARGS`` should be used to allow
+ for future extensions.
+
+ .. versionadded:: 3.8
+
+.. c:function:: vectorcallfunc _PyVectorcall_Function(PyObject *op)
+
+ If *op* does not support the vectorcall protocol (either because the type
+ does not or because the specific instance does not), return *NULL*.
+ Otherwise, return the vectorcall function pointer stored in *op*.
+ This function never raises an exception.
+
+ This is mostly useful to check whether or not *op* supports vectorcall,
+ which can be done by checking ``_PyVectorcall_Function(op) != NULL``.
+
+ .. versionadded:: 3.8
+
+.. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict)
+
+ Call *callable*'s :c:type:`vectorcallfunc` with positional and keyword
+ arguments given in a tuple and dict, respectively.
+
+ This is a specialized function, intended to be put in the
+ :c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``.
+ It does not check the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag
+ and it does not fall back to ``tp_call``.
+
+ .. versionadded:: 3.8
+
+
+.. _capi-call:
+
+Object Calling API
+------------------
+
+Various functions are available for calling a Python object.
+Each converts its arguments to a convention supported by the called object –
+either *tp_call* or vectorcall.
+In order to do as litle conversion as possible, pick one that best fits
+the format of data you have available.
+
+The following table summarizes the available functions;
+please see individual documentation for details.
+
++------------------------------------------+------------------+--------------------+---------------+
+| Function | callable | args | kwargs |
++==========================================+==================+====================+===============+
+| :c:func:`PyObject_Call` | ``PyObject *`` | tuple | dict/``NULL`` |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`PyObject_CallNoArgs` | ``PyObject *`` | --- | --- |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`_PyObject_CallOneArg` | ``PyObject *`` | 1 object | --- |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`PyObject_CallObject` | ``PyObject *`` | tuple/``NULL`` | --- |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`PyObject_CallFunction` | ``PyObject *`` | format | --- |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`PyObject_CallMethod` | obj + ``char*`` | format | --- |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`PyObject_CallFunctionObjArgs` | ``PyObject *`` | variadic | --- |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`PyObject_CallMethodObjArgs` | obj + name | variadic | --- |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`_PyObject_CallMethodNoArgs` | obj + name | --- | --- |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`_PyObject_CallMethodOneArg` | obj + name | 1 object | --- |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`_PyObject_Vectorcall` | ``PyObject *`` | vectorcall | vectorcall |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`_PyObject_FastCallDict` | ``PyObject *`` | vectorcall | dict/``NULL`` |
++------------------------------------------+------------------+--------------------+---------------+
+| :c:func:`_PyObject_VectorcallMethod` | arg + name | vectorcall | vectorcall |
++------------------------------------------+------------------+--------------------+---------------+
+
+
+.. c:function:: PyObject* PyObject_Call(PyObject *callable, PyObject *args, PyObject *kwargs)
+
+ Call a callable Python object *callable*, with arguments given by the
+ tuple *args*, and named arguments given by the dictionary *kwargs*.
+
+ *args* must not be *NULL*; use an empty tuple if no arguments are needed.
+ If no named arguments are needed, *kwargs* can be *NULL*.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ This is the equivalent of the Python expression:
+ ``callable(*args, **kwargs)``.
+
+
+.. c:function:: PyObject* PyObject_CallNoArgs(PyObject *callable)
+
+ Call a callable Python object *callable* without any arguments. It is the
+ most efficient way to call a callable Python object without any argument.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ .. versionadded:: 3.9
+
+
+.. c:function:: PyObject* _PyObject_CallOneArg(PyObject *callable, PyObject *arg)
+
+ Call a callable Python object *callable* with exactly 1 positional argument
+ *arg* and no keyword arguments.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ .. versionadded:: 3.9
+
+
+.. c:function:: PyObject* PyObject_CallObject(PyObject *callable, PyObject *args)
+
+ Call a callable Python object *callable*, with arguments given by the
+ tuple *args*. If no arguments are needed, then *args* can be *NULL*.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ This is the equivalent of the Python expression: ``callable(*args)``.
+
+
+.. c:function:: PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...)
+
+ Call a callable Python object *callable*, with a variable number of C arguments.
+ The C arguments are described using a :c:func:`Py_BuildValue` style format
+ string. The format can be *NULL*, indicating that no arguments are provided.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ This is the equivalent of the Python expression: ``callable(*args)``.
+
+ Note that if you only pass :c:type:`PyObject \*` args,
+ :c:func:`PyObject_CallFunctionObjArgs` is a faster alternative.
+
+ .. versionchanged:: 3.4
+ The type of *format* was changed from ``char *``.
+
+
+.. c:function:: PyObject* PyObject_CallMethod(PyObject *obj, const char *name, const char *format, ...)
+
+ Call the method named *name* of object *obj* with a variable number of C
+ arguments. The C arguments are described by a :c:func:`Py_BuildValue` format
+ string that should produce a tuple.
+
+ The format can be *NULL*, indicating that no arguments are provided.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ This is the equivalent of the Python expression:
+ ``obj.name(arg1, arg2, ...)``.
+
+ Note that if you only pass :c:type:`PyObject \*` args,
+ :c:func:`PyObject_CallMethodObjArgs` is a faster alternative.
+
+ .. versionchanged:: 3.4
+ The types of *name* and *format* were changed from ``char *``.
+
+
+.. c:function:: PyObject* PyObject_CallFunctionObjArgs(PyObject *callable, ..., NULL)
+
+ Call a callable Python object *callable*, with a variable number of
+ :c:type:`PyObject \*` arguments. The arguments are provided as a variable number
+ of parameters followed by *NULL*.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ This is the equivalent of the Python expression:
+ ``callable(arg1, arg2, ...)``.
+
+
+.. c:function:: PyObject* PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ..., NULL)
+
+ Call a method of the Python object *obj*, where the name of the method is given as a
+ Python string object in *name*. It is called with a variable number of
+ :c:type:`PyObject \*` arguments. The arguments are provided as a variable number
+ of parameters followed by *NULL*.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+
+.. c:function:: PyObject* _PyObject_CallMethodNoArgs(PyObject *obj, PyObject *name)
+
+ Call a method of the Python object *obj* without arguments,
+ where the name of the method is given as a Python string object in *name*.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ .. versionadded:: 3.9
+
+
+.. c:function:: PyObject* _PyObject_CallMethodOneArg(PyObject *obj, PyObject *name, PyObject *arg)
+
+ Call a method of the Python object *obj* with a single positional argument
+ *arg*, where the name of the method is given as a Python string object in
+ *name*.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ .. versionadded:: 3.9
+
+
+.. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)
+
+ Call a callable Python object *callable*.
+ The arguments are the same as for :c:type:`vectorcallfunc`.
+ If *callable* supports vectorcall_, this directly calls
+ the vectorcall function stored in *callable*.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ .. note::
+
+ This function is provisional and expected to become public in Python 3.9,
+ with a different name and, possibly, changed semantics.
+ If you use the function, plan for updating your code for Python 3.9.
+
+ .. versionadded:: 3.8
+
+.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict)
+
+ Call *callable* with positional arguments passed exactly as in the vectorcall_ protocol,
+ but with keyword arguments passed as a dictionary *kwdict*.
+ The *args* array contains only the positional arguments.
+
+ Regardless of which protocol is used internally,
+ a conversion of arguments needs to be done.
+ Therefore, this function should only be used if the caller
+ already has a dictionary ready to use for the keyword arguments,
+ but not a tuple for the positional arguments.
+
+ .. note::
+
+ This function is provisional and expected to become public in Python 3.9,
+ with a different name and, possibly, changed semantics.
+ If you use the function, plan for updating your code for Python 3.9.
+
+ .. versionadded:: 3.8
+
+.. c:function:: PyObject* _PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames)
+
+ Call a method using the vectorcall calling convention. The name of the method
+ is given as a Python string *name*. The object whose method is called is
+ *args[0]*, and the *args* array starting at *args[1]* represents the arguments
+ of the call. There must be at least one positional argument.
+ *nargsf* is the number of positional arguments including *args[0]*,
+ plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may
+ temporarily be changed. Keyword arguments can be passed just like in
+ :c:func:`_PyObject_Vectorcall`.
+
+ If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature,
+ this will call the unbound method object with the full
+ *args* vector as arguments.
+
+ Return the result of the call on success, or raise an exception and return
+ *NULL* on failure.
+
+ .. versionadded:: 3.9
+
+
+Call Support API
+----------------
+
+.. c:function:: int PyCallable_Check(PyObject *o)
+
+ Determine if the object *o* is callable. Return ``1`` if the object is callable
+ and ``0`` otherwise. This function always succeeds.
diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst
index cd6df00aeb5cd..2edcbf788d2ad 100644
--- a/Doc/c-api/exceptions.rst
+++ b/Doc/c-api/exceptions.rst
@@ -697,6 +697,8 @@ The following functions are used to create and modify Unicode exceptions from C.
``0`` on success, ``-1`` on failure.
+.. _recursion:
+
Recursion Control
=================
@@ -704,6 +706,8 @@ These two functions provide a way to perform safe recursive calls at the C
level, both in the core and in extension modules. They are needed if the
recursive code does not necessarily invoke Python code (which tracks its
recursion depth automatically).
+They are also not needed for *tp_call* implementations
+because the :ref:`call protocol <call>` takes care of recursion handling.
.. c:function:: int Py_EnterRecursiveCall(const char *where)
diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst
index 7d7a3be02fabc..ca9db1aee2f87 100644
--- a/Doc/c-api/object.rst
+++ b/Doc/c-api/object.rst
@@ -248,246 +248,6 @@ Object Protocol
of base classes).
-.. c:function:: int PyCallable_Check(PyObject *o)
-
- Determine if the object *o* is callable. Return ``1`` if the object is callable
- and ``0`` otherwise. This function always succeeds.
-
-
-.. c:function:: PyObject* PyObject_CallNoArgs(PyObject *callable)
-
- Call a callable Python object *callable* without any arguments. It is the
- most efficient way to call a callable Python object without any argument.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- .. versionadded:: 3.9
-
-
-.. c:function:: PyObject* _PyObject_CallOneArg(PyObject *callable, PyObject *arg)
-
- Call a callable Python object *callable* with exactly 1 positional argument
- *arg* and no keyword arguments.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- .. versionadded:: 3.9
-
-
-.. c:function:: PyObject* PyObject_Call(PyObject *callable, PyObject *args, PyObject *kwargs)
-
- Call a callable Python object *callable*, with arguments given by the
- tuple *args*, and named arguments given by the dictionary *kwargs*.
-
- *args* must not be ``NULL``, use an empty tuple if no arguments are needed.
- If no named arguments are needed, *kwargs* can be ``NULL``.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- This is the equivalent of the Python expression:
- ``callable(*args, **kwargs)``.
-
-
-.. c:function:: PyObject* PyObject_CallObject(PyObject *callable, PyObject *args)
-
- Call a callable Python object *callable*, with arguments given by the
- tuple *args*. If no arguments are needed, then *args* can be ``NULL``.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- This is the equivalent of the Python expression: ``callable(*args)``.
-
-
-.. c:function:: PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...)
-
- Call a callable Python object *callable*, with a variable number of C arguments.
- The C arguments are described using a :c:func:`Py_BuildValue` style format
- string. The format can be ``NULL``, indicating that no arguments are provided.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- This is the equivalent of the Python expression: ``callable(*args)``.
-
- Note that if you only pass :c:type:`PyObject \*` args,
- :c:func:`PyObject_CallFunctionObjArgs` is a faster alternative.
-
- .. versionchanged:: 3.4
- The type of *format* was changed from ``char *``.
-
-
-.. c:function:: PyObject* PyObject_CallMethod(PyObject *obj, const char *name, const char *format, ...)
-
- Call the method named *name* of object *obj* with a variable number of C
- arguments. The C arguments are described by a :c:func:`Py_BuildValue` format
- string that should produce a tuple.
-
- The format can be ``NULL``, indicating that no arguments are provided.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- This is the equivalent of the Python expression:
- ``obj.name(arg1, arg2, ...)``.
-
- Note that if you only pass :c:type:`PyObject \*` args,
- :c:func:`PyObject_CallMethodObjArgs` is a faster alternative.
-
- .. versionchanged:: 3.4
- The types of *name* and *format* were changed from ``char *``.
-
-
-.. c:function:: PyObject* PyObject_CallFunctionObjArgs(PyObject *callable, ..., NULL)
-
- Call a callable Python object *callable*, with a variable number of
- :c:type:`PyObject\*` arguments. The arguments are provided as a variable number
- of parameters followed by ``NULL``.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- This is the equivalent of the Python expression:
- ``callable(arg1, arg2, ...)``.
-
-
-.. c:function:: PyObject* PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ..., NULL)
-
- Calls a method of the Python object *obj*, where the name of the method is given as a
- Python string object in *name*. It is called with a variable number of
- :c:type:`PyObject\*` arguments. The arguments are provided as a variable number
- of parameters followed by ``NULL``.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
-
-.. c:function:: PyObject* _PyObject_CallMethodNoArgs(PyObject *obj, PyObject *name)
-
- Call a method of the Python object *obj* without arguments,
- where the name of the method is given as a Python string object in *name*.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- .. versionadded:: 3.9
-
-
-.. c:function:: PyObject* _PyObject_CallMethodOneArg(PyObject *obj, PyObject *name, PyObject *arg)
-
- Call a method of the Python object *obj* with a single positional argument
- *arg*, where the name of the method is given as a Python string object in
- *name*.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- .. versionadded:: 3.9
-
-
-.. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)
-
- Call a callable Python object *callable*, using
- :c:data:`vectorcall <PyTypeObject.tp_vectorcall_offset>` if possible.
-
- *args* is a C array with the positional arguments.
-
- *nargsf* is the number of positional arguments plus optionally the flag
- :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` (see below).
- To get actual number of arguments, use
- :c:func:`PyVectorcall_NARGS(nargsf) <PyVectorcall_NARGS>`.
-
- *kwnames* can be either ``NULL`` (no keyword arguments) or a tuple of keyword
- names, which must be strings. In the latter case, the values of the keyword
- arguments are stored in *args* after the positional arguments.
- The number of keyword arguments does not influence *nargsf*.
-
- *kwnames* must contain only objects of type ``str`` (not a subclass),
- and all keys must be unique.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- This uses the vectorcall protocol if the callable supports it;
- otherwise, the arguments are converted to use
- :c:member:`~PyTypeObject.tp_call`.
-
- .. note::
-
- This function is provisional and expected to become public in Python 3.9,
- with a different name and, possibly, changed semantics.
- If you use the function, plan for updating your code for Python 3.9.
-
- .. versionadded:: 3.8
-
-.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET
-
- If set in a vectorcall *nargsf* argument, the callee is allowed to
- temporarily change ``args[-1]``. In other words, *args* points to
- argument 1 (not 0) in the allocated vector.
- The callee must restore the value of ``args[-1]`` before returning.
-
- For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that
- ``args[0]`` may be changed.
-
- Whenever they can do so cheaply (without additional allocation), callers
- are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`.
- Doing so will allow callables such as bound methods to make their onward
- calls (which include a prepended *self* argument) cheaply.
-
- .. versionadded:: 3.8
-
-.. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf)
-
- Given a vectorcall *nargsf* argument, return the actual number of
- arguments.
- Currently equivalent to ``nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET``.
-
- .. versionadded:: 3.8
-
-.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict)
-
- Same as :c:func:`_PyObject_Vectorcall` except that the keyword arguments
- are passed as a dictionary in *kwdict*. This may be ``NULL`` if there
- are no keyword arguments.
-
- For callables supporting :c:data:`vectorcall <PyTypeObject.tp_vectorcall_offset>`,
- the arguments are internally converted to the vectorcall convention.
- Therefore, this function adds some overhead compared to
- :c:func:`_PyObject_Vectorcall`.
- It should only be used if the caller already has a dictionary ready to use.
-
- .. note::
-
- This function is provisional and expected to become public in Python 3.9,
- with a different name and, possibly, changed semantics.
- If you use the function, plan for updating your code for Python 3.9.
-
- .. versionadded:: 3.8
-
-.. c:function:: PyObject* _PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames)
-
- Call a method using the vectorcall calling convention. The name of the method
- is given as Python string *name*. The object whose method is called is
- *args[0]* and the *args* array starting at *args[1]* represents the arguments
- of the call. There must be at least one positional argument.
- *nargsf* is the number of positional arguments including *args[0]*,
- plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may
- temporarily be changed. Keyword arguments can be passed just like in
- :c:func:`_PyObject_Vectorcall`.
-
- If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature,
- this will actually call the unbound method object with the full
- *args* vector as arguments.
-
- Return the result of the call on success, or raise an exception and return
- ``NULL`` on failure.
-
- .. versionadded:: 3.9
-
.. c:function:: Py_hash_t PyObject_Hash(PyObject *o)
.. index:: builtin: hash
diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst
index 10352778f33b7..1bd769f275b35 100644
--- a/Doc/c-api/structures.rst
+++ b/Doc/c-api/structures.rst
@@ -208,7 +208,8 @@ also keyword arguments. So there are a total of 6 calling conventions:
Extension of :const:`METH_FASTCALL` supporting also keyword arguments,
with methods of type :c:type:`_PyCFunctionFastWithKeywords`.
- Keyword arguments are passed the same way as in the vectorcall protocol:
+ Keyword arguments are passed the same way as in the
+ :ref:`vectorcall protocol <vectorcall>`:
there is an additional fourth :c:type:`PyObject\*` parameter
which is a tuple representing the names of the keyword arguments
(which are guaranteed to be strings)
diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst
index b1b2df9f19042..41956b7dca503 100644
--- a/Doc/c-api/type.rst
+++ b/Doc/c-api/type.rst
@@ -193,6 +193,7 @@ The following functions and structs are used to create
(see :ref:`PyMemberDef <pymemberdef-offsets>`)
* :c:member:`~PyTypeObject.tp_dictoffset`
(see :ref:`PyMemberDef <pymemberdef-offsets>`)
+ * :c:member:`~PyTypeObject.tp_vectorcall_offset`
* :c:member:`~PyBufferProcs.bf_getbuffer`
* :c:member:`~PyBufferProcs.bf_releasebuffer`
diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
index bff5abfea48ee..b4ffd6b41c628 100644
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -49,7 +49,7 @@ Quick Reference
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
| :c:member:`~PyTypeObject.tp_dealloc` | :c:type:`destructor` | | X | X | | X |
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
- | :c:member:`~PyTypeObject.tp_vectorcall_offset` | Py_ssize_t | | | | | ? |
+ | :c:member:`~PyTypeObject.tp_vectorcall_offset` | Py_ssize_t | | | X | | X |
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
| (:c:member:`~PyTypeObject.tp_getattr`) | :c:type:`getattrfunc` | __getattribute__, | | | | G |
| | | __getattr__ | | | | |
@@ -145,6 +145,8 @@ Quick Reference
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
| :c:member:`~PyTypeObject.tp_finalize` | :c:type:`destructor` | __del__ | | | | X |
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
+ | :c:member:`~PyTypeObject.tp_vectorcall` | :c:type:`vectorcallfunc` | | | | | |
+ +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
If :const:`COUNT_ALLOCS` is defined then the following (internal-only)
fields exist as well:
@@ -180,7 +182,7 @@ fields exist as well:
.. code-block:: none
- X - type slot is inherited via PyType_Ready if defined with a NULL value
+ X - type slot is inherited via *PyType_Ready* if defined with a *NULL* value
% - the slots of the sub-struct are inherited individually
G - inherited, but only in combination with other slots; see the slot's description
? - it's complicated; see the slot's description
@@ -687,42 +689,29 @@ and :c:type:`PyType_Type` effectively act as defaults.)
.. c:member:: Py_ssize_t PyTypeObject.tp_vectorcall_offset
An optional offset to a per-instance function that implements calling
- the object using the *vectorcall* protocol, a more efficient alternative
+ the object using the :ref:`vectorcall protocol <vectorcall>`,
+ a more efficient alternative
of the simpler :c:member:`~PyTypeObject.tp_call`.
This field is only used if the flag :const:`_Py_TPFLAGS_HAVE_VECTORCALL`
is set. If so, this must be a positive integer containing the offset in the
instance of a :c:type:`vectorcallfunc` pointer.
- The signature is the same as for :c:func:`_PyObject_Vectorcall`::
-
- PyObject *vectorcallfunc(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)
- The *vectorcallfunc* pointer may be zero, in which case the instance behaves
+ The *vectorcallfunc* pointer may be ``NULL``, in which case the instance behaves
as if :const:`_Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance
falls back to :c:member:`~PyTypeObject.tp_call`.
Any class that sets ``_Py_TPFLAGS_HAVE_VECTORCALL`` must also set
:c:member:`~PyTypeObject.tp_call` and make sure its behaviour is consistent
with the *vectorcallfunc* function.
- This can be done by setting *tp_call* to ``PyVectorcall_Call``:
-
- .. c:function:: PyObject *PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict)
+ This can be done by setting *tp_call* to :c:func:`PyVectorcall_Call`.
- Call *callable*'s *vectorcallfunc* with positional and keyword
- arguments given in a tuple and dict, respectively.
-
- This function is intended to be used in the ``tp_call`` slot.
- It does not fall back to ``tp_call`` and it currently does not check the
- ``_Py_TPFLAGS_HAVE_VECTORCALL`` flag.
- To call an object, use one of the :c:func:`PyObject_Call <PyObject_Call>`
- functions instead.
-
- .. note::
+ .. warning::
It is not recommended for :ref:`heap types <heap-types>` to implement
the vectorcall protocol.
- When a user sets ``__call__`` in Python code, only ``tp_call`` is updated,
- possibly making it inconsistent with the vectorcall function.
+ When a user sets :attr:`__call__` in Python code, only *tp_call* is updated,
+ likely making it inconsistent with the vectorcall function.
.. note::
@@ -732,18 +721,19 @@ and :c:type:`PyType_Type` effectively act as defaults.)
.. versionchanged:: 3.8
- This slot was used for print formatting in Python 2.x.
- In Python 3.0 to 3.7, it was reserved and named ``tp_print``.
+ Before version 3.8, this slot was named ``tp_print``.
+ In Python 2.x, it was used for printing to a file.
+ In Python 3.0 to 3.7, it was unused.
**Inheritance:**
- This field is inherited by subtypes together with
- :c:member:`~PyTypeObject.tp_call`: a subtype inherits
- :c:member:`~PyTypeObject.tp_vectorcall_offset` from its base type when
- the subtype’s :c:member:`~PyTypeObject.tp_call` is ``NULL``.
-
- Note that `heap types`_ (including subclasses defined in Python) do not
- inherit the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag.
+ This field is always inherited.
+ However, the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag is not
+ always inherited. If it's not, then the subclass won't use
+ :ref:`vectorcall <vectorcall>`, except when
+ :c:func:`PyVectorcall_Call` is explicitly called.
+ This is in particular the case for `heap types`_
+ (including subclasses defined in Python).
.. c:member:: getattrfunc PyTypeObject.tp_getattr
@@ -1171,18 +1161,17 @@ and :c:type:`PyType_Type` effectively act as defaults.)
:c:member:`~PyTypeObject.tp_finalize` slot is always present in the
type structure.
+
.. data:: _Py_TPFLAGS_HAVE_VECTORCALL
- This bit is set when the class implements the vectorcall protocol.
+ This bit is set when the class implements
+ the :ref:`vectorcall protocol <vectorcall>`.
See :c:member:`~PyTypeObject.tp_vectorcall_offset` for details.
**Inheritance:**
- This bit is set on *static* subtypes if ``tp_flags`` is not overridden:
- a subtype inherits ``_Py_TPFLAGS_HAVE_VECTORCALL`` from its base type
- when the subtype’s :c:member:`~PyTypeObject.tp_call` is ``NULL``
- and the subtype's ``Py_TPFLAGS_HEAPTYPE`` is not set.
-
+ This bit is inherited for *static* subtypes if
+ :c:member:`~PyTypeObject.tp_call` is also inherited.
`Heap types`_ do not inherit ``_Py_TPFLAGS_HAVE_VECTORCALL``.
.. note::
@@ -1715,9 +1704,9 @@ and :c:type:`PyType_Type` effectively act as defaults.)
PyObject *tp_new(PyTypeObject *subtype, PyObject *args, PyObject *kwds);
- The subtype argument is the type of the object being created; the *args* and
+ The *subtype* argument is the type of the object being created; the *args* and
*kwds* arguments represent positional and keyword arguments of the call to the
- type. Note that subtype doesn't have to equal the type whose :c:member:`~PyTypeObject.tp_new`
+ type. Note that *subtype* doesn't have to equal the type whose :c:member:`~PyTypeObject.tp_new`
function is called; it may be a subtype of that type (but not an unrelated
type).
@@ -1900,6 +1889,21 @@ and :c:type:`PyType_Type` effectively act as defaults.)
.. seealso:: "Safe object finalization" (:pep:`442`)
+.. c:member:: vectorcallfunc PyTypeObject.tp_vectorcall
+
+ Vectorcall function to use for calls of this type object.
+ In other words, it is used to implement
+ :ref:`vectorcall <vectorcall>` for ``type.__call__``.
+ If ``tp_vectorcall`` is ``NULL``, the default call implementation
+ using :attr:`__new__` and :attr:`__init__` is used.
+
+ **Inheritance:**
+
+ This field is never inherited.
+
+ .. versionadded:: 3.9 (the field exists since 3.8 but it's only used since 3.9)
+
+
The remaining fields are only defined if the feature test macro
:const:`COUNT_ALLOCS` is defined, and are for internal use only. They are
documented here for completeness. None of these fields are inherited by
@@ -2369,14 +2373,6 @@ Slot Type typedefs
.. c:type:: void (*destructor)(PyObject *)
-.. c:type:: PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)
-
- See :c:member:`~PyTypeObject.tp_vectorcall_offset`.
-
- Arguments to ``vectorcallfunc`` are the same as for :c:func:`_PyObject_Vectorcall`.
-
- .. versionadded:: 3.8
-
.. c:type:: void (*freefunc)(void *)
See :c:member:`~PyTypeObject.tp_free`.
diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst
index 8338cb6cb4f4c..089095546fc72 100644
--- a/Doc/whatsnew/3.8.rst
+++ b/Doc/whatsnew/3.8.rst
@@ -347,20 +347,20 @@ See :pep:`587` for a full description.
(Contributed by Victor Stinner in :issue:`36763`.)
-Vectorcall: a fast calling protocol for CPython
------------------------------------------------
+PEP 590: Vectorcall: a fast calling protocol for CPython
+--------------------------------------------------------
-The "vectorcall" protocol is added to the Python/C API.
+:ref:`vectorcall` is added to the Python/C API.
It is meant to formalize existing optimizations which were already done
for various classes.
-Any extension type implementing a callable can use this protocol.
+Any static type implementing a callable can use this protocol.
This is currently provisional.
The aim is to make it fully public in Python 3.9.
See :pep:`590` for a full description.
-(Contributed by Jeroen Demeyer and Mark Shannon in :issue:`36974`.)
+(Contributed by Jeroen Demeyer, Mark Shannon and Petr Viktorin in :issue:`36974`.)
Pickle protocol 5 with out-of-band data buffers
More information about the Python-checkins
mailing list