From webhook-mailer at python.org Tue Feb 1 04:59:00 2022 From: webhook-mailer at python.org (Fidget-Spinner) Date: Tue, 01 Feb 2022 09:59:00 -0000 Subject: [Python-checkins] bpo-46482: add a test for `typing.Annotation.__new__` (GH-30821) Message-ID: https://github.com/python/cpython/commit/4c0612ad00ba45dbea2a86f7db6d21546cf243f8 commit: 4c0612ad00ba45dbea2a86f7db6d21546cf243f8 branch: main author: Nikita Sobolev committer: Fidget-Spinner <28750310+Fidget-Spinner at users.noreply.github.com> date: 2022-02-01T17:58:41+08:00 summary: bpo-46482: add a test for `typing.Annotation.__new__` (GH-30821) files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 8449affd03a76..090d4c70d3d84 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4643,6 +4643,13 @@ class A(typing.Match): class AnnotatedTests(BaseTestCase): + def test_new(self): + with self.assertRaisesRegex( + TypeError, + 'Type Annotated cannot be instantiated', + ): + Annotated() + def test_repr(self): self.assertEqual( repr(Annotated[int, 4, 5]), From webhook-mailer at python.org Tue Feb 1 05:22:38 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 10:22:38 -0000 Subject: [Python-checkins] bpo-46355: What's New: Note that PyFrameObject are private (GH-31032) Message-ID: https://github.com/python/cpython/commit/a4cb31927a1f0ee31025ea1ca82fcbfad44755dc commit: a4cb31927a1f0ee31025ea1ca82fcbfad44755dc branch: main author: Petr Viktorin committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T02:22:34-08:00 summary: bpo-46355: What's New: Note that PyFrameObject are private (GH-31032) This adds a slightly more detailed explanation of the change. The most important point is that the changed/removed fields were "subject to change" to begin with. files: M Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index e054008753a94..e7f3dab2b51db 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -759,12 +759,19 @@ Porting to Python 3.11 which are not available in the limited C API. (Contributed by Victor Stinner in :issue:`46007`.) -* Changes of the :c:type:`PyFrameObject` structure members: +* Changes of the private :c:type:`PyFrameObject` structure members. + + While the documentation notes that the fields of ``PyFrameObject`` are + subject to change at any time, they have been stable for a long time + and were used in several popular extensions. + In Python 3.11, the frame struct was reorganized to allow performance + optimizations. Rather than reading the fields directly, extensions should + use functions: * ``f_code``: removed, use :c:func:`PyFrame_GetCode` instead. Warning: the function returns a :term:`strong reference`, need to call :c:func:`Py_DECREF`. - * ``f_back``: changed, use :c:func:`PyFrame_GetBack`. + * ``f_back``: changed (see below), use :c:func:`PyFrame_GetBack`. * ``f_builtins``: removed, use ``PyObject_GetAttrString(frame, "f_builtins")``. * ``f_globals``: removed, @@ -773,13 +780,17 @@ Porting to Python 3.11 use ``PyObject_GetAttrString(frame, "f_locals")``. * ``f_lasti``: removed, use ``PyObject_GetAttrString(frame, "f_lasti")``. - * ``f_valuesstack``: removed. - * ``f_stackdepth``: removed. - * ``f_gen``: removed. - * ``f_iblock``: removed. - * ``f_state``: removed. - * ``f_blockstack``: removed. - * ``f_localsplus``: removed. + + The following fields were removed entirely, as they were details + of the old implementation: + + * ``f_valuesstack`` + * ``f_stackdepth`` + * ``f_gen`` + * ``f_iblock`` + * ``f_state`` + * ``f_blockstack`` + * ``f_localsplus`` The Python frame object is now created lazily. A side effect is that the ``f_back`` member must not be accessed directly, since its value is now also From webhook-mailer at python.org Tue Feb 1 05:25:43 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 10:25:43 -0000 Subject: [Python-checkins] [3.9] bpo-46482: add a test for `typing.Annotation.__new__` (GH-30821) (GH-31045) Message-ID: https://github.com/python/cpython/commit/2532b7c820ec2dc87b19eb322ab92b47f3c77866 commit: 2532b7c820ec2dc87b19eb322ab92b47f3c77866 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T02:25:25-08:00 summary: [3.9] bpo-46482: add a test for `typing.Annotation.__new__` (GH-30821) (GH-31045) (cherry picked from commit 4c0612ad00ba45dbea2a86f7db6d21546cf243f8) Co-authored-by: Nikita Sobolev Automerge-Triggered-By: GH:Fidget-Spinner files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index dfca96b909932..27ec5edd92b7a 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4158,6 +4158,13 @@ class A(typing.Match): class AnnotatedTests(BaseTestCase): + def test_new(self): + with self.assertRaisesRegex( + TypeError, + 'Type Annotated cannot be instantiated', + ): + Annotated() + def test_repr(self): self.assertEqual( repr(Annotated[int, 4, 5]), From webhook-mailer at python.org Tue Feb 1 05:27:41 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 10:27:41 -0000 Subject: [Python-checkins] bpo-46482: add a test for `typing.Annotation.__new__` (GH-30821) Message-ID: https://github.com/python/cpython/commit/6a188d88c562bfd68ef3a32d148d9b234d50646e commit: 6a188d88c562bfd68ef3a32d148d9b234d50646e branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T02:27:36-08:00 summary: bpo-46482: add a test for `typing.Annotation.__new__` (GH-30821) (cherry picked from commit 4c0612ad00ba45dbea2a86f7db6d21546cf243f8) Co-authored-by: Nikita Sobolev files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index e5b596e7f3d82..366f7d8eb31d1 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4543,6 +4543,13 @@ class A(typing.Match): class AnnotatedTests(BaseTestCase): + def test_new(self): + with self.assertRaisesRegex( + TypeError, + 'Type Annotated cannot be instantiated', + ): + Annotated() + def test_repr(self): self.assertEqual( repr(Annotated[int, 4, 5]), From webhook-mailer at python.org Tue Feb 1 05:32:08 2022 From: webhook-mailer at python.org (iritkatriel) Date: Tue, 01 Feb 2022 10:32:08 -0000 Subject: [Python-checkins] bpo-44031: fix test_tabnanny failure in non-ascii CWD (GH-31014) Message-ID: https://github.com/python/cpython/commit/108e66b6d23efd0fc2966163ead9434b328c5f17 commit: 108e66b6d23efd0fc2966163ead9434b328c5f17 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2022-02-01T10:31:57Z summary: bpo-44031: fix test_tabnanny failure in non-ascii CWD (GH-31014) files: M Lib/test/test_tabnanny.py diff --git a/Lib/test/test_tabnanny.py b/Lib/test/test_tabnanny.py index 4dfbd2985d5b5..59fdfc5573d37 100644 --- a/Lib/test/test_tabnanny.py +++ b/Lib/test/test_tabnanny.py @@ -293,8 +293,8 @@ def validate_cmd(self, *args, stdout="", stderr="", partial=False): _, out, err = script_helper.assert_python_ok('-m', 'tabnanny', *args) # Note: The `splitlines()` will solve the problem of CRLF(\r) added # by OS Windows. - out = out.decode('ascii') - err = err.decode('ascii') + out = os.fsdecode(out) + err = os.fsdecode(err) if partial: for std, output in ((stdout, out), (stderr, err)): _output = output.splitlines() From webhook-mailer at python.org Tue Feb 1 06:20:20 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 11:20:20 -0000 Subject: [Python-checkins] bpo-44031: fix test_tabnanny failure in non-ascii CWD (GH-31014) Message-ID: https://github.com/python/cpython/commit/1dcd77280410e4b3bd7b0680f00a38cea466ebd1 commit: 1dcd77280410e4b3bd7b0680f00a38cea466ebd1 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T03:20:07-08:00 summary: bpo-44031: fix test_tabnanny failure in non-ascii CWD (GH-31014) (cherry picked from commit 108e66b6d23efd0fc2966163ead9434b328c5f17) Co-authored-by: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> files: M Lib/test/test_tabnanny.py diff --git a/Lib/test/test_tabnanny.py b/Lib/test/test_tabnanny.py index 4dfbd2985d5b5..59fdfc5573d37 100644 --- a/Lib/test/test_tabnanny.py +++ b/Lib/test/test_tabnanny.py @@ -293,8 +293,8 @@ def validate_cmd(self, *args, stdout="", stderr="", partial=False): _, out, err = script_helper.assert_python_ok('-m', 'tabnanny', *args) # Note: The `splitlines()` will solve the problem of CRLF(\r) added # by OS Windows. - out = out.decode('ascii') - err = err.decode('ascii') + out = os.fsdecode(out) + err = os.fsdecode(err) if partial: for std, output in ((stdout, out), (stderr, err)): _output = output.splitlines() From webhook-mailer at python.org Tue Feb 1 06:32:08 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 11:32:08 -0000 Subject: [Python-checkins] [3.9] bpo-44031: fix test_tabnanny failure in non-ascii CWD (GH-31014) (GH-31048) Message-ID: https://github.com/python/cpython/commit/2e9f77f9d4fc282666ff85051b6e7ade0eed1d4c commit: 2e9f77f9d4fc282666ff85051b6e7ade0eed1d4c branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T03:31:49-08:00 summary: [3.9] bpo-44031: fix test_tabnanny failure in non-ascii CWD (GH-31014) (GH-31048) (cherry picked from commit 108e66b6d23efd0fc2966163ead9434b328c5f17) Co-authored-by: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> Automerge-Triggered-By: GH:iritkatriel files: M Lib/test/test_tabnanny.py diff --git a/Lib/test/test_tabnanny.py b/Lib/test/test_tabnanny.py index 95840d6ac0c5f..7f3308d745aa0 100644 --- a/Lib/test/test_tabnanny.py +++ b/Lib/test/test_tabnanny.py @@ -292,8 +292,8 @@ def validate_cmd(self, *args, stdout="", stderr="", partial=False): _, out, err = script_helper.assert_python_ok('-m', 'tabnanny', *args) # Note: The `splitlines()` will solve the problem of CRLF(\r) added # by OS Windows. - out = out.decode('ascii') - err = err.decode('ascii') + out = os.fsdecode(out) + err = os.fsdecode(err) if partial: for std, output in ((stdout, out), (stderr, err)): _output = output.splitlines() From webhook-mailer at python.org Tue Feb 1 08:41:24 2022 From: webhook-mailer at python.org (Fidget-Spinner) Date: Tue, 01 Feb 2022 13:41:24 -0000 Subject: [Python-checkins] bpo-46564: do not create frame object for super object (GH-31002) Message-ID: https://github.com/python/cpython/commit/b9ebde8db7e176e021103745cd012bae700828b5 commit: b9ebde8db7e176e021103745cd012bae700828b5 branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: Fidget-Spinner <28750310+Fidget-Spinner at users.noreply.github.com> date: 2022-02-01T21:40:46+08:00 summary: bpo-46564: do not create frame object for super object (GH-31002) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-01-10-23-21.bpo-46564.6Xc2_H.rst M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-10-23-21.bpo-46564.6Xc2_H.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-10-23-21.bpo-46564.6Xc2_H.rst new file mode 100644 index 0000000000000..4ffa6800989d7 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-10-23-21.bpo-46564.6Xc2_H.rst @@ -0,0 +1 @@ +Do not create frame objects when creating :class:`super` object. Patch by Kumar Aditya. \ No newline at end of file diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 621ad9745d844..f7e0775e2225b 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -9012,7 +9012,7 @@ super_descr_get(PyObject *self, PyObject *obj, PyObject *type) } static int -super_init_without_args(PyFrameObject *f, PyCodeObject *co, +super_init_without_args(InterpreterFrame *cframe, PyCodeObject *co, PyTypeObject **type_p, PyObject **obj_p) { if (co->co_argcount == 0) { @@ -9021,13 +9021,13 @@ super_init_without_args(PyFrameObject *f, PyCodeObject *co, return -1; } - assert(f->f_frame->f_code->co_nlocalsplus > 0); - PyObject *firstarg = _PyFrame_GetLocalsArray(f->f_frame)[0]; + assert(cframe->f_code->co_nlocalsplus > 0); + PyObject *firstarg = _PyFrame_GetLocalsArray(cframe)[0]; // The first argument might be a cell. if (firstarg != NULL && (_PyLocals_GetKind(co->co_localspluskinds, 0) & CO_FAST_CELL)) { // "firstarg" is a cell here unless (very unlikely) super() // was called from the C-API before the first MAKE_CELL op. - if (f->f_frame->f_lasti >= 0) { + if (cframe->f_lasti >= 0) { assert(_Py_OPCODE(*co->co_firstinstr) == MAKE_CELL || _Py_OPCODE(*co->co_firstinstr) == COPY_FREE_VARS); assert(PyCell_Check(firstarg)); firstarg = PyCell_GET(firstarg); @@ -9047,7 +9047,7 @@ super_init_without_args(PyFrameObject *f, PyCodeObject *co, PyObject *name = PyTuple_GET_ITEM(co->co_localsplusnames, i); assert(PyUnicode_Check(name)); if (_PyUnicode_EqualToASCIIId(name, &PyId___class__)) { - PyObject *cell = _PyFrame_GetLocalsArray(f->f_frame)[i]; + PyObject *cell = _PyFrame_GetLocalsArray(cframe)[i]; if (cell == NULL || !PyCell_Check(cell)) { PyErr_SetString(PyExc_RuntimeError, "super(): bad __class__ cell"); @@ -9096,17 +9096,13 @@ super_init(PyObject *self, PyObject *args, PyObject *kwds) /* Call super(), without args -- fill in from __class__ and first local variable on the stack. */ PyThreadState *tstate = _PyThreadState_GET(); - PyFrameObject *frame = PyThreadState_GetFrame(tstate); - if (frame == NULL) { + InterpreterFrame *cframe = tstate->cframe->current_frame; + if (cframe == NULL) { PyErr_SetString(PyExc_RuntimeError, "super(): no current frame"); return -1; } - - PyCodeObject *code = PyFrame_GetCode(frame); - int res = super_init_without_args(frame, code, &type, &obj); - Py_DECREF(frame); - Py_DECREF(code); + int res = super_init_without_args(cframe, cframe->f_code, &type, &obj); if (res < 0) { return -1; From webhook-mailer at python.org Tue Feb 1 08:47:26 2022 From: webhook-mailer at python.org (vstinner) Date: Tue, 01 Feb 2022 13:47:26 -0000 Subject: [Python-checkins] bpo-46600: ./configure --with-pydebug uses -Og with clang (GH-31052) Message-ID: https://github.com/python/cpython/commit/0515eafe55ce7699e3bbc3c1555f08073d43b790 commit: 0515eafe55ce7699e3bbc3c1555f08073d43b790 branch: main author: Victor Stinner committer: vstinner date: 2022-02-01T14:47:12+01:00 summary: bpo-46600: ./configure --with-pydebug uses -Og with clang (GH-31052) Fix the test checking if the C compiler supports -Og option in the ./configure script to also use -Og on clang which supports it. files: A Misc/NEWS.d/next/Build/2022-02-01-14-07-37.bpo-46600.NNLnfj.rst M configure M configure.ac diff --git a/Misc/NEWS.d/next/Build/2022-02-01-14-07-37.bpo-46600.NNLnfj.rst b/Misc/NEWS.d/next/Build/2022-02-01-14-07-37.bpo-46600.NNLnfj.rst new file mode 100644 index 0000000000000..1fab655f9b271 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2022-02-01-14-07-37.bpo-46600.NNLnfj.rst @@ -0,0 +1,3 @@ +Fix the test checking if the C compiler supports ``-Og`` option in the +``./configure`` script to also use ``-Og`` on clang which supports it. Patch +by Victor Stinner. diff --git a/configure b/configure index da4af32cc17aa..015b51645f5be 100755 --- a/configure +++ b/configure @@ -7623,6 +7623,50 @@ case $CC in fi esac +# Check if CC supports -Og optimization level +save_CFLAGS=$CFLAGS +CFLAGS="-Og" +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking if $CC supports -Og optimization level" >&5 +$as_echo_n "checking if $CC supports -Og optimization level... " >&6; } +if ${ac_cv_cc_supports_og+:} false; then : + $as_echo_n "(cached) " >&6 +else + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + + +int +main () +{ + + ; + return 0; +} + +_ACEOF +if ac_fn_c_try_compile "$LINENO"; then : + + ac_cv_cc_supports_og=yes + +else + + ac_cv_cc_supports_og=no + +fi +rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext + +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_cc_supports_og" >&5 +$as_echo "$ac_cv_cc_supports_og" >&6; } +CFLAGS=$save_CFLAGS + +# Optimization messes up debuggers, so turn it off for +# debug builds. +PYDEBUG_CFLAGS="-O0" +if test "x$ac_cv_cc_supports_og" = xyes; then : + PYDEBUG_CFLAGS="-Og" +fi + # tweak OPT based on compiler and platform, only if the user didn't set # it on the command line @@ -7648,13 +7692,7 @@ then case $ac_cv_prog_cc_g in yes) if test "$Py_DEBUG" = 'true' ; then - # Optimization messes up debuggers, so turn it off for - # debug builds. - if "$CC" -v --help 2>/dev/null |grep -- -Og > /dev/null; then - OPT="-g -Og -Wall" - else - OPT="-g -O0 -Wall" - fi + OPT="-g $PYDEBUG_CFLAGS -Wall" else OPT="-g $WRAP -O3 -Wall" fi diff --git a/configure.ac b/configure.ac index 5a076646b96dd..6204747edfc25 100644 --- a/configure.ac +++ b/configure.ac @@ -1791,6 +1791,28 @@ case $CC in fi esac +# Check if CC supports -Og optimization level +_SAVE_VAR([CFLAGS]) +CFLAGS="-Og" +AC_CACHE_CHECK([if $CC supports -Og optimization level], + [ac_cv_cc_supports_og], + AC_COMPILE_IFELSE( + [ + AC_LANG_PROGRAM([[]], [[]]) + ],[ + ac_cv_cc_supports_og=yes + ],[ + ac_cv_cc_supports_og=no + ]) +) +_RESTORE_VAR([CFLAGS]) + +# Optimization messes up debuggers, so turn it off for +# debug builds. +PYDEBUG_CFLAGS="-O0" +AS_VAR_IF([ac_cv_cc_supports_og], [yes], + [PYDEBUG_CFLAGS="-Og"]) + # tweak OPT based on compiler and platform, only if the user didn't set # it on the command line AC_SUBST(OPT) @@ -1816,13 +1838,7 @@ then case $ac_cv_prog_cc_g in yes) if test "$Py_DEBUG" = 'true' ; then - # Optimization messes up debuggers, so turn it off for - # debug builds. - if "$CC" -v --help 2>/dev/null |grep -- -Og > /dev/null; then - OPT="-g -Og -Wall" - else - OPT="-g -O0 -Wall" - fi + OPT="-g $PYDEBUG_CFLAGS -Wall" else OPT="-g $WRAP -O3 -Wall" fi From webhook-mailer at python.org Tue Feb 1 08:56:34 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 13:56:34 -0000 Subject: [Python-checkins] bpo-46584: remove check for `py2.3` from `ctypes/test_python_api` (GH-31024) Message-ID: https://github.com/python/cpython/commit/913e340a323c7e61ae6e4acbb1312b4342657bec commit: 913e340a323c7e61ae6e4acbb1312b4342657bec branch: main author: Nikita Sobolev committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T05:56:25-08:00 summary: bpo-46584: remove check for `py2.3` from `ctypes/test_python_api` (GH-31024) files: M Lib/ctypes/test/test_python_api.py diff --git a/Lib/ctypes/test/test_python_api.py b/Lib/ctypes/test/test_python_api.py index 9c137469c3b5a..49571f97bbe15 100644 --- a/Lib/ctypes/test/test_python_api.py +++ b/Lib/ctypes/test/test_python_api.py @@ -1,5 +1,5 @@ from ctypes import * -import unittest, sys +import unittest from test import support ################################################################ @@ -10,10 +10,6 @@ ################################################################ from sys import getrefcount as grc -if sys.version_info > (2, 4): - c_py_ssize_t = c_size_t -else: - c_py_ssize_t = c_int class PythonAPITestCase(unittest.TestCase): @@ -21,7 +17,7 @@ def test_PyBytes_FromStringAndSize(self): PyBytes_FromStringAndSize = pythonapi.PyBytes_FromStringAndSize PyBytes_FromStringAndSize.restype = py_object - PyBytes_FromStringAndSize.argtypes = c_char_p, c_py_ssize_t + PyBytes_FromStringAndSize.argtypes = c_char_p, c_size_t self.assertEqual(PyBytes_FromStringAndSize(b"abcdefghi", 3), b"abc") From webhook-mailer at python.org Tue Feb 1 09:42:46 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 14:42:46 -0000 Subject: [Python-checkins] [3.10] bpo-46584: remove check for `py2.3` from `ctypes/test_python_api` (GH-31024) (GH-31054) Message-ID: https://github.com/python/cpython/commit/8765b01bcf6514602affcca66e7deeeb998f9cef commit: 8765b01bcf6514602affcca66e7deeeb998f9cef branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T06:42:37-08:00 summary: [3.10] bpo-46584: remove check for `py2.3` from `ctypes/test_python_api` (GH-31024) (GH-31054) (cherry picked from commit 913e340a323c7e61ae6e4acbb1312b4342657bec) Co-authored-by: Nikita Sobolev files: M Lib/ctypes/test/test_python_api.py diff --git a/Lib/ctypes/test/test_python_api.py b/Lib/ctypes/test/test_python_api.py index 9c137469c3b5a..49571f97bbe15 100644 --- a/Lib/ctypes/test/test_python_api.py +++ b/Lib/ctypes/test/test_python_api.py @@ -1,5 +1,5 @@ from ctypes import * -import unittest, sys +import unittest from test import support ################################################################ @@ -10,10 +10,6 @@ ################################################################ from sys import getrefcount as grc -if sys.version_info > (2, 4): - c_py_ssize_t = c_size_t -else: - c_py_ssize_t = c_int class PythonAPITestCase(unittest.TestCase): @@ -21,7 +17,7 @@ def test_PyBytes_FromStringAndSize(self): PyBytes_FromStringAndSize = pythonapi.PyBytes_FromStringAndSize PyBytes_FromStringAndSize.restype = py_object - PyBytes_FromStringAndSize.argtypes = c_char_p, c_py_ssize_t + PyBytes_FromStringAndSize.argtypes = c_char_p, c_size_t self.assertEqual(PyBytes_FromStringAndSize(b"abcdefghi", 3), b"abc") From webhook-mailer at python.org Tue Feb 1 09:42:54 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 14:42:54 -0000 Subject: [Python-checkins] [3.9] bpo-46584: remove check for `py2.3` from `ctypes/test_python_api` (GH-31024) (GH-31053) Message-ID: https://github.com/python/cpython/commit/7368ca1b889d7fa16423e69034bf4c5fe7b00268 commit: 7368ca1b889d7fa16423e69034bf4c5fe7b00268 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T06:42:49-08:00 summary: [3.9] bpo-46584: remove check for `py2.3` from `ctypes/test_python_api` (GH-31024) (GH-31053) (cherry picked from commit 913e340a323c7e61ae6e4acbb1312b4342657bec) Co-authored-by: Nikita Sobolev files: M Lib/ctypes/test/test_python_api.py diff --git a/Lib/ctypes/test/test_python_api.py b/Lib/ctypes/test/test_python_api.py index 9c137469c3b5a..49571f97bbe15 100644 --- a/Lib/ctypes/test/test_python_api.py +++ b/Lib/ctypes/test/test_python_api.py @@ -1,5 +1,5 @@ from ctypes import * -import unittest, sys +import unittest from test import support ################################################################ @@ -10,10 +10,6 @@ ################################################################ from sys import getrefcount as grc -if sys.version_info > (2, 4): - c_py_ssize_t = c_size_t -else: - c_py_ssize_t = c_int class PythonAPITestCase(unittest.TestCase): @@ -21,7 +17,7 @@ def test_PyBytes_FromStringAndSize(self): PyBytes_FromStringAndSize = pythonapi.PyBytes_FromStringAndSize PyBytes_FromStringAndSize.restype = py_object - PyBytes_FromStringAndSize.argtypes = c_char_p, c_py_ssize_t + PyBytes_FromStringAndSize.argtypes = c_char_p, c_size_t self.assertEqual(PyBytes_FromStringAndSize(b"abcdefghi", 3), b"abc") From webhook-mailer at python.org Tue Feb 1 10:05:40 2022 From: webhook-mailer at python.org (markshannon) Date: Tue, 01 Feb 2022 15:05:40 -0000 Subject: [Python-checkins] bpo-46072: Add some object layout and allocation stats (GH-31051) Message-ID: https://github.com/python/cpython/commit/48be46ec1f3f8010570165daa1da4bf9961f3a83 commit: 48be46ec1f3f8010570165daa1da4bf9961f3a83 branch: main author: Mark Shannon committer: markshannon date: 2022-02-01T15:05:18Z summary: bpo-46072: Add some object layout and allocation stats (GH-31051) files: M Include/internal/pycore_code.h M Objects/dictobject.c M Objects/obmalloc.c M Python/specialize.c M Tools/scripts/summarize_stats.py diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index 68b536f75ca5e..45c7752112b46 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -305,9 +305,19 @@ typedef struct _call_stats { uint64_t pyeval_calls; } CallStats; +typedef struct _object_stats { + uint64_t allocations; + uint64_t frees; + uint64_t new_values; + uint64_t dict_materialized_on_request; + uint64_t dict_materialized_new_key; + uint64_t dict_materialized_too_big; +} ObjectStats; + typedef struct _stats { OpcodeStats opcode_stats[256]; CallStats call_stats; + ObjectStats object_stats; } PyStats; extern PyStats _py_stats; @@ -316,6 +326,7 @@ extern PyStats _py_stats; #define STAT_DEC(opname, name) _py_stats.opcode_stats[opname].specialization.name-- #define OPCODE_EXE_INC(opname) _py_stats.opcode_stats[opname].execution_count++ #define CALL_STAT_INC(name) _py_stats.call_stats.name++ +#define OBJECT_STAT_INC(name) _py_stats.object_stats.name++ void _Py_PrintSpecializationStats(int to_file); @@ -326,6 +337,7 @@ PyAPI_FUNC(PyObject*) _Py_GetSpecializationStats(void); #define STAT_DEC(opname, name) ((void)0) #define OPCODE_EXE_INC(opname) ((void)0) #define CALL_STAT_INC(name) ((void)0) +#define OBJECT_STAT_INC(name) ((void)0) #endif diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 39be189e12000..0ad0f0b59c87e 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -114,6 +114,7 @@ As a consequence of this, split keys have a maximum size of 16. #include "Python.h" #include "pycore_bitutils.h" // _Py_bit_length #include "pycore_call.h" // _PyObject_CallNoArgs() +#include "pycore_code.h" // stats #include "pycore_dict.h" // PyDictKeysObject #include "pycore_gc.h" // _PyObject_GC_IS_TRACKED() #include "pycore_object.h" // _PyObject_GC_TRACK() @@ -4990,6 +4991,7 @@ _PyObject_InitializeDict(PyObject *obj) return 0; } if (tp->tp_flags & Py_TPFLAGS_MANAGED_DICT) { + OBJECT_STAT_INC(new_values); return init_inline_values(obj, tp); } PyObject *dict; @@ -5033,6 +5035,7 @@ _PyObject_MakeDictFromInstanceAttributes(PyObject *obj, PyDictValues *values) { assert(Py_TYPE(obj)->tp_flags & Py_TPFLAGS_MANAGED_DICT); PyDictKeysObject *keys = CACHED_KEYS(Py_TYPE(obj)); + OBJECT_STAT_INC(dict_materialized_on_request); return make_dict_from_instance_attributes(keys, values); } @@ -5051,6 +5054,14 @@ _PyObject_StoreInstanceAttribute(PyObject *obj, PyDictValues *values, PyErr_SetObject(PyExc_AttributeError, name); return -1; } +#ifdef Py_STATS + if (shared_keys_usable_size(keys) > 14) { + OBJECT_STAT_INC(dict_materialized_too_big); + } + else { + OBJECT_STAT_INC(dict_materialized_new_key); + } +#endif PyObject *dict = make_dict_from_instance_attributes(keys, values); if (dict == NULL) { return -1; @@ -5183,6 +5194,7 @@ PyObject_GenericGetDict(PyObject *obj, void *context) PyObject **dictptr = _PyObject_ManagedDictPointer(obj); if (*values_ptr) { assert(*dictptr == NULL); + OBJECT_STAT_INC(dict_materialized_on_request); *dictptr = dict = make_dict_from_instance_attributes(CACHED_KEYS(tp), *values_ptr); if (dict != NULL) { *values_ptr = NULL; diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index e3df7e8cc410e..bad4dc0963921 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -1,5 +1,6 @@ #include "Python.h" #include "pycore_pymem.h" // _PyTraceMalloc_Config +#include "pycore_code.h" // stats #include #include // malloc() @@ -695,6 +696,7 @@ PyObject_Malloc(size_t size) /* see PyMem_RawMalloc() */ if (size > (size_t)PY_SSIZE_T_MAX) return NULL; + OBJECT_STAT_INC(allocations); return _PyObject.malloc(_PyObject.ctx, size); } @@ -704,6 +706,7 @@ PyObject_Calloc(size_t nelem, size_t elsize) /* see PyMem_RawMalloc() */ if (elsize != 0 && nelem > (size_t)PY_SSIZE_T_MAX / elsize) return NULL; + OBJECT_STAT_INC(allocations); return _PyObject.calloc(_PyObject.ctx, nelem, elsize); } @@ -719,6 +722,7 @@ PyObject_Realloc(void *ptr, size_t new_size) void PyObject_Free(void *ptr) { + OBJECT_STAT_INC(frees); _PyObject.free(_PyObject.ctx, ptr); } diff --git a/Python/specialize.c b/Python/specialize.c index aec94d9e60be4..5771a41dcfd2c 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -171,10 +171,22 @@ print_call_stats(FILE *out, CallStats *stats) fprintf(out, "Calls to Python functions inlined: %" PRIu64 "\n", stats->inlined_py_calls); } +static void +print_object_stats(FILE *out, ObjectStats *stats) +{ + fprintf(out, "Object allocations: %" PRIu64 "\n", stats->allocations); + fprintf(out, "Object frees: %" PRIu64 "\n", stats->frees); + fprintf(out, "Object new values: %" PRIu64 "\n", stats->new_values); + fprintf(out, "Object materialize dict (on request): %" PRIu64 "\n", stats->dict_materialized_on_request); + fprintf(out, "Object materialize dict (new key): %" PRIu64 "\n", stats->dict_materialized_new_key); + fprintf(out, "Object materialize dict (too big): %" PRIu64 "\n", stats->dict_materialized_too_big); +} + static void print_stats(FILE *out, PyStats *stats) { print_spec_stats(out, stats->opcode_stats); print_call_stats(out, &stats->call_stats); + print_object_stats(out, &stats->object_stats); } void diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index 34cbad5fa7ea2..2761dcb50985b 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -105,7 +105,17 @@ def main(): total += value for key, value in stats.items(): if "Calls to" in key: - print(f"{key}: {value} {100*value/total:0.1f}%") + print(f" {key}: {value} {100*value/total:0.1f}%") + print("Object stats:") + total = stats.get("Object new values") + for key, value in stats.items(): + if key.startswith("Object"): + if "materialize" in key: + print(f" {key}: {value} {100*value/total:0.1f}%") + else: + print(f" {key}: {value}") + total = 0 + if __name__ == "__main__": main() From webhook-mailer at python.org Tue Feb 1 10:58:48 2022 From: webhook-mailer at python.org (ned-deily) Date: Tue, 01 Feb 2022 15:58:48 -0000 Subject: [Python-checkins] bpo-45925: Update macOS installer to SQLite 3.37.2 (GH-30921) Message-ID: https://github.com/python/cpython/commit/0e4bef7a7f6f25a6f39755778c73e7026901611f commit: 0e4bef7a7f6f25a6f39755778c73e7026901611f branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: ned-deily date: 2022-02-01T10:58:29-05:00 summary: bpo-45925: Update macOS installer to SQLite 3.37.2 (GH-30921) Co-authored-by: Erlend Egeberg Aasland files: A Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst M Mac/BuildScript/build-installer.py diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index 0f8924bb72740..7e89a3bc82ae2 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -358,9 +358,9 @@ def library_recipes(): ), ), dict( - name="SQLite 3.36.0", - url="https://sqlite.org/2021/sqlite-autoconf-3360000.tar.gz", - checksum='f5752052fc5b8e1b539af86a3671eac7', + name="SQLite 3.37.2", + url="https://sqlite.org/2022/sqlite-autoconf-3370200.tar.gz", + checksum='683cc5312ee74e71079c14d24b7a6d27', extra_cflags=('-Os ' '-DSQLITE_ENABLE_FTS5 ' '-DSQLITE_ENABLE_FTS4 ' diff --git a/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst b/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst new file mode 100644 index 0000000000000..3705266c154b7 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst @@ -0,0 +1 @@ +Update macOS installer to SQLite 3.37.2. \ No newline at end of file From webhook-mailer at python.org Tue Feb 1 11:05:11 2022 From: webhook-mailer at python.org (asvetlov) Date: Tue, 01 Feb 2022 16:05:11 -0000 Subject: [Python-checkins] bpo-46487: Add `get_write_buffer_limits` to Write and _SSLProtocol transports (GH-30958) Message-ID: https://github.com/python/cpython/commit/64568acbd88a88d54ac9b8215447f88280448dd5 commit: 64568acbd88a88d54ac9b8215447f88280448dd5 branch: main author: Emiya committer: asvetlov date: 2022-02-01T18:05:02+02:00 summary: bpo-46487: Add `get_write_buffer_limits` to Write and _SSLProtocol transports (GH-30958) Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst M Lib/asyncio/sslproto.py M Lib/asyncio/transports.py diff --git a/Lib/asyncio/sslproto.py b/Lib/asyncio/sslproto.py index cad25b26539f5..00fc16c014c9b 100644 --- a/Lib/asyncio/sslproto.py +++ b/Lib/asyncio/sslproto.py @@ -367,6 +367,12 @@ def get_write_buffer_size(self): """Return the current size of the write buffer.""" return self._ssl_protocol._transport.get_write_buffer_size() + def get_write_buffer_limits(self): + """Get the high and low watermarks for write flow control. + Return a tuple (low, high) where low and high are + positive number of bytes.""" + return self._ssl_protocol._transport.get_write_buffer_limits() + @property def _protocol_paused(self): # Required for sendfile fallback pause_writing/resume_writing logic diff --git a/Lib/asyncio/transports.py b/Lib/asyncio/transports.py index 45e155c94cad1..73b1fa2de416c 100644 --- a/Lib/asyncio/transports.py +++ b/Lib/asyncio/transports.py @@ -99,6 +99,12 @@ def get_write_buffer_size(self): """Return the current size of the write buffer.""" raise NotImplementedError + def get_write_buffer_limits(self): + """Get the high and low watermarks for write flow control. + Return a tuple (low, high) where low and high are + positive number of bytes.""" + raise NotImplementedError + def write(self, data): """Write some data bytes to the transport. diff --git a/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst b/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst new file mode 100644 index 0000000000000..adbc50a4bf9b7 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst @@ -0,0 +1 @@ +Add the ``get_write_buffer_limits`` method to :class:`asyncio.transports.WriteTransport` and to the SSL transport. From webhook-mailer at python.org Tue Feb 1 11:29:11 2022 From: webhook-mailer at python.org (ned-deily) Date: Tue, 01 Feb 2022 16:29:11 -0000 Subject: [Python-checkins] bpo-45925: Update macOS installer to SQLite 3.37.2 (GH-30921) (GH-31057) Message-ID: https://github.com/python/cpython/commit/519eb6ad74f946a9aa7676e2d6579a3a765a8b50 commit: 519eb6ad74f946a9aa7676e2d6579a3a765a8b50 branch: 3.10 author: Ned Deily committer: ned-deily date: 2022-02-01T11:29:02-05:00 summary: bpo-45925: Update macOS installer to SQLite 3.37.2 (GH-30921) (GH-31057) Co-authored-by: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> Co-authored-by: Erlend Egeberg Aasland files: A Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst M Mac/BuildScript/build-installer.py diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index 1eef40fc940f4..6720c0ba1785e 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -358,9 +358,9 @@ def library_recipes(): ), ), dict( - name="SQLite 3.35.5", - url="https://sqlite.org/2021/sqlite-autoconf-3350500.tar.gz", - checksum='d1d1aba394c8e0443077dc9f1a681bb8', + name="SQLite 3.37.2", + url="https://sqlite.org/2022/sqlite-autoconf-3370200.tar.gz", + checksum='683cc5312ee74e71079c14d24b7a6d27', extra_cflags=('-Os ' '-DSQLITE_ENABLE_FTS5 ' '-DSQLITE_ENABLE_FTS4 ' diff --git a/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst b/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst new file mode 100644 index 0000000000000..3705266c154b7 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst @@ -0,0 +1 @@ +Update macOS installer to SQLite 3.37.2. \ No newline at end of file From webhook-mailer at python.org Tue Feb 1 12:02:42 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 17:02:42 -0000 Subject: [Python-checkins] bpo-45953: Preserve backward compatibility on some PyThreadState field names. (GH-31038) Message-ID: https://github.com/python/cpython/commit/f78be59c83c151d94902daef56218530c52e29e7 commit: f78be59c83c151d94902daef56218530c52e29e7 branch: main author: Eric Snow committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T09:02:25-08:00 summary: bpo-45953: Preserve backward compatibility on some PyThreadState field names. (GH-31038) The gevent project is using the two `PyThreadState` fields I renamed in gh-30590. This PR fixes the names. See #msg412046. files: M Include/cpython/pystate.h M Python/pystate.c diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index a35e5b803bd08..74dd44d6edccb 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -187,16 +187,19 @@ struct _ts { /* The following fields are here to avoid allocation during init. The data is exposed through PyThreadState pointer fields. These fields should not be accessed directly outside of init. + This is indicated by an underscore prefix on the field names. All other PyInterpreterState pointer fields are populated when needed and default to NULL. */ + // Note some fields do not have a leading underscore for backward + // compatibility. See https://bugs.python.org/issue45953#msg412046. /* The thread's exception stack entry. (Always the last entry.) */ - _PyErr_StackItem _exc_state; + _PyErr_StackItem exc_state; /* The bottom-most frame on the stack. */ - CFrame _root_cframe; + CFrame root_cframe; }; diff --git a/Python/pystate.c b/Python/pystate.c index 4b698f2b1d771..4378d78a8b781 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -776,9 +776,9 @@ init_threadstate(PyThreadState *tstate, tstate->recursion_limit = interp->ceval.recursion_limit, tstate->recursion_remaining = interp->ceval.recursion_limit, - tstate->exc_info = &tstate->_exc_state; + tstate->exc_info = &tstate->exc_state; - tstate->cframe = &tstate->_root_cframe; + tstate->cframe = &tstate->root_cframe; tstate->datastack_chunk = NULL; tstate->datastack_top = NULL; tstate->datastack_limit = NULL; @@ -1016,10 +1016,10 @@ PyThreadState_Clear(PyThreadState *tstate) Py_CLEAR(tstate->curexc_value); Py_CLEAR(tstate->curexc_traceback); - Py_CLEAR(tstate->_exc_state.exc_value); + Py_CLEAR(tstate->exc_state.exc_value); /* The stack of exception states should contain just this thread. */ - if (verbose && tstate->exc_info != &tstate->_exc_state) { + if (verbose && tstate->exc_info != &tstate->exc_state) { fprintf(stderr, "PyThreadState_Clear: warning: thread still has a generator\n"); } From webhook-mailer at python.org Tue Feb 1 12:12:36 2022 From: webhook-mailer at python.org (vstinner) Date: Tue, 01 Feb 2022 17:12:36 -0000 Subject: [Python-checkins] bpo-46600: Fix test_gdb.test_pycfunction() for clang -Og (GH-31058) Message-ID: https://github.com/python/cpython/commit/bebaa95fd0f44babf8b6bcffd8f2908c73ca259e commit: bebaa95fd0f44babf8b6bcffd8f2908c73ca259e branch: main author: Victor Stinner committer: vstinner date: 2022-02-01T18:12:26+01:00 summary: bpo-46600: Fix test_gdb.test_pycfunction() for clang -Og (GH-31058) Fix test_gdb.test_pycfunction() for Python built with clang -Og. Tolerate inlined functions in the gdb traceback. When _testcapimodule.c is built by clang -Og, _null_to_none() is inlined in meth_varargs() and so gdb returns _null_to_none() as the frame #1. If it's not inlined, meth_varargs() is the frame #1. files: A Misc/NEWS.d/next/Tests/2022-02-01-17-13-53.bpo-46600.FMCk8Z.rst M Lib/test/test_gdb.py diff --git a/Lib/test/test_gdb.py b/Lib/test/test_gdb.py index eaeb6fb8ff568..344fd3dd3f7fc 100644 --- a/Lib/test/test_gdb.py +++ b/Lib/test/test_gdb.py @@ -897,15 +897,19 @@ def test_gc(self): # to suppress these. See also the comment in DebuggerTests.get_stack_trace def test_pycfunction(self): 'Verify that "py-bt" displays invocations of PyCFunction instances' + # bpo-46600: If the compiler inlines _null_to_none() in meth_varargs() + # (ex: clang -Og), _null_to_none() is the frame #1. Otherwise, + # meth_varargs() is the frame #1. + expected_frame = r'#(1|2)' # Various optimizations multiply the code paths by which these are # called, so test a variety of calling conventions. - for func_name, args, expected_frame in ( - ('meth_varargs', '', 1), - ('meth_varargs_keywords', '', 1), - ('meth_o', '[]', 1), - ('meth_noargs', '', 1), - ('meth_fastcall', '', 1), - ('meth_fastcall_keywords', '', 1), + for func_name, args in ( + ('meth_varargs', ''), + ('meth_varargs_keywords', ''), + ('meth_o', '[]'), + ('meth_noargs', ''), + ('meth_fastcall', ''), + ('meth_fastcall_keywords', ''), ): for obj in ( '_testcapi', @@ -945,10 +949,9 @@ def bar(): # defined.' message in stderr. ignore_stderr=True, ) - self.assertIn( - f'#{expected_frame} https://github.com/python/cpython/commit/a22dd00f2da4f6ff96f58fd8d551f16bdb870c8d commit: a22dd00f2da4f6ff96f58fd8d551f16bdb870c8d branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ned-deily date: 2022-02-01T12:12:59-05:00 summary: bpo-45925: Update macOS installer to SQLite 3.37.2 (GH-30921) (GH-31057) (GH-31059) Co-authored-by: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> Co-authored-by: Erlend Egeberg Aasland (cherry picked from commit 519eb6ad74f946a9aa7676e2d6579a3a765a8b50) files: A Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst M Mac/BuildScript/build-installer.py diff --git a/Mac/BuildScript/build-installer.py b/Mac/BuildScript/build-installer.py index 3f5acfdf56648..47a01056f2e5c 100755 --- a/Mac/BuildScript/build-installer.py +++ b/Mac/BuildScript/build-installer.py @@ -356,9 +356,9 @@ def library_recipes(): ), ), dict( - name="SQLite 3.35.5", - url="https://sqlite.org/2021/sqlite-autoconf-3350500.tar.gz", - checksum='d1d1aba394c8e0443077dc9f1a681bb8', + name="SQLite 3.37.2", + url="https://sqlite.org/2022/sqlite-autoconf-3370200.tar.gz", + checksum='683cc5312ee74e71079c14d24b7a6d27', extra_cflags=('-Os ' '-DSQLITE_ENABLE_FTS5 ' '-DSQLITE_ENABLE_FTS4 ' diff --git a/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst b/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst new file mode 100644 index 0000000000000..3705266c154b7 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst @@ -0,0 +1 @@ +Update macOS installer to SQLite 3.37.2. \ No newline at end of file From webhook-mailer at python.org Tue Feb 1 14:29:58 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 19:29:58 -0000 Subject: [Python-checkins] bpo-46487: Add `get_write_buffer_limits` to Write and _SSLProtocol transports (GH-30958) Message-ID: https://github.com/python/cpython/commit/e4a6e549027b33bbe87f49fcfccc880243e49834 commit: e4a6e549027b33bbe87f49fcfccc880243e49834 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T11:29:45-08:00 summary: bpo-46487: Add `get_write_buffer_limits` to Write and _SSLProtocol transports (GH-30958) Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> (cherry picked from commit 64568acbd88a88d54ac9b8215447f88280448dd5) Co-authored-by: Emiya files: A Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst M Lib/asyncio/sslproto.py M Lib/asyncio/transports.py diff --git a/Lib/asyncio/sslproto.py b/Lib/asyncio/sslproto.py index cad25b26539f5..00fc16c014c9b 100644 --- a/Lib/asyncio/sslproto.py +++ b/Lib/asyncio/sslproto.py @@ -367,6 +367,12 @@ def get_write_buffer_size(self): """Return the current size of the write buffer.""" return self._ssl_protocol._transport.get_write_buffer_size() + def get_write_buffer_limits(self): + """Get the high and low watermarks for write flow control. + Return a tuple (low, high) where low and high are + positive number of bytes.""" + return self._ssl_protocol._transport.get_write_buffer_limits() + @property def _protocol_paused(self): # Required for sendfile fallback pause_writing/resume_writing logic diff --git a/Lib/asyncio/transports.py b/Lib/asyncio/transports.py index 45e155c94cad1..73b1fa2de416c 100644 --- a/Lib/asyncio/transports.py +++ b/Lib/asyncio/transports.py @@ -99,6 +99,12 @@ def get_write_buffer_size(self): """Return the current size of the write buffer.""" raise NotImplementedError + def get_write_buffer_limits(self): + """Get the high and low watermarks for write flow control. + Return a tuple (low, high) where low and high are + positive number of bytes.""" + raise NotImplementedError + def write(self, data): """Write some data bytes to the transport. diff --git a/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst b/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst new file mode 100644 index 0000000000000..adbc50a4bf9b7 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst @@ -0,0 +1 @@ +Add the ``get_write_buffer_limits`` method to :class:`asyncio.transports.WriteTransport` and to the SSL transport. From webhook-mailer at python.org Tue Feb 1 14:30:08 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 19:30:08 -0000 Subject: [Python-checkins] bpo-46487: Add `get_write_buffer_limits` to Write and _SSLProtocol transports (GH-30958) Message-ID: https://github.com/python/cpython/commit/e5e1441d41907f92cc3bb5de675a2c519068173d commit: e5e1441d41907f92cc3bb5de675a2c519068173d branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T11:30:03-08:00 summary: bpo-46487: Add `get_write_buffer_limits` to Write and _SSLProtocol transports (GH-30958) Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> (cherry picked from commit 64568acbd88a88d54ac9b8215447f88280448dd5) Co-authored-by: Emiya files: A Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst M Lib/asyncio/sslproto.py M Lib/asyncio/transports.py diff --git a/Lib/asyncio/sslproto.py b/Lib/asyncio/sslproto.py index cad25b26539f5..00fc16c014c9b 100644 --- a/Lib/asyncio/sslproto.py +++ b/Lib/asyncio/sslproto.py @@ -367,6 +367,12 @@ def get_write_buffer_size(self): """Return the current size of the write buffer.""" return self._ssl_protocol._transport.get_write_buffer_size() + def get_write_buffer_limits(self): + """Get the high and low watermarks for write flow control. + Return a tuple (low, high) where low and high are + positive number of bytes.""" + return self._ssl_protocol._transport.get_write_buffer_limits() + @property def _protocol_paused(self): # Required for sendfile fallback pause_writing/resume_writing logic diff --git a/Lib/asyncio/transports.py b/Lib/asyncio/transports.py index 45e155c94cad1..73b1fa2de416c 100644 --- a/Lib/asyncio/transports.py +++ b/Lib/asyncio/transports.py @@ -99,6 +99,12 @@ def get_write_buffer_size(self): """Return the current size of the write buffer.""" raise NotImplementedError + def get_write_buffer_limits(self): + """Get the high and low watermarks for write flow control. + Return a tuple (low, high) where low and high are + positive number of bytes.""" + raise NotImplementedError + def write(self, data): """Write some data bytes to the transport. diff --git a/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst b/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst new file mode 100644 index 0000000000000..adbc50a4bf9b7 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst @@ -0,0 +1 @@ +Add the ``get_write_buffer_limits`` method to :class:`asyncio.transports.WriteTransport` and to the SSL transport. From webhook-mailer at python.org Tue Feb 1 16:41:43 2022 From: webhook-mailer at python.org (brandtbucher) Date: Tue, 01 Feb 2022 21:41:43 -0000 Subject: [Python-checkins] bpo-46528: Simplify BUILD_TUPLE/UNPACK_SEQUENCE folding (GH-31039) Message-ID: https://github.com/python/cpython/commit/a0e55a571cf01885fd5826266c37abaee307c309 commit: a0e55a571cf01885fd5826266c37abaee307c309 branch: main author: Brandt Bucher committer: brandtbucher date: 2022-02-01T13:41:32-08:00 summary: bpo-46528: Simplify BUILD_TUPLE/UNPACK_SEQUENCE folding (GH-31039) files: M Lib/test/test_peepholer.py M Python/compile.c diff --git a/Lib/test/test_peepholer.py b/Lib/test/test_peepholer.py index 659f654b5c676..2df5883ce7918 100644 --- a/Lib/test/test_peepholer.py +++ b/Lib/test/test_peepholer.py @@ -126,7 +126,7 @@ def test_pack_unpack(self): code = compile(line,'','single') self.assertInBytecode(code, elem) self.assertNotInBytecode(code, 'BUILD_TUPLE') - self.assertNotInBytecode(code, 'UNPACK_TUPLE') + self.assertNotInBytecode(code, 'UNPACK_SEQUENCE') self.check_lnotab(code) def test_folding_of_tuples_of_constants(self): diff --git a/Python/compile.c b/Python/compile.c index eda708e21f16a..1d2d567e85e93 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8661,29 +8661,22 @@ optimize_basic_block(struct compiler *c, basicblock *bb, PyObject *consts) } /* Try to fold tuples of constants. - Skip over BUILD_SEQN 1 UNPACK_SEQN 1. - Replace BUILD_SEQN 2 UNPACK_SEQN 2 with ROT2. - Replace BUILD_SEQN 3 UNPACK_SEQN 3 with ROT3 ROT2. */ + Skip over BUILD_TUPLE(1) UNPACK_SEQUENCE(1). + Replace BUILD_TUPLE(2) UNPACK_SEQUENCE(2) with SWAP(2). + Replace BUILD_TUPLE(3) UNPACK_SEQUENCE(3) with SWAP(3). */ case BUILD_TUPLE: if (nextop == UNPACK_SEQUENCE && oparg == bb->b_instr[i+1].i_oparg) { switch(oparg) { case 1: inst->i_opcode = NOP; bb->b_instr[i+1].i_opcode = NOP; - break; + continue; case 2: - inst->i_opcode = SWAP; - inst->i_oparg = 2; - bb->b_instr[i+1].i_opcode = NOP; - i--; - break; case 3: - inst->i_opcode = SWAP; - inst->i_oparg = 3; - bb->b_instr[i+1].i_opcode = NOP; - i--; + inst->i_opcode = NOP; + bb->b_instr[i+1].i_opcode = SWAP; + continue; } - break; } if (i >= oparg) { if (fold_tuple_on_constants(c, inst-oparg, oparg, consts)) { From webhook-mailer at python.org Tue Feb 1 17:34:02 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Feb 2022 22:34:02 -0000 Subject: [Python-checkins] bpo-14916: use specified tokenizer fd for file input (GH-31006) Message-ID: https://github.com/python/cpython/commit/89b13042fcfc95bae21a49806a205ef62f1cdd73 commit: 89b13042fcfc95bae21a49806a205ef62f1cdd73 branch: main author: Paul m. p. P committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T14:33:52-08:00 summary: bpo-14916: use specified tokenizer fd for file input (GH-31006) @pablogsal, sorry i failed to rebase to main, so i recreated https://github.com/python/cpython/pull/22190#issuecomment-1024633392 > PyRun_InteractiveOne\*() functions allow to explicitily set fd instead of stdin. but stdin was hardcoded in readline call. > This patch does not fix target file for prompt unlike original bpo one : prompt fd is unrelated to tokenizer source which could be read only. It is more of a bugfix regarding the docs : actual documentation say "prompt the user" so one would expect prompt to go on stdout not a file for both PyRun_InteractiveOne\*() and PyRun_InteractiveLoop\*(). Automerge-Triggered-By: GH:pablogsal files: A Misc/NEWS.d/next/C API/2020-09-11-02-50-41.bpo-14916.QN1Y03.rst M Parser/tokenizer.c diff --git a/Misc/NEWS.d/next/C API/2020-09-11-02-50-41.bpo-14916.QN1Y03.rst b/Misc/NEWS.d/next/C API/2020-09-11-02-50-41.bpo-14916.QN1Y03.rst new file mode 100644 index 0000000000000..885cfc53aba38 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2020-09-11-02-50-41.bpo-14916.QN1Y03.rst @@ -0,0 +1 @@ +Fixed bug in the tokenizer that prevented ``PyRun_InteractiveOne`` from parsing from the provided FD. diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index cd4254f8b9077..5b5cbdb809ebe 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -855,7 +855,7 @@ tok_underflow_interactive(struct tok_state *tok) { tok->done = E_INTERACT_STOP; return 1; } - char *newtok = PyOS_Readline(stdin, stdout, tok->prompt); + char *newtok = PyOS_Readline(tok->fp ? tok->fp : stdin, stdout, tok->prompt); if (newtok != NULL) { char *translated = translate_newlines(newtok, 0, tok); PyMem_Free(newtok); From webhook-mailer at python.org Tue Feb 1 20:20:33 2022 From: webhook-mailer at python.org (terryjreedy) Date: Wed, 02 Feb 2022 01:20:33 -0000 Subject: [Python-checkins] bpo-46591: Make About IDLE doc link label clickable (GH-30251) Message-ID: https://github.com/python/cpython/commit/53c78080573b3bae4c4e782b9f47dce48aac9688 commit: 53c78080573b3bae4c4e782b9f47dce48aac9688 branch: main author: Wes <5124946+wesinator at users.noreply.github.com> committer: terryjreedy date: 2022-02-01T20:20:23-05:00 summary: bpo-46591: Make About IDLE doc link label clickable (GH-30251) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/help_about.py M Lib/idlelib/idle_test/htest.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index ed2b05ed662e8..34b2c08a92256 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,6 +4,8 @@ Released on 2022-10-03 ========================= +bpo-46591: Make the IDLE doc URL on the About IDLE dialog clickable. + bpo-45296: Clarify close, quit, and exit in IDLE. In the File menu, 'Close' and 'Exit' are now 'Close Window' (the current one) and 'Exit' is now 'Exit IDLE' (by closing all windows). In Shell, 'quit()' and diff --git a/Lib/idlelib/help_about.py b/Lib/idlelib/help_about.py index 019aacbd0faa2..c59f494599806 100644 --- a/Lib/idlelib/help_about.py +++ b/Lib/idlelib/help_about.py @@ -3,6 +3,7 @@ """ import os import sys +import webbrowser from platform import python_version, architecture from tkinter import Toplevel, Frame, Label, Button, PhotoImage @@ -94,6 +95,7 @@ def create_widgets(self): f"{version[:version.rindex('.')]}/library/idle.html", justify=LEFT, fg=self.fg, bg=self.bg) docs.grid(row=7, column=0, columnspan=2, sticky=W, padx=10, pady=0) + docs.bind("", lambda event: webbrowser.open(docs['text'])) Frame(frame_background, borderwidth=1, relief=SUNKEN, height=2, bg=self.bg).grid(row=8, column=0, sticky=EW, diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py index 666ff4cb84851..d297f8aa0094e 100644 --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -77,8 +77,8 @@ def _wrapper(parent): # htest # 'kwds': {'title': 'help_about test', '_htest': True, }, - 'msg': "Test every button. Ensure Python, TK and IDLE versions " - "are correctly displayed.\n [Close] to exit.", + 'msg': "Click on URL to open in default browser.\n" + "Verify x.y.z versions and test each button, including Close.\n " } # TODO implement ^\; adding '' to function does not work. diff --git a/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst b/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst new file mode 100644 index 0000000000000..7785faa1c4cbf --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst @@ -0,0 +1 @@ +Make the IDLE doc URL on the About IDLE dialog clickable. \ No newline at end of file From webhook-mailer at python.org Tue Feb 1 20:47:18 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 01:47:18 -0000 Subject: [Python-checkins] bpo-46591: Make About IDLE doc link label clickable (GH-30251) Message-ID: https://github.com/python/cpython/commit/a1869e385b17bfdfe8d9850047f5b2dbb88e58fa commit: a1869e385b17bfdfe8d9850047f5b2dbb88e58fa branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T17:47:09-08:00 summary: bpo-46591: Make About IDLE doc link label clickable (GH-30251) Co-authored-by: Terry Jan Reedy (cherry picked from commit 53c78080573b3bae4c4e782b9f47dce48aac9688) Co-authored-by: Wes <5124946+wesinator at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/help_about.py M Lib/idlelib/idle_test/htest.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 175c907951034..2ba8d581bbc73 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,6 +4,8 @@ Released on 2022-05-16 ========================= +bpo-46591: Make the IDLE doc URL on the About IDLE dialog clickable. + bpo-45296: Clarify close, quit, and exit in IDLE. In the File menu, 'Close' and 'Exit' are now 'Close Window' (the current one) and 'Exit' is now 'Exit IDLE' (by closing all windows). In Shell, 'quit()' and diff --git a/Lib/idlelib/help_about.py b/Lib/idlelib/help_about.py index 019aacbd0faa2..c59f494599806 100644 --- a/Lib/idlelib/help_about.py +++ b/Lib/idlelib/help_about.py @@ -3,6 +3,7 @@ """ import os import sys +import webbrowser from platform import python_version, architecture from tkinter import Toplevel, Frame, Label, Button, PhotoImage @@ -94,6 +95,7 @@ def create_widgets(self): f"{version[:version.rindex('.')]}/library/idle.html", justify=LEFT, fg=self.fg, bg=self.bg) docs.grid(row=7, column=0, columnspan=2, sticky=W, padx=10, pady=0) + docs.bind("", lambda event: webbrowser.open(docs['text'])) Frame(frame_background, borderwidth=1, relief=SUNKEN, height=2, bg=self.bg).grid(row=8, column=0, sticky=EW, diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py index 666ff4cb84851..d297f8aa0094e 100644 --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -77,8 +77,8 @@ def _wrapper(parent): # htest # 'kwds': {'title': 'help_about test', '_htest': True, }, - 'msg': "Test every button. Ensure Python, TK and IDLE versions " - "are correctly displayed.\n [Close] to exit.", + 'msg': "Click on URL to open in default browser.\n" + "Verify x.y.z versions and test each button, including Close.\n " } # TODO implement ^\; adding '' to function does not work. diff --git a/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst b/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst new file mode 100644 index 0000000000000..7785faa1c4cbf --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst @@ -0,0 +1 @@ +Make the IDLE doc URL on the About IDLE dialog clickable. \ No newline at end of file From webhook-mailer at python.org Tue Feb 1 20:47:49 2022 From: webhook-mailer at python.org (vstinner) Date: Wed, 02 Feb 2022 01:47:49 -0000 Subject: [Python-checkins] bpo-44359: Fix test_ftplib unhandled thread exceptions (GH-31069) Message-ID: https://github.com/python/cpython/commit/0611eafc709cbe8a2a0bdde082d25df0c5034de7 commit: 0611eafc709cbe8a2a0bdde082d25df0c5034de7 branch: main author: Victor Stinner committer: vstinner date: 2022-02-02T02:47:40+01:00 summary: bpo-44359: Fix test_ftplib unhandled thread exceptions (GH-31069) test_ftplib now silently ignores socket errors to prevent logging unhandled threading exceptions. files: A Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst M Lib/test/test_ftplib.py diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py index 56e3d8ab8528a..2f5cc06ca4b9a 100644 --- a/Lib/test/test_ftplib.py +++ b/Lib/test/test_ftplib.py @@ -56,6 +56,13 @@ "type=file;perm=r;unique==SGP2; file \xAE non-ascii char\r\n") +def default_error_handler(): + # bpo-44359: Silently ignore socket errors. Such errors occur when a client + # socket is closed, in TestFTPClass.tearDown() and makepasv() tests, and + # the server gets an error on its side. + pass + + class DummyDTPHandler(asynchat.async_chat): dtp_conn_closed = False @@ -87,7 +94,7 @@ def push(self, what): super(DummyDTPHandler, self).push(what.encode(self.encoding)) def handle_error(self): - raise Exception + default_error_handler() class DummyFTPHandler(asynchat.async_chat): @@ -137,7 +144,7 @@ def found_terminator(self): self.push('550 command "%s" not understood.' %cmd) def handle_error(self): - raise Exception + default_error_handler() def push(self, data): asynchat.async_chat.push(self, data.encode(self.encoding) + b'\r\n') @@ -315,7 +322,7 @@ def writable(self): return 0 def handle_error(self): - raise Exception + default_error_handler() if ssl is not None: @@ -418,7 +425,7 @@ def recv(self, buffer_size): raise def handle_error(self): - raise Exception + default_error_handler() def close(self): if (isinstance(self.socket, ssl.SSLSocket) and diff --git a/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst b/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst new file mode 100644 index 0000000000000..00c29b1ca4089 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst @@ -0,0 +1,2 @@ +test_ftplib now silently ignores socket errors to prevent logging unhandled +threading exceptions. Patch by Victor Stinner. From webhook-mailer at python.org Tue Feb 1 20:47:55 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 01:47:55 -0000 Subject: [Python-checkins] bpo-46591: Make About IDLE doc link label clickable (GH-30251) Message-ID: https://github.com/python/cpython/commit/7dee93c2d223a080f410f6901987e9d363743471 commit: 7dee93c2d223a080f410f6901987e9d363743471 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T17:47:51-08:00 summary: bpo-46591: Make About IDLE doc link label clickable (GH-30251) Co-authored-by: Terry Jan Reedy (cherry picked from commit 53c78080573b3bae4c4e782b9f47dce48aac9688) Co-authored-by: Wes <5124946+wesinator at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/help_about.py M Lib/idlelib/idle_test/htest.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index bb78289ba8261..724709a29ff63 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,6 +4,8 @@ Released on 2022-05-16 ========================= +bpo-46591: Make the IDLE doc URL on the About IDLE dialog clickable. + bpo-45296: Clarify close, quit, and exit in IDLE. In the File menu, 'Close' and 'Exit' are now 'Close Window' (the current one) and 'Exit' is now 'Exit IDLE' (by closing all windows). In Shell, 'quit()' and diff --git a/Lib/idlelib/help_about.py b/Lib/idlelib/help_about.py index 019aacbd0faa2..c59f494599806 100644 --- a/Lib/idlelib/help_about.py +++ b/Lib/idlelib/help_about.py @@ -3,6 +3,7 @@ """ import os import sys +import webbrowser from platform import python_version, architecture from tkinter import Toplevel, Frame, Label, Button, PhotoImage @@ -94,6 +95,7 @@ def create_widgets(self): f"{version[:version.rindex('.')]}/library/idle.html", justify=LEFT, fg=self.fg, bg=self.bg) docs.grid(row=7, column=0, columnspan=2, sticky=W, padx=10, pady=0) + docs.bind("", lambda event: webbrowser.open(docs['text'])) Frame(frame_background, borderwidth=1, relief=SUNKEN, height=2, bg=self.bg).grid(row=8, column=0, sticky=EW, diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py index 666ff4cb84851..d297f8aa0094e 100644 --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -77,8 +77,8 @@ def _wrapper(parent): # htest # 'kwds': {'title': 'help_about test', '_htest': True, }, - 'msg': "Test every button. Ensure Python, TK and IDLE versions " - "are correctly displayed.\n [Close] to exit.", + 'msg': "Click on URL to open in default browser.\n" + "Verify x.y.z versions and test each button, including Close.\n " } # TODO implement ^\; adding '' to function does not work. diff --git a/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst b/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst new file mode 100644 index 0000000000000..7785faa1c4cbf --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst @@ -0,0 +1 @@ +Make the IDLE doc URL on the About IDLE dialog clickable. \ No newline at end of file From webhook-mailer at python.org Tue Feb 1 21:12:35 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 02:12:35 -0000 Subject: [Python-checkins] bpo-44359: Fix test_ftplib unhandled thread exceptions (GH-31069) Message-ID: https://github.com/python/cpython/commit/85b421fbff23ff41c52fa5bde191920b3701f0d0 commit: 85b421fbff23ff41c52fa5bde191920b3701f0d0 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T18:12:27-08:00 summary: bpo-44359: Fix test_ftplib unhandled thread exceptions (GH-31069) test_ftplib now silently ignores socket errors to prevent logging unhandled threading exceptions. (cherry picked from commit 0611eafc709cbe8a2a0bdde082d25df0c5034de7) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst M Lib/test/test_ftplib.py diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py index 56e3d8ab8528a..2f5cc06ca4b9a 100644 --- a/Lib/test/test_ftplib.py +++ b/Lib/test/test_ftplib.py @@ -56,6 +56,13 @@ "type=file;perm=r;unique==SGP2; file \xAE non-ascii char\r\n") +def default_error_handler(): + # bpo-44359: Silently ignore socket errors. Such errors occur when a client + # socket is closed, in TestFTPClass.tearDown() and makepasv() tests, and + # the server gets an error on its side. + pass + + class DummyDTPHandler(asynchat.async_chat): dtp_conn_closed = False @@ -87,7 +94,7 @@ def push(self, what): super(DummyDTPHandler, self).push(what.encode(self.encoding)) def handle_error(self): - raise Exception + default_error_handler() class DummyFTPHandler(asynchat.async_chat): @@ -137,7 +144,7 @@ def found_terminator(self): self.push('550 command "%s" not understood.' %cmd) def handle_error(self): - raise Exception + default_error_handler() def push(self, data): asynchat.async_chat.push(self, data.encode(self.encoding) + b'\r\n') @@ -315,7 +322,7 @@ def writable(self): return 0 def handle_error(self): - raise Exception + default_error_handler() if ssl is not None: @@ -418,7 +425,7 @@ def recv(self, buffer_size): raise def handle_error(self): - raise Exception + default_error_handler() def close(self): if (isinstance(self.socket, ssl.SSLSocket) and diff --git a/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst b/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst new file mode 100644 index 0000000000000..00c29b1ca4089 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst @@ -0,0 +1,2 @@ +test_ftplib now silently ignores socket errors to prevent logging unhandled +threading exceptions. Patch by Victor Stinner. From webhook-mailer at python.org Tue Feb 1 21:17:16 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 02:17:16 -0000 Subject: [Python-checkins] bpo-44359: Fix test_ftplib unhandled thread exceptions (GH-31069) Message-ID: https://github.com/python/cpython/commit/0371e5d7f1248abb0712fc98c3135d01d265cec8 commit: 0371e5d7f1248abb0712fc98c3135d01d265cec8 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-01T18:16:58-08:00 summary: bpo-44359: Fix test_ftplib unhandled thread exceptions (GH-31069) test_ftplib now silently ignores socket errors to prevent logging unhandled threading exceptions. (cherry picked from commit 0611eafc709cbe8a2a0bdde082d25df0c5034de7) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst M Lib/test/test_ftplib.py diff --git a/Lib/test/test_ftplib.py b/Lib/test/test_ftplib.py index 6f8f977b3950d..b51e5440800cd 100644 --- a/Lib/test/test_ftplib.py +++ b/Lib/test/test_ftplib.py @@ -49,6 +49,13 @@ "type=file;perm=r;unique==SGP2; file \xAE non-ascii char\r\n") +def default_error_handler(): + # bpo-44359: Silently ignore socket errors. Such errors occur when a client + # socket is closed, in TestFTPClass.tearDown() and makepasv() tests, and + # the server gets an error on its side. + pass + + class DummyDTPHandler(asynchat.async_chat): dtp_conn_closed = False @@ -80,7 +87,7 @@ def push(self, what): super(DummyDTPHandler, self).push(what.encode(self.encoding)) def handle_error(self): - raise Exception + default_error_handler() class DummyFTPHandler(asynchat.async_chat): @@ -130,7 +137,7 @@ def found_terminator(self): self.push('550 command "%s" not understood.' %cmd) def handle_error(self): - raise Exception + default_error_handler() def push(self, data): asynchat.async_chat.push(self, data.encode(self.encoding) + b'\r\n') @@ -308,7 +315,7 @@ def writable(self): return 0 def handle_error(self): - raise Exception + default_error_handler() if ssl is not None: @@ -411,7 +418,7 @@ def recv(self, buffer_size): raise def handle_error(self): - raise Exception + default_error_handler() def close(self): if (isinstance(self.socket, ssl.SSLSocket) and diff --git a/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst b/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst new file mode 100644 index 0000000000000..00c29b1ca4089 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst @@ -0,0 +1,2 @@ +test_ftplib now silently ignores socket errors to prevent logging unhandled +threading exceptions. Patch by Victor Stinner. From webhook-mailer at python.org Tue Feb 1 21:38:23 2022 From: webhook-mailer at python.org (ned-deily) Date: Wed, 02 Feb 2022 02:38:23 -0000 Subject: [Python-checkins] bpo-46602: Do not append conftest.c (GH-31062) Message-ID: https://github.com/python/cpython/commit/b1288964e31069bdf81abe560c82874f6f620928 commit: b1288964e31069bdf81abe560c82874f6f620928 branch: main author: adanhawth <36526095+adanhawth at users.noreply.github.com> committer: ned-deily date: 2022-02-01T21:38:15-05:00 summary: bpo-46602: Do not append conftest.c (GH-31062) The heredoc creation statements use >> to append conftest.c. This can cause tricky build issues if the file is not correctly removed prior to its name being reused (such name is reused several times for different contextual tests during the build). One such result from appending may cause #include to persist when testing to acquire PLATFORM_TRIPLET. This can then lead to downstream issues concerning SOABI. files: A Misc/NEWS.d/next/Build/2022-02-02-02-06-07.bpo-46602.8GaOZ2.rst M configure M configure.ac diff --git a/Misc/NEWS.d/next/Build/2022-02-02-02-06-07.bpo-46602.8GaOZ2.rst b/Misc/NEWS.d/next/Build/2022-02-02-02-06-07.bpo-46602.8GaOZ2.rst new file mode 100644 index 0000000000000..a1123b44ecd59 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2022-02-02-02-06-07.bpo-46602.8GaOZ2.rst @@ -0,0 +1 @@ +Tidied up configure.ac so that conftest.c is truncated rather than appended. This assists in the case where the 'rm' of conftest.c fails to happen between tests. Downstream issues such as a clobbered SOABI can result. \ No newline at end of file diff --git a/configure b/configure index 015b51645f5be..9097c0514b57c 100755 --- a/configure +++ b/configure @@ -5942,7 +5942,7 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for the platform triplet based on compiler characteristics" >&5 $as_echo_n "checking for the platform triplet based on compiler characteristics... " >&6; } -cat >> conftest.c < conftest.c <&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking for the Android API level" >&5 $as_echo_n "checking for the Android API level... " >&6; } -cat >> conftest.c < conftest.c <> conftest.c < conftest.c <> conftest.c < conftest.c < https://github.com/python/cpython/commit/abcc3d75f6e570519cb37c62130a2295c6928bff commit: abcc3d75f6e570519cb37c62130a2295c6928bff branch: main author: Jelle Zijlstra committer: gvanrossum date: 2022-02-01T18:48:55-08:00 summary: bpo-46414: Add typing.reveal_type (#30646) Co-authored-by: Nikita Sobolev files: A Misc/NEWS.d/next/Library/2022-01-17-10-00-02.bpo-46414.Ld0b_y.rst M Doc/library/typing.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index cdfd403a34ef9..9007c0daf59a4 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1932,6 +1932,37 @@ Functions and decorators runtime we intentionally don't check anything (we want this to be as fast as possible). +.. function:: reveal_type(obj) + + Reveal the inferred static type of an expression. + + When a static type checker encounters a call to this function, + it emits a diagnostic with the type of the argument. For example:: + + x: int = 1 + reveal_type(x) # Revealed type is "builtins.int" + + This can be useful when you want to debug how your type checker + handles a particular piece of code. + + The function returns its argument unchanged, which allows using + it within an expression:: + + x = reveal_type(1) # Revealed type is "builtins.int" + + Most type checkers support ``reveal_type()`` anywhere, even if the + name is not imported from ``typing``. Importing the name from + ``typing`` allows your code to run without runtime errors and + communicates intent more clearly. + + At runtime, this function prints the runtime type of its argument to stderr + and returns it unchanged:: + + x = reveal_type(1) # prints "Runtime type is int" + print(x) # prints "1" + + .. versionadded:: 3.11 + .. decorator:: overload The ``@overload`` decorator allows describing functions and methods diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 090d4c70d3d84..85f74064458f2 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -20,6 +20,7 @@ from typing import get_type_hints from typing import get_origin, get_args from typing import is_typeddict +from typing import reveal_type from typing import no_type_check, no_type_check_decorator from typing import Type from typing import NamedTuple, TypedDict @@ -34,7 +35,7 @@ import weakref import types -from test.support import import_helper +from test.support import import_helper, captured_stderr from test import mod_generics_cache from test import _typed_dict_helper @@ -5289,6 +5290,14 @@ def bar(self): self.assertIn('baz', dir(Foo[int])) +class RevealTypeTests(BaseTestCase): + def test_reveal_type(self): + obj = object() + with captured_stderr() as stderr: + self.assertIs(obj, reveal_type(obj)) + self.assertEqual(stderr.getvalue(), "Runtime type is 'object'\n") + + class AllTests(BaseTestCase): """Tests for __all__.""" diff --git a/Lib/typing.py b/Lib/typing.py index dac9c6c4f87cf..0cf9755022e9b 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -130,6 +130,7 @@ def _idfunc(_, x): 'overload', 'ParamSpecArgs', 'ParamSpecKwargs', + 'reveal_type', 'runtime_checkable', 'Text', 'TYPE_CHECKING', @@ -2675,3 +2676,23 @@ class re(metaclass=_DeprecatedType): re.__name__ = __name__ + '.re' sys.modules[re.__name__] = re + + +def reveal_type(obj: T, /) -> T: + """Reveal the inferred type of a variable. + + When a static type checker encounters a call to ``reveal_type()``, + it will emit the inferred type of the argument:: + + x: int = 1 + reveal_type(x) + + Running a static type checker (e.g., ``mypy``) on this example + will produce output similar to 'Revealed type is "builtins.int"'. + + At runtime, the function prints the runtime type of the + argument and returns it unchanged. + + """ + print(f"Runtime type is {type(obj).__name__!r}", file=sys.stderr) + return obj diff --git a/Misc/NEWS.d/next/Library/2022-01-17-10-00-02.bpo-46414.Ld0b_y.rst b/Misc/NEWS.d/next/Library/2022-01-17-10-00-02.bpo-46414.Ld0b_y.rst new file mode 100644 index 0000000000000..0fdbfa77d9bc8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-17-10-00-02.bpo-46414.Ld0b_y.rst @@ -0,0 +1 @@ +Add :func:`typing.reveal_type`. Patch by Jelle Zijlstra. From webhook-mailer at python.org Tue Feb 1 23:18:25 2022 From: webhook-mailer at python.org (rhettinger) Date: Wed, 02 Feb 2022 04:18:25 -0000 Subject: [Python-checkins] Fix minor details in the Counter docs (GH-31029) Message-ID: https://github.com/python/cpython/commit/f77beacf015e9bf8762e9b9a9a631396b05d4d9a commit: f77beacf015e9bf8762e9b9a9a631396b05d4d9a branch: main author: Raymond Hettinger committer: rhettinger date: 2022-02-01T22:18:11-06:00 summary: Fix minor details in the Counter docs (GH-31029) files: M Doc/library/collections.rst M Lib/collections/__init__.py diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index b8a717d883c09..b97bc425de9eb 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -271,7 +271,7 @@ For example:: .. versionadded:: 3.1 .. versionchanged:: 3.7 As a :class:`dict` subclass, :class:`Counter` - Inherited the capability to remember insertion order. Math operations + inherited the capability to remember insertion order. Math operations on *Counter* objects also preserve order. Results are ordered according to when an element is first encountered in the left operand and then by the order encountered in the right operand. @@ -366,19 +366,26 @@ Several mathematical operations are provided for combining :class:`Counter` objects to produce multisets (counters that have counts greater than zero). Addition and subtraction combine counters by adding or subtracting the counts of corresponding elements. Intersection and union return the minimum and -maximum of corresponding counts. Each operation can accept inputs with signed +maximum of corresponding counts. Equality and inclusion compare +corresponding counts. Each operation can accept inputs with signed counts, but the output will exclude results with counts of zero or less. +.. doctest:: + >>> c = Counter(a=3, b=1) >>> d = Counter(a=1, b=2) >>> c + d # add two counters together: c[x] + d[x] Counter({'a': 4, 'b': 3}) >>> c - d # subtract (keeping only positive counts) Counter({'a': 2}) - >>> c & d # intersection: min(c[x], d[x]) # doctest: +SKIP + >>> c & d # intersection: min(c[x], d[x]) Counter({'a': 1, 'b': 1}) >>> c | d # union: max(c[x], d[x]) Counter({'a': 3, 'b': 2}) + >>> c == d # equality: c[x] == d[x] + False + >>> c <= d # inclusion: c[x] <= d[x] + False Unary addition and subtraction are shortcuts for adding an empty counter or subtracting from an empty counter. diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py index fa8b30985a435..264435cb41b2f 100644 --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -736,6 +736,10 @@ def __repr__(self): # To strip negative and zero counts, add-in an empty counter: # c += Counter() # + # Results are ordered according to when an element is first + # encountered in the left operand and then by the order + # encountered in the right operand. + # # When the multiplicities are all zero or one, multiset operations # are guaranteed to be equivalent to the corresponding operations # for regular sets. From webhook-mailer at python.org Tue Feb 1 23:18:56 2022 From: webhook-mailer at python.org (rhettinger) Date: Wed, 02 Feb 2022 04:18:56 -0000 Subject: [Python-checkins] Add recipe for subslices (GH-31028) Message-ID: https://github.com/python/cpython/commit/06a491179f2de106d34c6b4973a32dce08fc4247 commit: 06a491179f2de106d34c6b4973a32dce08fc4247 branch: main author: Raymond Hettinger committer: rhettinger date: 2022-02-01T22:18:52-06:00 summary: Add recipe for subslices (GH-31028) files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index f0d93ebb6b21a..8c3b7842f7f6a 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -893,6 +893,12 @@ which incur interpreter overhead. yield from it return true_iterator(), remainder_iterator() + def subslices(seq): + "Return all contiguous non-empty subslices of a sequence" + # subslices('ABCD') --> A AB ABC ABCD B BC BCD C CD D + slices = starmap(slice, combinations(range(len(seq) + 1), 2)) + return map(operator.getitem, repeat(seq), slices) + def powerset(iterable): "powerset([1,2,3]) --> () (1,) (2,) (3,) (1,2) (1,3) (2,3) (1,2,3)" s = list(iterable) @@ -1188,6 +1194,9 @@ which incur interpreter overhead. >>> ''.join(remainder) 'dEfGhI' + >>> list(subslices('ABCD')) + ['A', 'AB', 'ABC', 'ABCD', 'B', 'BC', 'BCD', 'C', 'CD', 'D'] + >>> list(powerset([1,2,3])) [(), (1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)] From webhook-mailer at python.org Wed Feb 2 00:28:29 2022 From: webhook-mailer at python.org (rhettinger) Date: Wed, 02 Feb 2022 05:28:29 -0000 Subject: [Python-checkins] Fix minor details in the Counter docs (GH-31029) (GH-31072) Message-ID: https://github.com/python/cpython/commit/e480def027b68a570dc0c2e235463842373dc741 commit: e480def027b68a570dc0c2e235463842373dc741 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: rhettinger date: 2022-02-01T23:28:07-06:00 summary: Fix minor details in the Counter docs (GH-31029) (GH-31072) files: M Doc/library/collections.rst M Lib/collections/__init__.py diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index b8a717d883c09..b97bc425de9eb 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -271,7 +271,7 @@ For example:: .. versionadded:: 3.1 .. versionchanged:: 3.7 As a :class:`dict` subclass, :class:`Counter` - Inherited the capability to remember insertion order. Math operations + inherited the capability to remember insertion order. Math operations on *Counter* objects also preserve order. Results are ordered according to when an element is first encountered in the left operand and then by the order encountered in the right operand. @@ -366,19 +366,26 @@ Several mathematical operations are provided for combining :class:`Counter` objects to produce multisets (counters that have counts greater than zero). Addition and subtraction combine counters by adding or subtracting the counts of corresponding elements. Intersection and union return the minimum and -maximum of corresponding counts. Each operation can accept inputs with signed +maximum of corresponding counts. Equality and inclusion compare +corresponding counts. Each operation can accept inputs with signed counts, but the output will exclude results with counts of zero or less. +.. doctest:: + >>> c = Counter(a=3, b=1) >>> d = Counter(a=1, b=2) >>> c + d # add two counters together: c[x] + d[x] Counter({'a': 4, 'b': 3}) >>> c - d # subtract (keeping only positive counts) Counter({'a': 2}) - >>> c & d # intersection: min(c[x], d[x]) # doctest: +SKIP + >>> c & d # intersection: min(c[x], d[x]) Counter({'a': 1, 'b': 1}) >>> c | d # union: max(c[x], d[x]) Counter({'a': 3, 'b': 2}) + >>> c == d # equality: c[x] == d[x] + False + >>> c <= d # inclusion: c[x] <= d[x] + False Unary addition and subtraction are shortcuts for adding an empty counter or subtracting from an empty counter. diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py index bae0805d6686c..818588f740318 100644 --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -766,6 +766,10 @@ def __repr__(self): # To strip negative and zero counts, add-in an empty counter: # c += Counter() # + # Results are ordered according to when an element is first + # encountered in the left operand and then by the order + # encountered in the right operand. + # # When the multiplicities are all zero or one, multiset operations # are guaranteed to be equivalent to the corresponding operations # for regular sets. From webhook-mailer at python.org Wed Feb 2 03:15:30 2022 From: webhook-mailer at python.org (asvetlov) Date: Wed, 02 Feb 2022 08:15:30 -0000 Subject: [Python-checkins] bpo-46583: remove unused `sys.version_info` check from `selectors` (GH-31023) Message-ID: https://github.com/python/cpython/commit/3483aa65dfebfc60a87ea9db3f20da84be41f606 commit: 3483aa65dfebfc60a87ea9db3f20da84be41f606 branch: main author: Nikita Sobolev committer: asvetlov date: 2022-02-02T10:15:02+02:00 summary: bpo-46583: remove unused `sys.version_info` check from `selectors` (GH-31023) files: M Lib/selectors.py diff --git a/Lib/selectors.py b/Lib/selectors.py index bb15a1cb1bada..af6a4f94b5008 100644 --- a/Lib/selectors.py +++ b/Lib/selectors.py @@ -50,12 +50,11 @@ def _fileobj_to_fd(fileobj): Object used to associate a file object to its backing file descriptor, selected event mask, and attached data. """ -if sys.version_info >= (3, 5): - SelectorKey.fileobj.__doc__ = 'File object registered.' - SelectorKey.fd.__doc__ = 'Underlying file descriptor.' - SelectorKey.events.__doc__ = 'Events that must be waited for on this file object.' - SelectorKey.data.__doc__ = ('''Optional opaque data associated to this file object. - For example, this could be used to store a per-client session ID.''') +SelectorKey.fileobj.__doc__ = 'File object registered.' +SelectorKey.fd.__doc__ = 'Underlying file descriptor.' +SelectorKey.events.__doc__ = 'Events that must be waited for on this file object.' +SelectorKey.data.__doc__ = ('''Optional opaque data associated to this file object. +For example, this could be used to store a per-client session ID.''') class _SelectorMapping(Mapping): From webhook-mailer at python.org Wed Feb 2 03:16:44 2022 From: webhook-mailer at python.org (asvetlov) Date: Wed, 02 Feb 2022 08:16:44 -0000 Subject: [Python-checkins] Remove Python 3.3 compatibility code from overlapped.c (GH-31049) Message-ID: https://github.com/python/cpython/commit/a05866ce3e617e2b74c205f27a89eab63c7b3101 commit: a05866ce3e617e2b74c205f27a89eab63c7b3101 branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: asvetlov date: 2022-02-02T10:16:36+02:00 summary: Remove Python 3.3 compatibility code from overlapped.c (GH-31049) files: M Modules/overlapped.c diff --git a/Modules/overlapped.c b/Modules/overlapped.c index 7c4570896bc59..2ba48c8b845f5 100644 --- a/Modules/overlapped.c +++ b/Modules/overlapped.c @@ -23,12 +23,6 @@ # define T_POINTER T_ULONGLONG #endif -/* Compatibility with Python 3.3 */ -#if PY_VERSION_HEX < 0x03040000 -# define PyMem_RawMalloc PyMem_Malloc -# define PyMem_RawFree PyMem_Free -#endif - #define F_HANDLE F_POINTER #define F_ULONG_PTR F_POINTER #define F_DWORD "k" From webhook-mailer at python.org Wed Feb 2 06:02:01 2022 From: webhook-mailer at python.org (markshannon) Date: Wed, 02 Feb 2022 11:02:01 -0000 Subject: [Python-checkins] bpo-46072: Add some frame stats. (GH-31060) Message-ID: https://github.com/python/cpython/commit/187930f74c44e460ba09c60ba5d9bb4fac543d8f commit: 187930f74c44e460ba09c60ba5d9bb4fac543d8f branch: main author: Mark Shannon committer: markshannon date: 2022-02-02T11:01:33Z summary: bpo-46072: Add some frame stats. (GH-31060) files: M Include/internal/pycore_code.h M Objects/frameobject.c M Python/ceval.c M Python/frame.c M Python/pystate.c M Python/specialize.c M Tools/scripts/summarize_stats.py diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index 45c7752112b46..3897ea0ab6a1a 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -303,6 +303,8 @@ typedef struct _opcode_stats { typedef struct _call_stats { uint64_t inlined_py_calls; uint64_t pyeval_calls; + uint64_t frames_pushed; + uint64_t frame_objects_created; } CallStats; typedef struct _object_stats { diff --git a/Objects/frameobject.c b/Objects/frameobject.c index 81ad4cc65d150..15da1325d1480 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -794,6 +794,7 @@ init_frame(InterpreterFrame *frame, PyFunctionObject *func, PyObject *locals) PyFrameObject* _PyFrame_New_NoTrack(PyCodeObject *code) { + CALL_STAT_INC(frame_objects_created); int slots = code->co_nlocalsplus + code->co_stacksize; PyFrameObject *f = PyObject_GC_NewVar(PyFrameObject, &PyFrame_Type, slots); if (f == NULL) { diff --git a/Python/ceval.c b/Python/ceval.c index b69d5aa9d3206..70748e8911f9f 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2242,6 +2242,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (new_frame == NULL) { goto error; } + CALL_STAT_INC(frames_pushed); _PyFrame_InitializeSpecials(new_frame, getitem, NULL, code->co_nlocalsplus); STACK_SHRINK(2); @@ -4660,6 +4661,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (new_frame == NULL) { goto error; } + CALL_STAT_INC(inlined_py_calls); STACK_SHRINK(argcount); for (int i = 0; i < argcount; i++) { new_frame->localsplus[i] = stack_pointer[i]; @@ -4690,6 +4692,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (new_frame == NULL) { goto error; } + CALL_STAT_INC(inlined_py_calls); STACK_SHRINK(argcount); for (int i = 0; i < argcount; i++) { new_frame->localsplus[i] = stack_pointer[i]; @@ -4708,7 +4711,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr _PyFrame_SetStackPointer(frame, stack_pointer); new_frame->previous = frame; frame = cframe.current_frame = new_frame; - CALL_STAT_INC(inlined_py_calls); goto start_frame; } @@ -6078,6 +6080,7 @@ _PyEvalFramePushAndInit(PyThreadState *tstate, PyFunctionObject *func, { PyCodeObject * code = (PyCodeObject *)func->func_code; size_t size = code->co_nlocalsplus + code->co_stacksize + FRAME_SPECIALS_SIZE; + CALL_STAT_INC(frames_pushed); InterpreterFrame *frame = _PyThreadState_BumpFramePointer(tstate, size); if (frame == NULL) { goto fail; diff --git a/Python/frame.c b/Python/frame.c index 771de7583be79..ca7c5f9c94e07 100644 --- a/Python/frame.c +++ b/Python/frame.c @@ -1,6 +1,7 @@ #include "Python.h" #include "frameobject.h" +#include "pycore_code.h" // stats #include "pycore_frame.h" #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "opcode.h" @@ -113,6 +114,7 @@ _PyFrame_Push(PyThreadState *tstate, PyFunctionObject *func) { PyCodeObject *code = (PyCodeObject *)func->func_code; size_t size = code->co_nlocalsplus + code->co_stacksize + FRAME_SPECIALS_SIZE; + CALL_STAT_INC(frames_pushed); InterpreterFrame *new_frame = _PyThreadState_BumpFramePointer(tstate, size); if (new_frame == NULL) { return NULL; diff --git a/Python/pystate.c b/Python/pystate.c index 4378d78a8b781..77467944e2afb 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -3,6 +3,7 @@ #include "Python.h" #include "pycore_ceval.h" +#include "pycore_code.h" // stats #include "pycore_frame.h" #include "pycore_initconfig.h" #include "pycore_object.h" // _PyType_InitCache() @@ -2219,6 +2220,7 @@ _PyThreadState_PushFrame(PyThreadState *tstate, PyFunctionObject *func, PyObject int nlocalsplus = code->co_nlocalsplus; size_t size = nlocalsplus + code->co_stacksize + FRAME_SPECIALS_SIZE; + CALL_STAT_INC(frames_pushed); InterpreterFrame *frame = _PyThreadState_BumpFramePointer(tstate, size); if (frame == NULL) { return NULL; diff --git a/Python/specialize.c b/Python/specialize.c index 5771a41dcfd2c..9290fbe823956 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -169,6 +169,8 @@ print_call_stats(FILE *out, CallStats *stats) { fprintf(out, "Calls to PyEval_EvalDefault: %" PRIu64 "\n", stats->pyeval_calls); fprintf(out, "Calls to Python functions inlined: %" PRIu64 "\n", stats->inlined_py_calls); + fprintf(out, "Frames pushed: %" PRIu64 "\n", stats->frames_pushed); + fprintf(out, "Frame objects created: %" PRIu64 "\n", stats->frame_objects_created); } static void diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index 2761dcb50985b..319b251c854a8 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -106,6 +106,9 @@ def main(): for key, value in stats.items(): if "Calls to" in key: print(f" {key}: {value} {100*value/total:0.1f}%") + for key, value in stats.items(): + if key.startswith("Frame"): + print(f" {key}: {value} {100*value/total:0.1f}%") print("Object stats:") total = stats.get("Object new values") for key, value in stats.items(): From webhook-mailer at python.org Wed Feb 2 07:38:50 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 12:38:50 -0000 Subject: [Python-checkins] bpo-43012: remove `pathlib._Accessor` (GH-25701) Message-ID: https://github.com/python/cpython/commit/08f8301b21648d58d053e1a513db8ed32fbf37dd commit: 08f8301b21648d58d053e1a513db8ed32fbf37dd branch: main author: Barney Gale committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T04:38:25-08:00 summary: bpo-43012: remove `pathlib._Accessor` (GH-25701) Per Pitrou: > The original intent for the ?accessor? thing was to have a variant that did all accesses under a filesystem tree in a race condition-free way using openat and friends. It turned out to be much too hairy to actually implement, so was entirely abandoned, but the accessor abstraction was left there. https://discuss.python.org/t/make-pathlib-extensible/3428/2 Accessors are: - Lacking any internal purpose - '_NormalAccessor' is the only implementation - Lacking any firm conceptual difference to `Path` objects themselves (inc. subclasses) - Non-public, i.e. underscore prefixed - '_Accessor' and '_NormalAccessor' - Unofficially used to implement customized `Path` objects, but once once [bpo-24132]() is addressed there will be a supported route for that. This patch preserves all existing behaviour. files: A Misc/NEWS.d/next/Library/2022-01-05-03-09-29.bpo-43012.RVhLIL.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Lib/pathlib.py b/Lib/pathlib.py index 1603325180795..920e1f425a0c0 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -275,93 +275,6 @@ def make_uri(self, path): _posix_flavour = _PosixFlavour() -class _Accessor: - """An accessor implements a particular (system-specific or not) way of - accessing paths on the filesystem.""" - - -class _NormalAccessor(_Accessor): - - stat = os.stat - - open = io.open - - listdir = os.listdir - - scandir = os.scandir - - chmod = os.chmod - - mkdir = os.mkdir - - unlink = os.unlink - - if hasattr(os, "link"): - link = os.link - else: - def link(self, src, dst): - raise NotImplementedError("os.link() not available on this system") - - rmdir = os.rmdir - - rename = os.rename - - replace = os.replace - - if hasattr(os, "symlink"): - symlink = os.symlink - else: - def symlink(self, src, dst, target_is_directory=False): - raise NotImplementedError("os.symlink() not available on this system") - - def touch(self, path, mode=0o666, exist_ok=True): - if exist_ok: - # First try to bump modification time - # Implementation note: GNU touch uses the UTIME_NOW option of - # the utimensat() / futimens() functions. - try: - os.utime(path, None) - except OSError: - # Avoid exception chaining - pass - else: - return - flags = os.O_CREAT | os.O_WRONLY - if not exist_ok: - flags |= os.O_EXCL - fd = os.open(path, flags, mode) - os.close(fd) - - if hasattr(os, "readlink"): - readlink = os.readlink - else: - def readlink(self, path): - raise NotImplementedError("os.readlink() not available on this system") - - def owner(self, path): - try: - import pwd - return pwd.getpwuid(self.stat(path).st_uid).pw_name - except ImportError: - raise NotImplementedError("Path.owner() is unsupported on this system") - - def group(self, path): - try: - import grp - return grp.getgrgid(self.stat(path).st_gid).gr_name - except ImportError: - raise NotImplementedError("Path.group() is unsupported on this system") - - getcwd = os.getcwd - - expanduser = staticmethod(os.path.expanduser) - - realpath = staticmethod(os.path.realpath) - - -_normal_accessor = _NormalAccessor() - - # # Globbing helpers # @@ -402,7 +315,7 @@ def select_from(self, parent_path): path_cls = type(parent_path) is_dir = path_cls.is_dir exists = path_cls.exists - scandir = parent_path._accessor.scandir + scandir = path_cls._scandir if not is_dir(parent_path): return iter([]) return self._select_from(parent_path, is_dir, exists, scandir) @@ -949,7 +862,6 @@ class Path(PurePath): object. You can also instantiate a PosixPath or WindowsPath directly, but cannot instantiate a WindowsPath on a POSIX system or vice versa. """ - _accessor = _normal_accessor __slots__ = () def __new__(cls, *args, **kwargs): @@ -988,7 +900,7 @@ def cwd(cls): """Return a new path pointing to the current working directory (as returned by os.getcwd()). """ - return cls(cls._accessor.getcwd()) + return cls(os.getcwd()) @classmethod def home(cls): @@ -1005,16 +917,22 @@ def samefile(self, other_path): try: other_st = other_path.stat() except AttributeError: - other_st = self._accessor.stat(other_path) + other_st = self.__class__(other_path).stat() return os.path.samestat(st, other_st) def iterdir(self): """Iterate over the files in this directory. Does not yield any result for the special paths '.' and '..'. """ - for name in self._accessor.listdir(self): + for name in os.listdir(self): yield self._make_child_relpath(name) + def _scandir(self): + # bpo-24132: a future version of pathlib will support subclassing of + # pathlib.Path to customize how the filesystem is accessed. This + # includes scandir(), which is used to implement glob(). + return os.scandir(self) + def glob(self, pattern): """Iterate over this subtree and yield all existing files (of any kind, including directories) matching the given relative pattern. @@ -1050,7 +968,7 @@ def absolute(self): """ if self.is_absolute(): return self - return self._from_parts([self._accessor.getcwd()] + self._parts) + return self._from_parts([self.cwd()] + self._parts) def resolve(self, strict=False): """ @@ -1064,7 +982,7 @@ def check_eloop(e): raise RuntimeError("Symlink loop from %r" % e.filename) try: - s = self._accessor.realpath(self, strict=strict) + s = os.path.realpath(self, strict=strict) except OSError as e: check_eloop(e) raise @@ -1084,19 +1002,28 @@ def stat(self, *, follow_symlinks=True): Return the result of the stat() system call on this path, like os.stat() does. """ - return self._accessor.stat(self, follow_symlinks=follow_symlinks) + return os.stat(self, follow_symlinks=follow_symlinks) def owner(self): """ Return the login name of the file owner. """ - return self._accessor.owner(self) + try: + import pwd + return pwd.getpwuid(self.stat().st_uid).pw_name + except ImportError: + raise NotImplementedError("Path.owner() is unsupported on this system") def group(self): """ Return the group name of the file gid. """ - return self._accessor.group(self) + + try: + import grp + return grp.getgrgid(self.stat().st_gid).gr_name + except ImportError: + raise NotImplementedError("Path.group() is unsupported on this system") def open(self, mode='r', buffering=-1, encoding=None, errors=None, newline=None): @@ -1106,8 +1033,7 @@ def open(self, mode='r', buffering=-1, encoding=None, """ if "b" not in mode: encoding = io.text_encoding(encoding) - return self._accessor.open(self, mode, buffering, encoding, errors, - newline) + return io.open(self, mode, buffering, encoding, errors, newline) def read_bytes(self): """ @@ -1148,21 +1074,38 @@ def readlink(self): """ Return the path to which the symbolic link points. """ - path = self._accessor.readlink(self) - return self._from_parts((path,)) + if not hasattr(os, "readlink"): + raise NotImplementedError("os.readlink() not available on this system") + return self._from_parts((os.readlink(self),)) def touch(self, mode=0o666, exist_ok=True): """ Create this file with the given access mode, if it doesn't exist. """ - self._accessor.touch(self, mode, exist_ok) + + if exist_ok: + # First try to bump modification time + # Implementation note: GNU touch uses the UTIME_NOW option of + # the utimensat() / futimens() functions. + try: + os.utime(self, None) + except OSError: + # Avoid exception chaining + pass + else: + return + flags = os.O_CREAT | os.O_WRONLY + if not exist_ok: + flags |= os.O_EXCL + fd = os.open(self, flags, mode) + os.close(fd) def mkdir(self, mode=0o777, parents=False, exist_ok=False): """ Create a new directory at this given path. """ try: - self._accessor.mkdir(self, mode) + os.mkdir(self, mode) except FileNotFoundError: if not parents or self.parent == self: raise @@ -1178,7 +1121,7 @@ def chmod(self, mode, *, follow_symlinks=True): """ Change the permissions of the path, like os.chmod(). """ - self._accessor.chmod(self, mode, follow_symlinks=follow_symlinks) + os.chmod(self, mode, follow_symlinks=follow_symlinks) def lchmod(self, mode): """ @@ -1193,7 +1136,7 @@ def unlink(self, missing_ok=False): If the path is a directory, use rmdir() instead. """ try: - self._accessor.unlink(self) + os.unlink(self) except FileNotFoundError: if not missing_ok: raise @@ -1202,7 +1145,7 @@ def rmdir(self): """ Remove this directory. The directory must be empty. """ - self._accessor.rmdir(self) + os.rmdir(self) def lstat(self): """ @@ -1221,7 +1164,7 @@ def rename(self, target): Returns the new Path instance pointing to the target path. """ - self._accessor.rename(self, target) + os.rename(self, target) return self.__class__(target) def replace(self, target): @@ -1234,7 +1177,7 @@ def replace(self, target): Returns the new Path instance pointing to the target path. """ - self._accessor.replace(self, target) + os.replace(self, target) return self.__class__(target) def symlink_to(self, target, target_is_directory=False): @@ -1242,7 +1185,9 @@ def symlink_to(self, target, target_is_directory=False): Make this path a symlink pointing to the target path. Note the order of arguments (link, target) is the reverse of os.symlink. """ - self._accessor.symlink(target, self, target_is_directory) + if not hasattr(os, "symlink"): + raise NotImplementedError("os.symlink() not available on this system") + os.symlink(target, self, target_is_directory) def hardlink_to(self, target): """ @@ -1250,7 +1195,9 @@ def hardlink_to(self, target): Note the order of arguments (self, target) is the reverse of os.link's. """ - self._accessor.link(target, self) + if not hasattr(os, "link"): + raise NotImplementedError("os.link() not available on this system") + os.link(target, self) def link_to(self, target): """ @@ -1268,7 +1215,7 @@ def link_to(self, target): "for removal in Python 3.12. " "Use pathlib.Path.hardlink_to() instead.", DeprecationWarning, stacklevel=2) - self._accessor.link(self, target) + self.__class__(target).hardlink_to(self) # Convenience functions for querying the stat results @@ -1425,7 +1372,7 @@ def expanduser(self): """ if (not (self._drv or self._root) and self._parts and self._parts[0][:1] == '~'): - homedir = self._accessor.expanduser(self._parts[0]) + homedir = os.path.expanduser(self._parts[0]) if homedir[:1] == "~": raise RuntimeError("Could not determine home directory.") return self._from_parts([homedir] + self._parts[1:]) diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index 3fbf1d1f7cd9d..5e46b4ffeae10 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1,3 +1,4 @@ +import contextlib import collections.abc import io import os @@ -1459,7 +1460,7 @@ def test_cwd(self): def test_absolute_common(self): P = self.cls - with mock.patch("pathlib._normal_accessor.getcwd") as getcwd: + with mock.patch("os.getcwd") as getcwd: getcwd.return_value = BASE # Simple relative paths. @@ -1738,21 +1739,18 @@ def test_glob_permissions(self): # Patching is needed to avoid relying on the filesystem # to return the order of the files as the error will not # happen if the symlink is the last item. - - with mock.patch("os.scandir") as scandir: - scandir.return_value = sorted(os.scandir(base)) + real_scandir = os.scandir + def my_scandir(path): + with real_scandir(path) as scandir_it: + entries = list(scandir_it) + entries.sort(key=lambda entry: entry.name) + return contextlib.nullcontext(entries) + + with mock.patch("os.scandir", my_scandir): self.assertEqual(len(set(base.glob("*"))), 3) - - subdir.mkdir() - - with mock.patch("os.scandir") as scandir: - scandir.return_value = sorted(os.scandir(base)) + subdir.mkdir() self.assertEqual(len(set(base.glob("*"))), 4) - - subdir.chmod(000) - - with mock.patch("os.scandir") as scandir: - scandir.return_value = sorted(os.scandir(base)) + subdir.chmod(000) self.assertEqual(len(set(base.glob("*"))), 4) def _check_resolve(self, p, expected, strict=True): @@ -2199,6 +2197,7 @@ def test_mkdir_concurrent_parent_creation(self): p = self.cls(BASE, 'dirCPC%d' % pattern_num) self.assertFalse(p.exists()) + real_mkdir = os.mkdir def my_mkdir(path, mode=0o777): path = str(path) # Emulate another process that would create the directory @@ -2207,15 +2206,15 @@ def my_mkdir(path, mode=0o777): # function is called at most 5 times (dirCPC/dir1/dir2, # dirCPC/dir1, dirCPC, dirCPC/dir1, dirCPC/dir1/dir2). if pattern.pop(): - os.mkdir(path, mode) # From another process. + real_mkdir(path, mode) # From another process. concurrently_created.add(path) - os.mkdir(path, mode) # Our real call. + real_mkdir(path, mode) # Our real call. pattern = [bool(pattern_num & (1 << n)) for n in range(5)] concurrently_created = set() p12 = p / 'dir1' / 'dir2' try: - with mock.patch("pathlib._normal_accessor.mkdir", my_mkdir): + with mock.patch("os.mkdir", my_mkdir): p12.mkdir(parents=True, exist_ok=False) except FileExistsError: self.assertIn(str(p12), concurrently_created) @@ -2676,7 +2675,7 @@ def test_absolute(self): self.assertEqual(str(P(share + 'a\\b').absolute()), share + 'a\\b') # UNC relative paths. - with mock.patch("pathlib._normal_accessor.getcwd") as getcwd: + with mock.patch("os.getcwd") as getcwd: getcwd.return_value = share self.assertEqual(str(P().absolute()), share) diff --git a/Misc/NEWS.d/next/Library/2022-01-05-03-09-29.bpo-43012.RVhLIL.rst b/Misc/NEWS.d/next/Library/2022-01-05-03-09-29.bpo-43012.RVhLIL.rst new file mode 100644 index 0000000000000..c6a6723a96b34 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-05-03-09-29.bpo-43012.RVhLIL.rst @@ -0,0 +1,2 @@ +The pathlib module's obsolete and internal ``_Accessor`` class has been +removed to prepare the terrain for upcoming enhancements to the module. From webhook-mailer at python.org Wed Feb 2 08:50:56 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 13:50:56 -0000 Subject: [Python-checkins] [3.9] bpo-45703: Invalidate _NamespacePath cache on importlib.invalidate_cache (GH-29384) (GH-30922) (GH-31076) Message-ID: https://github.com/python/cpython/commit/8d239bfdcc98f4e4a592d9cda569380a9c15a5c0 commit: 8d239bfdcc98f4e4a592d9cda569380a9c15a5c0 branch: 3.9 author: Petr Viktorin committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T05:50:43-08:00 summary: [3.9] bpo-45703: Invalidate _NamespacePath cache on importlib.invalidate_cache (GH-29384) (GH-30922) (GH-31076) Consider the following directory structure: . ??? PATH1 ??? namespace ??? sub1 ??? __init__.py And both PATH1 and PATH2 in sys path: $ PYTHONPATH=PATH1:PATH2 python3.11 >>> import namespace >>> import namespace.sub1 >>> namespace.__path__ _NamespacePath(['.../PATH1/namespace']) >>> ... While this interpreter still runs, PATH2/namespace/sub2 is created: . ??? PATH1 ? ??? namespace ? ??? sub1 ? ??? __init__.py ??? PATH2 ??? namespace ??? sub2 ??? __init__.py The newly created module cannot be imported: >>> ... >>> namespace.__path__ _NamespacePath(['.../PATH1/namespace']) >>> import namespace.sub2 Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'namespace.sub2' Calling importlib.invalidate_caches() now newly allows to import it: >>> import importlib >>> importlib.invalidate_caches() >>> namespace.__path__ _NamespacePath(['.../PATH1/namespace']) >>> import namespace.sub2 >>> namespace.__path__ _NamespacePath(['.../PATH1/namespace', '.../PATH2/namespace']) This was not previously possible. Co-Authored-By: Miro Hron?ok Automerge-Triggered-By: GH:encukou files: A Misc/NEWS.d/next/Library/2021-11-03-13-41-49.bpo-45703.35AagL.rst M Lib/importlib/_bootstrap_external.py M Lib/test/test_importlib/test_namespace_pkgs.py M Python/importlib_external.h diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py index fe31f437dac2ab..f3828b10e1c1ba 100644 --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -1209,10 +1209,15 @@ class _NamespacePath: using path_finder. For top-level modules, the parent module's path is sys.path.""" + # When invalidate_caches() is called, this epoch is incremented + # https://bugs.python.org/issue45703 + _epoch = 0 + def __init__(self, name, path, path_finder): self._name = name self._path = path self._last_parent_path = tuple(self._get_parent_path()) + self._last_epoch = self._epoch self._path_finder = path_finder def _find_parent_path_names(self): @@ -1232,7 +1237,7 @@ def _get_parent_path(self): def _recalculate(self): # If the parent's path has changed, recalculate _path parent_path = tuple(self._get_parent_path()) # Make a copy - if parent_path != self._last_parent_path: + if parent_path != self._last_parent_path or self._epoch != self._last_epoch: spec = self._path_finder(self._name, parent_path) # Note that no changes are made if a loader is returned, but we # do remember the new parent path @@ -1240,6 +1245,7 @@ def _recalculate(self): if spec.submodule_search_locations: self._path = spec.submodule_search_locations self._last_parent_path = parent_path # Save the copy + self._last_epoch = self._epoch return self._path def __iter__(self): @@ -1320,6 +1326,9 @@ def invalidate_caches(cls): del sys.path_importer_cache[name] elif hasattr(finder, 'invalidate_caches'): finder.invalidate_caches() + # Also invalidate the caches of _NamespacePaths + # https://bugs.python.org/issue45703 + _NamespacePath._epoch += 1 @classmethod def _path_hooks(cls, path): diff --git a/Lib/test/test_importlib/test_namespace_pkgs.py b/Lib/test/test_importlib/test_namespace_pkgs.py index a8f95a035e2450..92771e71e70106 100644 --- a/Lib/test/test_importlib/test_namespace_pkgs.py +++ b/Lib/test/test_importlib/test_namespace_pkgs.py @@ -2,6 +2,7 @@ import importlib import os import sys +import tempfile import unittest from test.test_importlib import util @@ -124,6 +125,40 @@ def test_imports(self): self.assertEqual(foo.two.attr, 'portion2 foo two') +class SeparatedNamespacePackagesCreatedWhileRunning(NamespacePackageTest): + paths = ['portion1'] + + def test_invalidate_caches(self): + with tempfile.TemporaryDirectory() as temp_dir: + # we manipulate sys.path before anything is imported to avoid + # accidental cache invalidation when changing it + sys.path.append(temp_dir) + + import foo.one + self.assertEqual(foo.one.attr, 'portion1 foo one') + + # the module does not exist, so it cannot be imported + with self.assertRaises(ImportError): + import foo.just_created + + # util.create_modules() manipulates sys.path + # so we must create the modules manually instead + namespace_path = os.path.join(temp_dir, 'foo') + os.mkdir(namespace_path) + module_path = os.path.join(namespace_path, 'just_created.py') + with open(module_path, 'w', encoding='utf-8') as file: + file.write('attr = "just_created foo"') + + # the module is not known, so it cannot be imported yet + with self.assertRaises(ImportError): + import foo.just_created + + # but after explicit cache invalidation, it is importable + importlib.invalidate_caches() + import foo.just_created + self.assertEqual(foo.just_created.attr, 'just_created foo') + + class SeparatedOverlappingNamespacePackages(NamespacePackageTest): paths = ['portion1', 'both_portions'] diff --git a/Misc/NEWS.d/next/Library/2021-11-03-13-41-49.bpo-45703.35AagL.rst b/Misc/NEWS.d/next/Library/2021-11-03-13-41-49.bpo-45703.35AagL.rst new file mode 100644 index 00000000000000..9fa9be56b8327a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-11-03-13-41-49.bpo-45703.35AagL.rst @@ -0,0 +1,5 @@ +When a namespace package is imported before another module from the same +namespace is created/installed in a different :data:`sys.path` location +while the program is running, calling the +:func:`importlib.invalidate_caches` function will now also guarantee the new +module is noticed. diff --git a/Python/importlib_external.h b/Python/importlib_external.h index 6d40249e9cd76e..23af2cd20b9250 100644 --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -1882,944 +1882,951 @@ const unsigned char _Py_M__importlib_bootstrap_external[] = { 4,6,8,9,8,4,8,3,8,8,8,6,8,6,8,4, 8,4,2,1,114,19,1,0,0,99,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,2,0,0,0,64,0, - 0,0,115,104,0,0,0,101,0,90,1,100,0,90,2,100, - 1,90,3,100,2,100,3,132,0,90,4,100,4,100,5,132, - 0,90,5,100,6,100,7,132,0,90,6,100,8,100,9,132, - 0,90,7,100,10,100,11,132,0,90,8,100,12,100,13,132, - 0,90,9,100,14,100,15,132,0,90,10,100,16,100,17,132, - 0,90,11,100,18,100,19,132,0,90,12,100,20,100,21,132, - 0,90,13,100,22,100,23,132,0,90,14,100,24,83,0,41, - 25,218,14,95,78,97,109,101,115,112,97,99,101,80,97,116, - 104,97,38,1,0,0,82,101,112,114,101,115,101,110,116,115, - 32,97,32,110,97,109,101,115,112,97,99,101,32,112,97,99, - 107,97,103,101,39,115,32,112,97,116,104,46,32,32,73,116, - 32,117,115,101,115,32,116,104,101,32,109,111,100,117,108,101, - 32,110,97,109,101,10,32,32,32,32,116,111,32,102,105,110, - 100,32,105,116,115,32,112,97,114,101,110,116,32,109,111,100, - 117,108,101,44,32,97,110,100,32,102,114,111,109,32,116,104, - 101,114,101,32,105,116,32,108,111,111,107,115,32,117,112,32, - 116,104,101,32,112,97,114,101,110,116,39,115,10,32,32,32, - 32,95,95,112,97,116,104,95,95,46,32,32,87,104,101,110, - 32,116,104,105,115,32,99,104,97,110,103,101,115,44,32,116, - 104,101,32,109,111,100,117,108,101,39,115,32,111,119,110,32, - 112,97,116,104,32,105,115,32,114,101,99,111,109,112,117,116, - 101,100,44,10,32,32,32,32,117,115,105,110,103,32,112,97, - 116,104,95,102,105,110,100,101,114,46,32,32,70,111,114,32, - 116,111,112,45,108,101,118,101,108,32,109,111,100,117,108,101, - 115,44,32,116,104,101,32,112,97,114,101,110,116,32,109,111, - 100,117,108,101,39,115,32,112,97,116,104,10,32,32,32,32, - 105,115,32,115,121,115,46,112,97,116,104,46,99,4,0,0, - 0,0,0,0,0,0,0,0,0,4,0,0,0,3,0,0, - 0,67,0,0,0,115,36,0,0,0,124,1,124,0,95,0, - 124,2,124,0,95,1,116,2,124,0,160,3,161,0,131,1, - 124,0,95,4,124,3,124,0,95,5,100,0,83,0,114,69, - 0,0,0,41,6,218,5,95,110,97,109,101,218,5,95,112, - 97,116,104,114,135,0,0,0,218,16,95,103,101,116,95,112, - 97,114,101,110,116,95,112,97,116,104,218,17,95,108,97,115, - 116,95,112,97,114,101,110,116,95,112,97,116,104,218,12,95, - 112,97,116,104,95,102,105,110,100,101,114,169,4,114,142,0, - 0,0,114,140,0,0,0,114,65,0,0,0,90,11,112,97, - 116,104,95,102,105,110,100,101,114,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,232,0,0,0,188,4,0, - 0,115,8,0,0,0,0,1,6,1,6,1,14,1,122,23, - 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, - 95,105,110,105,116,95,95,99,1,0,0,0,0,0,0,0, - 0,0,0,0,4,0,0,0,3,0,0,0,67,0,0,0, - 115,38,0,0,0,124,0,106,0,160,1,100,1,161,1,92, - 3,125,1,125,2,125,3,124,2,100,2,107,2,114,30,100, - 3,83,0,124,1,100,4,102,2,83,0,41,5,122,62,82, - 101,116,117,114,110,115,32,97,32,116,117,112,108,101,32,111, - 102,32,40,112,97,114,101,110,116,45,109,111,100,117,108,101, - 45,110,97,109,101,44,32,112,97,114,101,110,116,45,112,97, - 116,104,45,97,116,116,114,45,110,97,109,101,41,114,96,0, - 0,0,114,14,0,0,0,41,2,114,21,0,0,0,114,65, - 0,0,0,90,8,95,95,112,97,116,104,95,95,41,2,114, - 46,1,0,0,114,103,0,0,0,41,4,114,142,0,0,0, - 114,38,1,0,0,218,3,100,111,116,90,2,109,101,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,218,23,95, - 102,105,110,100,95,112,97,114,101,110,116,95,112,97,116,104, - 95,110,97,109,101,115,194,4,0,0,115,8,0,0,0,0, - 2,18,1,8,2,4,3,122,38,95,78,97,109,101,115,112, - 97,99,101,80,97,116,104,46,95,102,105,110,100,95,112,97, - 114,101,110,116,95,112,97,116,104,95,110,97,109,101,115,99, - 1,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0, - 3,0,0,0,67,0,0,0,115,28,0,0,0,124,0,160, - 0,161,0,92,2,125,1,125,2,116,1,116,2,106,3,124, - 1,25,0,124,2,131,2,83,0,114,69,0,0,0,41,4, - 114,53,1,0,0,114,154,0,0,0,114,21,0,0,0,218, - 7,109,111,100,117,108,101,115,41,3,114,142,0,0,0,90, - 18,112,97,114,101,110,116,95,109,111,100,117,108,101,95,110, - 97,109,101,90,14,112,97,116,104,95,97,116,116,114,95,110, - 97,109,101,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,48,1,0,0,204,4,0,0,115,4,0,0,0, - 0,1,12,1,122,31,95,78,97,109,101,115,112,97,99,101, - 80,97,116,104,46,95,103,101,116,95,112,97,114,101,110,116, - 95,112,97,116,104,99,1,0,0,0,0,0,0,0,0,0, - 0,0,3,0,0,0,4,0,0,0,67,0,0,0,115,80, - 0,0,0,116,0,124,0,160,1,161,0,131,1,125,1,124, - 1,124,0,106,2,107,3,114,74,124,0,160,3,124,0,106, - 4,124,1,161,2,125,2,124,2,100,0,117,1,114,68,124, - 2,106,5,100,0,117,0,114,68,124,2,106,6,114,68,124, - 2,106,6,124,0,95,7,124,1,124,0,95,2,124,0,106, - 7,83,0,114,69,0,0,0,41,8,114,135,0,0,0,114, - 48,1,0,0,114,49,1,0,0,114,50,1,0,0,114,46, - 1,0,0,114,164,0,0,0,114,202,0,0,0,114,47,1, - 0,0,41,3,114,142,0,0,0,90,11,112,97,114,101,110, - 116,95,112,97,116,104,114,210,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,218,12,95,114,101,99, - 97,108,99,117,108,97,116,101,208,4,0,0,115,16,0,0, - 0,0,2,12,1,10,1,14,3,18,1,6,1,8,1,6, - 1,122,27,95,78,97,109,101,115,112,97,99,101,80,97,116, - 104,46,95,114,101,99,97,108,99,117,108,97,116,101,99,1, - 0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,3, - 0,0,0,67,0,0,0,115,12,0,0,0,116,0,124,0, - 160,1,161,0,131,1,83,0,114,69,0,0,0,41,2,114, - 29,1,0,0,114,55,1,0,0,114,13,1,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,218,8,95, - 95,105,116,101,114,95,95,221,4,0,0,115,2,0,0,0, - 0,1,122,23,95,78,97,109,101,115,112,97,99,101,80,97, - 116,104,46,95,95,105,116,101,114,95,95,99,2,0,0,0, - 0,0,0,0,0,0,0,0,2,0,0,0,2,0,0,0, - 67,0,0,0,115,12,0,0,0,124,0,160,0,161,0,124, - 1,25,0,83,0,114,69,0,0,0,169,1,114,55,1,0, - 0,41,2,114,142,0,0,0,218,5,105,110,100,101,120,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,11, - 95,95,103,101,116,105,116,101,109,95,95,224,4,0,0,115, - 2,0,0,0,0,1,122,26,95,78,97,109,101,115,112,97, - 99,101,80,97,116,104,46,95,95,103,101,116,105,116,101,109, - 95,95,99,3,0,0,0,0,0,0,0,0,0,0,0,3, - 0,0,0,3,0,0,0,67,0,0,0,115,14,0,0,0, - 124,2,124,0,106,0,124,1,60,0,100,0,83,0,114,69, - 0,0,0,41,1,114,47,1,0,0,41,3,114,142,0,0, - 0,114,58,1,0,0,114,65,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,218,11,95,95,115,101, - 116,105,116,101,109,95,95,227,4,0,0,115,2,0,0,0, - 0,1,122,26,95,78,97,109,101,115,112,97,99,101,80,97, - 116,104,46,95,95,115,101,116,105,116,101,109,95,95,99,1, - 0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,3, - 0,0,0,67,0,0,0,115,12,0,0,0,116,0,124,0, - 160,1,161,0,131,1,83,0,114,69,0,0,0,41,2,114, - 6,0,0,0,114,55,1,0,0,114,13,1,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,218,7,95, - 95,108,101,110,95,95,230,4,0,0,115,2,0,0,0,0, - 1,122,22,95,78,97,109,101,115,112,97,99,101,80,97,116, - 104,46,95,95,108,101,110,95,95,99,1,0,0,0,0,0, - 0,0,0,0,0,0,1,0,0,0,3,0,0,0,67,0, - 0,0,115,12,0,0,0,100,1,160,0,124,0,106,1,161, - 1,83,0,41,2,78,122,20,95,78,97,109,101,115,112,97, - 99,101,80,97,116,104,40,123,33,114,125,41,41,2,114,88, - 0,0,0,114,47,1,0,0,114,13,1,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,218,8,95,95, - 114,101,112,114,95,95,233,4,0,0,115,2,0,0,0,0, - 1,122,23,95,78,97,109,101,115,112,97,99,101,80,97,116, - 104,46,95,95,114,101,112,114,95,95,99,2,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,67, - 0,0,0,115,12,0,0,0,124,1,124,0,160,0,161,0, - 118,0,83,0,114,69,0,0,0,114,57,1,0,0,169,2, - 114,142,0,0,0,218,4,105,116,101,109,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,218,12,95,95,99,111, - 110,116,97,105,110,115,95,95,236,4,0,0,115,2,0,0, - 0,0,1,122,27,95,78,97,109,101,115,112,97,99,101,80, - 97,116,104,46,95,95,99,111,110,116,97,105,110,115,95,95, - 99,2,0,0,0,0,0,0,0,0,0,0,0,2,0,0, - 0,3,0,0,0,67,0,0,0,115,16,0,0,0,124,0, - 106,0,160,1,124,1,161,1,1,0,100,0,83,0,114,69, - 0,0,0,41,2,114,47,1,0,0,114,61,0,0,0,114, - 63,1,0,0,114,10,0,0,0,114,10,0,0,0,114,11, - 0,0,0,114,61,0,0,0,239,4,0,0,115,2,0,0, - 0,0,1,122,21,95,78,97,109,101,115,112,97,99,101,80, - 97,116,104,46,97,112,112,101,110,100,78,41,15,114,149,0, - 0,0,114,148,0,0,0,114,150,0,0,0,114,151,0,0, - 0,114,232,0,0,0,114,53,1,0,0,114,48,1,0,0, - 114,55,1,0,0,114,56,1,0,0,114,59,1,0,0,114, - 60,1,0,0,114,61,1,0,0,114,62,1,0,0,114,65, - 1,0,0,114,61,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,114,45,1,0, - 0,181,4,0,0,115,24,0,0,0,8,1,4,6,8,6, - 8,10,8,4,8,13,8,3,8,3,8,3,8,3,8,3, - 8,3,114,45,1,0,0,99,0,0,0,0,0,0,0,0, - 0,0,0,0,0,0,0,0,3,0,0,0,64,0,0,0, - 115,80,0,0,0,101,0,90,1,100,0,90,2,100,1,100, - 2,132,0,90,3,101,4,100,3,100,4,132,0,131,1,90, - 5,100,5,100,6,132,0,90,6,100,7,100,8,132,0,90, - 7,100,9,100,10,132,0,90,8,100,11,100,12,132,0,90, - 9,100,13,100,14,132,0,90,10,100,15,100,16,132,0,90, - 11,100,17,83,0,41,18,218,16,95,78,97,109,101,115,112, - 97,99,101,76,111,97,100,101,114,99,4,0,0,0,0,0, - 0,0,0,0,0,0,4,0,0,0,4,0,0,0,67,0, - 0,0,115,18,0,0,0,116,0,124,1,124,2,124,3,131, - 3,124,0,95,1,100,0,83,0,114,69,0,0,0,41,2, - 114,45,1,0,0,114,47,1,0,0,114,51,1,0,0,114, + 0,0,115,108,0,0,0,101,0,90,1,100,0,90,2,100, + 1,90,3,100,2,90,4,100,3,100,4,132,0,90,5,100, + 5,100,6,132,0,90,6,100,7,100,8,132,0,90,7,100, + 9,100,10,132,0,90,8,100,11,100,12,132,0,90,9,100, + 13,100,14,132,0,90,10,100,15,100,16,132,0,90,11,100, + 17,100,18,132,0,90,12,100,19,100,20,132,0,90,13,100, + 21,100,22,132,0,90,14,100,23,100,24,132,0,90,15,100, + 25,83,0,41,26,218,14,95,78,97,109,101,115,112,97,99, + 101,80,97,116,104,97,38,1,0,0,82,101,112,114,101,115, + 101,110,116,115,32,97,32,110,97,109,101,115,112,97,99,101, + 32,112,97,99,107,97,103,101,39,115,32,112,97,116,104,46, + 32,32,73,116,32,117,115,101,115,32,116,104,101,32,109,111, + 100,117,108,101,32,110,97,109,101,10,32,32,32,32,116,111, + 32,102,105,110,100,32,105,116,115,32,112,97,114,101,110,116, + 32,109,111,100,117,108,101,44,32,97,110,100,32,102,114,111, + 109,32,116,104,101,114,101,32,105,116,32,108,111,111,107,115, + 32,117,112,32,116,104,101,32,112,97,114,101,110,116,39,115, + 10,32,32,32,32,95,95,112,97,116,104,95,95,46,32,32, + 87,104,101,110,32,116,104,105,115,32,99,104,97,110,103,101, + 115,44,32,116,104,101,32,109,111,100,117,108,101,39,115,32, + 111,119,110,32,112,97,116,104,32,105,115,32,114,101,99,111, + 109,112,117,116,101,100,44,10,32,32,32,32,117,115,105,110, + 103,32,112,97,116,104,95,102,105,110,100,101,114,46,32,32, + 70,111,114,32,116,111,112,45,108,101,118,101,108,32,109,111, + 100,117,108,101,115,44,32,116,104,101,32,112,97,114,101,110, + 116,32,109,111,100,117,108,101,39,115,32,112,97,116,104,10, + 32,32,32,32,105,115,32,115,121,115,46,112,97,116,104,46, + 114,0,0,0,0,99,4,0,0,0,0,0,0,0,0,0, + 0,0,4,0,0,0,3,0,0,0,67,0,0,0,115,44, + 0,0,0,124,1,124,0,95,0,124,2,124,0,95,1,116, + 2,124,0,160,3,161,0,131,1,124,0,95,4,124,0,106, + 5,124,0,95,6,124,3,124,0,95,7,100,0,83,0,114, + 69,0,0,0,41,8,218,5,95,110,97,109,101,218,5,95, + 112,97,116,104,114,135,0,0,0,218,16,95,103,101,116,95, + 112,97,114,101,110,116,95,112,97,116,104,218,17,95,108,97, + 115,116,95,112,97,114,101,110,116,95,112,97,116,104,218,6, + 95,101,112,111,99,104,218,11,95,108,97,115,116,95,101,112, + 111,99,104,218,12,95,112,97,116,104,95,102,105,110,100,101, + 114,169,4,114,142,0,0,0,114,140,0,0,0,114,65,0, + 0,0,90,11,112,97,116,104,95,102,105,110,100,101,114,114, 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,232, - 0,0,0,245,4,0,0,115,2,0,0,0,0,1,122,25, - 95,78,97,109,101,115,112,97,99,101,76,111,97,100,101,114, - 46,95,95,105,110,105,116,95,95,99,2,0,0,0,0,0, - 0,0,0,0,0,0,2,0,0,0,3,0,0,0,67,0, - 0,0,115,12,0,0,0,100,1,160,0,124,1,106,1,161, - 1,83,0,41,2,122,115,82,101,116,117,114,110,32,114,101, - 112,114,32,102,111,114,32,116,104,101,32,109,111,100,117,108, - 101,46,10,10,32,32,32,32,32,32,32,32,84,104,101,32, - 109,101,116,104,111,100,32,105,115,32,100,101,112,114,101,99, - 97,116,101,100,46,32,32,84,104,101,32,105,109,112,111,114, - 116,32,109,97,99,104,105,110,101,114,121,32,100,111,101,115, - 32,116,104,101,32,106,111,98,32,105,116,115,101,108,102,46, - 10,10,32,32,32,32,32,32,32,32,122,25,60,109,111,100, - 117,108,101,32,123,33,114,125,32,40,110,97,109,101,115,112, - 97,99,101,41,62,41,2,114,88,0,0,0,114,149,0,0, - 0,41,2,114,216,0,0,0,114,239,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,218,11,109,111, - 100,117,108,101,95,114,101,112,114,248,4,0,0,115,2,0, - 0,0,0,7,122,28,95,78,97,109,101,115,112,97,99,101, - 76,111,97,100,101,114,46,109,111,100,117,108,101,95,114,101, - 112,114,99,2,0,0,0,0,0,0,0,0,0,0,0,2, - 0,0,0,1,0,0,0,67,0,0,0,115,4,0,0,0, - 100,1,83,0,41,2,78,84,114,10,0,0,0,114,242,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,206,0,0,0,1,5,0,0,115,2,0,0,0,0, - 1,122,27,95,78,97,109,101,115,112,97,99,101,76,111,97, - 100,101,114,46,105,115,95,112,97,99,107,97,103,101,99,2, - 0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,1, - 0,0,0,67,0,0,0,115,4,0,0,0,100,1,83,0, - 41,2,78,114,14,0,0,0,114,10,0,0,0,114,242,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,252,0,0,0,4,5,0,0,115,2,0,0,0,0, - 1,122,27,95,78,97,109,101,115,112,97,99,101,76,111,97, - 100,101,114,46,103,101,116,95,115,111,117,114,99,101,99,2, - 0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,6, - 0,0,0,67,0,0,0,115,16,0,0,0,116,0,100,1, - 100,2,100,3,100,4,100,5,141,4,83,0,41,6,78,114, - 14,0,0,0,122,8,60,115,116,114,105,110,103,62,114,238, - 0,0,0,84,41,1,114,254,0,0,0,41,1,114,255,0, - 0,0,114,242,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,114,236,0,0,0,7,5,0,0,115, - 2,0,0,0,0,1,122,25,95,78,97,109,101,115,112,97, - 99,101,76,111,97,100,101,114,46,103,101,116,95,99,111,100, - 101,99,2,0,0,0,0,0,0,0,0,0,0,0,2,0, - 0,0,1,0,0,0,67,0,0,0,115,4,0,0,0,100, - 1,83,0,114,233,0,0,0,114,10,0,0,0,114,234,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,235,0,0,0,10,5,0,0,115,2,0,0,0,0, - 1,122,30,95,78,97,109,101,115,112,97,99,101,76,111,97, - 100,101,114,46,99,114,101,97,116,101,95,109,111,100,117,108, - 101,99,2,0,0,0,0,0,0,0,0,0,0,0,2,0, - 0,0,1,0,0,0,67,0,0,0,115,4,0,0,0,100, - 0,83,0,114,69,0,0,0,114,10,0,0,0,114,20,1, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,240,0,0,0,13,5,0,0,115,2,0,0,0,0, - 1,122,28,95,78,97,109,101,115,112,97,99,101,76,111,97, - 100,101,114,46,101,120,101,99,95,109,111,100,117,108,101,99, - 2,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0, - 4,0,0,0,67,0,0,0,115,26,0,0,0,116,0,160, - 1,100,1,124,0,106,2,161,2,1,0,116,0,160,3,124, - 0,124,1,161,2,83,0,41,2,122,98,76,111,97,100,32, - 97,32,110,97,109,101,115,112,97,99,101,32,109,111,100,117, - 108,101,46,10,10,32,32,32,32,32,32,32,32,84,104,105, - 115,32,109,101,116,104,111,100,32,105,115,32,100,101,112,114, - 101,99,97,116,101,100,46,32,32,85,115,101,32,101,120,101, - 99,95,109,111,100,117,108,101,40,41,32,105,110,115,116,101, - 97,100,46,10,10,32,32,32,32,32,32,32,32,122,38,110, - 97,109,101,115,112,97,99,101,32,109,111,100,117,108,101,32, - 108,111,97,100,101,100,32,119,105,116,104,32,112,97,116,104, - 32,123,33,114,125,41,4,114,158,0,0,0,114,173,0,0, - 0,114,47,1,0,0,114,241,0,0,0,114,242,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 243,0,0,0,16,5,0,0,115,8,0,0,0,0,7,6, - 1,4,255,4,2,122,28,95,78,97,109,101,115,112,97,99, - 101,76,111,97,100,101,114,46,108,111,97,100,95,109,111,100, - 117,108,101,78,41,12,114,149,0,0,0,114,148,0,0,0, - 114,150,0,0,0,114,232,0,0,0,114,230,0,0,0,114, - 67,1,0,0,114,206,0,0,0,114,252,0,0,0,114,236, - 0,0,0,114,235,0,0,0,114,240,0,0,0,114,243,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,114,66,1,0,0,244,4,0,0,115, - 18,0,0,0,8,1,8,3,2,1,10,8,8,3,8,3, - 8,3,8,3,8,3,114,66,1,0,0,99,0,0,0,0, - 0,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0, - 64,0,0,0,115,118,0,0,0,101,0,90,1,100,0,90, - 2,100,1,90,3,101,4,100,2,100,3,132,0,131,1,90, - 5,101,4,100,4,100,5,132,0,131,1,90,6,101,4,100, - 6,100,7,132,0,131,1,90,7,101,4,100,8,100,9,132, - 0,131,1,90,8,101,4,100,19,100,11,100,12,132,1,131, - 1,90,9,101,4,100,20,100,13,100,14,132,1,131,1,90, - 10,101,4,100,21,100,15,100,16,132,1,131,1,90,11,101, - 4,100,17,100,18,132,0,131,1,90,12,100,10,83,0,41, - 22,218,10,80,97,116,104,70,105,110,100,101,114,122,62,77, - 101,116,97,32,112,97,116,104,32,102,105,110,100,101,114,32, - 102,111,114,32,115,121,115,46,112,97,116,104,32,97,110,100, - 32,112,97,99,107,97,103,101,32,95,95,112,97,116,104,95, - 95,32,97,116,116,114,105,98,117,116,101,115,46,99,1,0, + 0,0,0,192,4,0,0,115,10,0,0,0,0,1,6,1, + 6,1,14,1,8,1,122,23,95,78,97,109,101,115,112,97, + 99,101,80,97,116,104,46,95,95,105,110,105,116,95,95,99, + 1,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0, + 3,0,0,0,67,0,0,0,115,38,0,0,0,124,0,106, + 0,160,1,100,1,161,1,92,3,125,1,125,2,125,3,124, + 2,100,2,107,2,114,30,100,3,83,0,124,1,100,4,102, + 2,83,0,41,5,122,62,82,101,116,117,114,110,115,32,97, + 32,116,117,112,108,101,32,111,102,32,40,112,97,114,101,110, + 116,45,109,111,100,117,108,101,45,110,97,109,101,44,32,112, + 97,114,101,110,116,45,112,97,116,104,45,97,116,116,114,45, + 110,97,109,101,41,114,96,0,0,0,114,14,0,0,0,41, + 2,114,21,0,0,0,114,65,0,0,0,90,8,95,95,112, + 97,116,104,95,95,41,2,114,46,1,0,0,114,103,0,0, + 0,41,4,114,142,0,0,0,114,38,1,0,0,218,3,100, + 111,116,90,2,109,101,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,218,23,95,102,105,110,100,95,112,97,114, + 101,110,116,95,112,97,116,104,95,110,97,109,101,115,199,4, + 0,0,115,8,0,0,0,0,2,18,1,8,2,4,3,122, + 38,95,78,97,109,101,115,112,97,99,101,80,97,116,104,46, + 95,102,105,110,100,95,112,97,114,101,110,116,95,112,97,116, + 104,95,110,97,109,101,115,99,1,0,0,0,0,0,0,0, + 0,0,0,0,3,0,0,0,3,0,0,0,67,0,0,0, + 115,28,0,0,0,124,0,160,0,161,0,92,2,125,1,125, + 2,116,1,116,2,106,3,124,1,25,0,124,2,131,2,83, + 0,114,69,0,0,0,41,4,114,55,1,0,0,114,154,0, + 0,0,114,21,0,0,0,218,7,109,111,100,117,108,101,115, + 41,3,114,142,0,0,0,90,18,112,97,114,101,110,116,95, + 109,111,100,117,108,101,95,110,97,109,101,90,14,112,97,116, + 104,95,97,116,116,114,95,110,97,109,101,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,114,48,1,0,0,209, + 4,0,0,115,4,0,0,0,0,1,12,1,122,31,95,78, + 97,109,101,115,112,97,99,101,80,97,116,104,46,95,103,101, + 116,95,112,97,114,101,110,116,95,112,97,116,104,99,1,0, 0,0,0,0,0,0,0,0,0,0,3,0,0,0,4,0, - 0,0,67,0,0,0,115,64,0,0,0,116,0,116,1,106, - 2,160,3,161,0,131,1,68,0,93,44,92,2,125,1,125, - 2,124,2,100,1,117,0,114,40,116,1,106,2,124,1,61, - 0,113,14,116,4,124,2,100,2,131,2,114,14,124,2,160, - 5,161,0,1,0,113,14,100,1,83,0,41,3,122,125,67, - 97,108,108,32,116,104,101,32,105,110,118,97,108,105,100,97, - 116,101,95,99,97,99,104,101,115,40,41,32,109,101,116,104, - 111,100,32,111,110,32,97,108,108,32,112,97,116,104,32,101, - 110,116,114,121,32,102,105,110,100,101,114,115,10,32,32,32, - 32,32,32,32,32,115,116,111,114,101,100,32,105,110,32,115, - 121,115,46,112,97,116,104,95,105,109,112,111,114,116,101,114, - 95,99,97,99,104,101,115,32,40,119,104,101,114,101,32,105, - 109,112,108,101,109,101,110,116,101,100,41,46,78,218,17,105, - 110,118,97,108,105,100,97,116,101,95,99,97,99,104,101,115, - 41,6,218,4,108,105,115,116,114,21,0,0,0,218,19,112, - 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, - 104,101,218,5,105,116,101,109,115,114,152,0,0,0,114,69, - 1,0,0,41,3,114,216,0,0,0,114,140,0,0,0,218, - 6,102,105,110,100,101,114,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,114,69,1,0,0,34,5,0,0,115, - 10,0,0,0,0,4,22,1,8,1,10,1,10,1,122,28, - 80,97,116,104,70,105,110,100,101,114,46,105,110,118,97,108, - 105,100,97,116,101,95,99,97,99,104,101,115,99,2,0,0, - 0,0,0,0,0,0,0,0,0,3,0,0,0,9,0,0, - 0,67,0,0,0,115,82,0,0,0,116,0,106,1,100,1, - 117,1,114,28,116,0,106,1,115,28,116,2,160,3,100,2, - 116,4,161,2,1,0,116,0,106,1,68,0,93,42,125,2, - 122,14,124,2,124,1,131,1,87,0,2,0,1,0,83,0, - 4,0,116,5,121,74,1,0,1,0,1,0,89,0,113,34, - 89,0,113,34,48,0,113,34,100,1,83,0,41,3,122,46, - 83,101,97,114,99,104,32,115,121,115,46,112,97,116,104,95, - 104,111,111,107,115,32,102,111,114,32,97,32,102,105,110,100, - 101,114,32,102,111,114,32,39,112,97,116,104,39,46,78,122, - 23,115,121,115,46,112,97,116,104,95,104,111,111,107,115,32, - 105,115,32,101,109,112,116,121,41,6,114,21,0,0,0,218, - 10,112,97,116,104,95,104,111,111,107,115,114,98,0,0,0, - 114,99,0,0,0,114,162,0,0,0,114,141,0,0,0,41, - 3,114,216,0,0,0,114,65,0,0,0,90,4,104,111,111, - 107,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 218,11,95,112,97,116,104,95,104,111,111,107,115,44,5,0, - 0,115,16,0,0,0,0,3,16,1,12,1,10,1,2,1, - 14,1,12,1,12,2,122,22,80,97,116,104,70,105,110,100, - 101,114,46,95,112,97,116,104,95,104,111,111,107,115,99,2, - 0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,8, - 0,0,0,67,0,0,0,115,100,0,0,0,124,1,100,1, - 107,2,114,42,122,12,116,0,160,1,161,0,125,1,87,0, - 110,20,4,0,116,2,121,40,1,0,1,0,1,0,89,0, - 100,2,83,0,48,0,122,14,116,3,106,4,124,1,25,0, - 125,2,87,0,110,38,4,0,116,5,121,94,1,0,1,0, - 1,0,124,0,160,6,124,1,161,1,125,2,124,2,116,3, - 106,4,124,1,60,0,89,0,110,2,48,0,124,2,83,0, - 41,3,122,210,71,101,116,32,116,104,101,32,102,105,110,100, - 101,114,32,102,111,114,32,116,104,101,32,112,97,116,104,32, - 101,110,116,114,121,32,102,114,111,109,32,115,121,115,46,112, - 97,116,104,95,105,109,112,111,114,116,101,114,95,99,97,99, - 104,101,46,10,10,32,32,32,32,32,32,32,32,73,102,32, - 116,104,101,32,112,97,116,104,32,101,110,116,114,121,32,105, - 115,32,110,111,116,32,105,110,32,116,104,101,32,99,97,99, - 104,101,44,32,102,105,110,100,32,116,104,101,32,97,112,112, - 114,111,112,114,105,97,116,101,32,102,105,110,100,101,114,10, - 32,32,32,32,32,32,32,32,97,110,100,32,99,97,99,104, - 101,32,105,116,46,32,73,102,32,110,111,32,102,105,110,100, - 101,114,32,105,115,32,97,118,97,105,108,97,98,108,101,44, - 32,115,116,111,114,101,32,78,111,110,101,46,10,10,32,32, - 32,32,32,32,32,32,114,14,0,0,0,78,41,7,114,24, - 0,0,0,114,81,0,0,0,114,26,1,0,0,114,21,0, - 0,0,114,71,1,0,0,218,8,75,101,121,69,114,114,111, - 114,114,75,1,0,0,41,3,114,216,0,0,0,114,65,0, - 0,0,114,73,1,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,218,20,95,112,97,116,104,95,105,109, - 112,111,114,116,101,114,95,99,97,99,104,101,57,5,0,0, - 115,22,0,0,0,0,8,8,1,2,1,12,1,12,3,8, - 1,2,1,14,1,12,1,10,1,16,1,122,31,80,97,116, - 104,70,105,110,100,101,114,46,95,112,97,116,104,95,105,109, - 112,111,114,116,101,114,95,99,97,99,104,101,99,3,0,0, - 0,0,0,0,0,0,0,0,0,6,0,0,0,4,0,0, - 0,67,0,0,0,115,82,0,0,0,116,0,124,2,100,1, - 131,2,114,26,124,2,160,1,124,1,161,1,92,2,125,3, - 125,4,110,14,124,2,160,2,124,1,161,1,125,3,103,0, - 125,4,124,3,100,0,117,1,114,60,116,3,160,4,124,1, - 124,3,161,2,83,0,116,3,160,5,124,1,100,0,161,2, - 125,5,124,4,124,5,95,6,124,5,83,0,41,2,78,114, - 161,0,0,0,41,7,114,152,0,0,0,114,161,0,0,0, - 114,229,0,0,0,114,158,0,0,0,114,224,0,0,0,114, - 207,0,0,0,114,202,0,0,0,41,6,114,216,0,0,0, - 114,163,0,0,0,114,73,1,0,0,114,164,0,0,0,114, - 165,0,0,0,114,210,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,218,16,95,108,101,103,97,99, - 121,95,103,101,116,95,115,112,101,99,79,5,0,0,115,18, - 0,0,0,0,4,10,1,16,2,10,1,4,1,8,1,12, - 1,12,1,6,1,122,27,80,97,116,104,70,105,110,100,101, - 114,46,95,108,101,103,97,99,121,95,103,101,116,95,115,112, - 101,99,78,99,4,0,0,0,0,0,0,0,0,0,0,0, - 9,0,0,0,5,0,0,0,67,0,0,0,115,166,0,0, - 0,103,0,125,4,124,2,68,0,93,134,125,5,116,0,124, - 5,116,1,116,2,102,2,131,2,115,28,113,8,124,0,160, - 3,124,5,161,1,125,6,124,6,100,1,117,1,114,8,116, - 4,124,6,100,2,131,2,114,70,124,6,160,5,124,1,124, - 3,161,2,125,7,110,12,124,0,160,6,124,1,124,6,161, - 2,125,7,124,7,100,1,117,0,114,92,113,8,124,7,106, - 7,100,1,117,1,114,110,124,7,2,0,1,0,83,0,124, - 7,106,8,125,8,124,8,100,1,117,0,114,132,116,9,100, - 3,131,1,130,1,124,4,160,10,124,8,161,1,1,0,113, - 8,116,11,160,12,124,1,100,1,161,2,125,7,124,4,124, - 7,95,8,124,7,83,0,41,4,122,63,70,105,110,100,32, - 116,104,101,32,108,111,97,100,101,114,32,111,114,32,110,97, - 109,101,115,112,97,99,101,95,112,97,116,104,32,102,111,114, - 32,116,104,105,115,32,109,111,100,117,108,101,47,112,97,99, - 107,97,103,101,32,110,97,109,101,46,78,114,226,0,0,0, - 122,19,115,112,101,99,32,109,105,115,115,105,110,103,32,108, - 111,97,100,101,114,41,13,114,185,0,0,0,114,108,0,0, - 0,218,5,98,121,116,101,115,114,77,1,0,0,114,152,0, - 0,0,114,226,0,0,0,114,78,1,0,0,114,164,0,0, - 0,114,202,0,0,0,114,141,0,0,0,114,191,0,0,0, - 114,158,0,0,0,114,207,0,0,0,41,9,114,216,0,0, - 0,114,163,0,0,0,114,65,0,0,0,114,225,0,0,0, - 218,14,110,97,109,101,115,112,97,99,101,95,112,97,116,104, - 90,5,101,110,116,114,121,114,73,1,0,0,114,210,0,0, - 0,114,165,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,9,95,103,101,116,95,115,112,101,99, - 94,5,0,0,115,40,0,0,0,0,5,4,1,8,1,14, - 1,2,1,10,1,8,1,10,1,14,2,12,1,8,1,2, - 1,10,1,8,1,6,1,8,1,8,5,12,2,12,1,6, - 1,122,20,80,97,116,104,70,105,110,100,101,114,46,95,103, - 101,116,95,115,112,101,99,99,4,0,0,0,0,0,0,0, - 0,0,0,0,6,0,0,0,5,0,0,0,67,0,0,0, - 115,100,0,0,0,124,2,100,1,117,0,114,14,116,0,106, - 1,125,2,124,0,160,2,124,1,124,2,124,3,161,3,125, - 4,124,4,100,1,117,0,114,40,100,1,83,0,124,4,106, - 3,100,1,117,0,114,92,124,4,106,4,125,5,124,5,114, - 86,100,1,124,4,95,5,116,6,124,1,124,5,124,0,106, - 2,131,3,124,4,95,4,124,4,83,0,100,1,83,0,110, - 4,124,4,83,0,100,1,83,0,41,2,122,141,84,114,121, - 32,116,111,32,102,105,110,100,32,97,32,115,112,101,99,32, - 102,111,114,32,39,102,117,108,108,110,97,109,101,39,32,111, - 110,32,115,121,115,46,112,97,116,104,32,111,114,32,39,112, - 97,116,104,39,46,10,10,32,32,32,32,32,32,32,32,84, - 104,101,32,115,101,97,114,99,104,32,105,115,32,98,97,115, - 101,100,32,111,110,32,115,121,115,46,112,97,116,104,95,104, - 111,111,107,115,32,97,110,100,32,115,121,115,46,112,97,116, + 0,0,67,0,0,0,115,100,0,0,0,116,0,124,0,160, + 1,161,0,131,1,125,1,124,1,124,0,106,2,107,3,115, + 34,124,0,106,3,124,0,106,4,107,3,114,94,124,0,160, + 5,124,0,106,6,124,1,161,2,125,2,124,2,100,0,117, + 1,114,80,124,2,106,7,100,0,117,0,114,80,124,2,106, + 8,114,80,124,2,106,8,124,0,95,9,124,1,124,0,95, + 2,124,0,106,3,124,0,95,4,124,0,106,9,83,0,114, + 69,0,0,0,41,10,114,135,0,0,0,114,48,1,0,0, + 114,49,1,0,0,114,50,1,0,0,114,51,1,0,0,114, + 52,1,0,0,114,46,1,0,0,114,164,0,0,0,114,202, + 0,0,0,114,47,1,0,0,41,3,114,142,0,0,0,90, + 11,112,97,114,101,110,116,95,112,97,116,104,114,210,0,0, + 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, + 218,12,95,114,101,99,97,108,99,117,108,97,116,101,213,4, + 0,0,115,18,0,0,0,0,2,12,1,22,1,14,3,18, + 1,6,1,8,1,6,1,8,1,122,27,95,78,97,109,101, + 115,112,97,99,101,80,97,116,104,46,95,114,101,99,97,108, + 99,117,108,97,116,101,99,1,0,0,0,0,0,0,0,0, + 0,0,0,1,0,0,0,3,0,0,0,67,0,0,0,115, + 12,0,0,0,116,0,124,0,160,1,161,0,131,1,83,0, + 114,69,0,0,0,41,2,114,29,1,0,0,114,57,1,0, + 0,114,13,1,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,218,8,95,95,105,116,101,114,95,95,227, + 4,0,0,115,2,0,0,0,0,1,122,23,95,78,97,109, + 101,115,112,97,99,101,80,97,116,104,46,95,95,105,116,101, + 114,95,95,99,2,0,0,0,0,0,0,0,0,0,0,0, + 2,0,0,0,2,0,0,0,67,0,0,0,115,12,0,0, + 0,124,0,160,0,161,0,124,1,25,0,83,0,114,69,0, + 0,0,169,1,114,57,1,0,0,41,2,114,142,0,0,0, + 218,5,105,110,100,101,120,114,10,0,0,0,114,10,0,0, + 0,114,11,0,0,0,218,11,95,95,103,101,116,105,116,101, + 109,95,95,230,4,0,0,115,2,0,0,0,0,1,122,26, + 95,78,97,109,101,115,112,97,99,101,80,97,116,104,46,95, + 95,103,101,116,105,116,101,109,95,95,99,3,0,0,0,0, + 0,0,0,0,0,0,0,3,0,0,0,3,0,0,0,67, + 0,0,0,115,14,0,0,0,124,2,124,0,106,0,124,1, + 60,0,100,0,83,0,114,69,0,0,0,41,1,114,47,1, + 0,0,41,3,114,142,0,0,0,114,60,1,0,0,114,65, + 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,218,11,95,95,115,101,116,105,116,101,109,95,95,233, + 4,0,0,115,2,0,0,0,0,1,122,26,95,78,97,109, + 101,115,112,97,99,101,80,97,116,104,46,95,95,115,101,116, + 105,116,101,109,95,95,99,1,0,0,0,0,0,0,0,0, + 0,0,0,1,0,0,0,3,0,0,0,67,0,0,0,115, + 12,0,0,0,116,0,124,0,160,1,161,0,131,1,83,0, + 114,69,0,0,0,41,2,114,6,0,0,0,114,57,1,0, + 0,114,13,1,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,218,7,95,95,108,101,110,95,95,236,4, + 0,0,115,2,0,0,0,0,1,122,22,95,78,97,109,101, + 115,112,97,99,101,80,97,116,104,46,95,95,108,101,110,95, + 95,99,1,0,0,0,0,0,0,0,0,0,0,0,1,0, + 0,0,3,0,0,0,67,0,0,0,115,12,0,0,0,100, + 1,160,0,124,0,106,1,161,1,83,0,41,2,78,122,20, + 95,78,97,109,101,115,112,97,99,101,80,97,116,104,40,123, + 33,114,125,41,41,2,114,88,0,0,0,114,47,1,0,0, + 114,13,1,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,218,8,95,95,114,101,112,114,95,95,239,4, + 0,0,115,2,0,0,0,0,1,122,23,95,78,97,109,101, + 115,112,97,99,101,80,97,116,104,46,95,95,114,101,112,114, + 95,95,99,2,0,0,0,0,0,0,0,0,0,0,0,2, + 0,0,0,3,0,0,0,67,0,0,0,115,12,0,0,0, + 124,1,124,0,160,0,161,0,118,0,83,0,114,69,0,0, + 0,114,59,1,0,0,169,2,114,142,0,0,0,218,4,105, + 116,101,109,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,218,12,95,95,99,111,110,116,97,105,110,115,95,95, + 242,4,0,0,115,2,0,0,0,0,1,122,27,95,78,97, + 109,101,115,112,97,99,101,80,97,116,104,46,95,95,99,111, + 110,116,97,105,110,115,95,95,99,2,0,0,0,0,0,0, + 0,0,0,0,0,2,0,0,0,3,0,0,0,67,0,0, + 0,115,16,0,0,0,124,0,106,0,160,1,124,1,161,1, + 1,0,100,0,83,0,114,69,0,0,0,41,2,114,47,1, + 0,0,114,61,0,0,0,114,65,1,0,0,114,10,0,0, + 0,114,10,0,0,0,114,11,0,0,0,114,61,0,0,0, + 245,4,0,0,115,2,0,0,0,0,1,122,21,95,78,97, + 109,101,115,112,97,99,101,80,97,116,104,46,97,112,112,101, + 110,100,78,41,16,114,149,0,0,0,114,148,0,0,0,114, + 150,0,0,0,114,151,0,0,0,114,50,1,0,0,114,232, + 0,0,0,114,55,1,0,0,114,48,1,0,0,114,57,1, + 0,0,114,58,1,0,0,114,61,1,0,0,114,62,1,0, + 0,114,63,1,0,0,114,64,1,0,0,114,67,1,0,0, + 114,61,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,114,45,1,0,0,181,4, + 0,0,115,26,0,0,0,8,1,4,8,4,2,8,7,8, + 10,8,4,8,14,8,3,8,3,8,3,8,3,8,3,8, + 3,114,45,1,0,0,99,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,3,0,0,0,64,0,0,0,115, + 80,0,0,0,101,0,90,1,100,0,90,2,100,1,100,2, + 132,0,90,3,101,4,100,3,100,4,132,0,131,1,90,5, + 100,5,100,6,132,0,90,6,100,7,100,8,132,0,90,7, + 100,9,100,10,132,0,90,8,100,11,100,12,132,0,90,9, + 100,13,100,14,132,0,90,10,100,15,100,16,132,0,90,11, + 100,17,83,0,41,18,218,16,95,78,97,109,101,115,112,97, + 99,101,76,111,97,100,101,114,99,4,0,0,0,0,0,0, + 0,0,0,0,0,4,0,0,0,4,0,0,0,67,0,0, + 0,115,18,0,0,0,116,0,124,1,124,2,124,3,131,3, + 124,0,95,1,100,0,83,0,114,69,0,0,0,41,2,114, + 45,1,0,0,114,47,1,0,0,114,53,1,0,0,114,10, + 0,0,0,114,10,0,0,0,114,11,0,0,0,114,232,0, + 0,0,251,4,0,0,115,2,0,0,0,0,1,122,25,95, + 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,46, + 95,95,105,110,105,116,95,95,99,2,0,0,0,0,0,0, + 0,0,0,0,0,2,0,0,0,3,0,0,0,67,0,0, + 0,115,12,0,0,0,100,1,160,0,124,1,106,1,161,1, + 83,0,41,2,122,115,82,101,116,117,114,110,32,114,101,112, + 114,32,102,111,114,32,116,104,101,32,109,111,100,117,108,101, + 46,10,10,32,32,32,32,32,32,32,32,84,104,101,32,109, + 101,116,104,111,100,32,105,115,32,100,101,112,114,101,99,97, + 116,101,100,46,32,32,84,104,101,32,105,109,112,111,114,116, + 32,109,97,99,104,105,110,101,114,121,32,100,111,101,115,32, + 116,104,101,32,106,111,98,32,105,116,115,101,108,102,46,10, + 10,32,32,32,32,32,32,32,32,122,25,60,109,111,100,117, + 108,101,32,123,33,114,125,32,40,110,97,109,101,115,112,97, + 99,101,41,62,41,2,114,88,0,0,0,114,149,0,0,0, + 41,2,114,216,0,0,0,114,239,0,0,0,114,10,0,0, + 0,114,10,0,0,0,114,11,0,0,0,218,11,109,111,100, + 117,108,101,95,114,101,112,114,254,4,0,0,115,2,0,0, + 0,0,7,122,28,95,78,97,109,101,115,112,97,99,101,76, + 111,97,100,101,114,46,109,111,100,117,108,101,95,114,101,112, + 114,99,2,0,0,0,0,0,0,0,0,0,0,0,2,0, + 0,0,1,0,0,0,67,0,0,0,115,4,0,0,0,100, + 1,83,0,41,2,78,84,114,10,0,0,0,114,242,0,0, + 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, + 114,206,0,0,0,7,5,0,0,115,2,0,0,0,0,1, + 122,27,95,78,97,109,101,115,112,97,99,101,76,111,97,100, + 101,114,46,105,115,95,112,97,99,107,97,103,101,99,2,0, + 0,0,0,0,0,0,0,0,0,0,2,0,0,0,1,0, + 0,0,67,0,0,0,115,4,0,0,0,100,1,83,0,41, + 2,78,114,14,0,0,0,114,10,0,0,0,114,242,0,0, + 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, + 114,252,0,0,0,10,5,0,0,115,2,0,0,0,0,1, + 122,27,95,78,97,109,101,115,112,97,99,101,76,111,97,100, + 101,114,46,103,101,116,95,115,111,117,114,99,101,99,2,0, + 0,0,0,0,0,0,0,0,0,0,2,0,0,0,6,0, + 0,0,67,0,0,0,115,16,0,0,0,116,0,100,1,100, + 2,100,3,100,4,100,5,141,4,83,0,41,6,78,114,14, + 0,0,0,122,8,60,115,116,114,105,110,103,62,114,238,0, + 0,0,84,41,1,114,254,0,0,0,41,1,114,255,0,0, + 0,114,242,0,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,114,236,0,0,0,13,5,0,0,115,2, + 0,0,0,0,1,122,25,95,78,97,109,101,115,112,97,99, + 101,76,111,97,100,101,114,46,103,101,116,95,99,111,100,101, + 99,2,0,0,0,0,0,0,0,0,0,0,0,2,0,0, + 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,1, + 83,0,114,233,0,0,0,114,10,0,0,0,114,234,0,0, + 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, + 114,235,0,0,0,16,5,0,0,115,2,0,0,0,0,1, + 122,30,95,78,97,109,101,115,112,97,99,101,76,111,97,100, + 101,114,46,99,114,101,97,116,101,95,109,111,100,117,108,101, + 99,2,0,0,0,0,0,0,0,0,0,0,0,2,0,0, + 0,1,0,0,0,67,0,0,0,115,4,0,0,0,100,0, + 83,0,114,69,0,0,0,114,10,0,0,0,114,20,1,0, + 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, + 114,240,0,0,0,19,5,0,0,115,2,0,0,0,0,1, + 122,28,95,78,97,109,101,115,112,97,99,101,76,111,97,100, + 101,114,46,101,120,101,99,95,109,111,100,117,108,101,99,2, + 0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,4, + 0,0,0,67,0,0,0,115,26,0,0,0,116,0,160,1, + 100,1,124,0,106,2,161,2,1,0,116,0,160,3,124,0, + 124,1,161,2,83,0,41,2,122,98,76,111,97,100,32,97, + 32,110,97,109,101,115,112,97,99,101,32,109,111,100,117,108, + 101,46,10,10,32,32,32,32,32,32,32,32,84,104,105,115, + 32,109,101,116,104,111,100,32,105,115,32,100,101,112,114,101, + 99,97,116,101,100,46,32,32,85,115,101,32,101,120,101,99, + 95,109,111,100,117,108,101,40,41,32,105,110,115,116,101,97, + 100,46,10,10,32,32,32,32,32,32,32,32,122,38,110,97, + 109,101,115,112,97,99,101,32,109,111,100,117,108,101,32,108, + 111,97,100,101,100,32,119,105,116,104,32,112,97,116,104,32, + 123,33,114,125,41,4,114,158,0,0,0,114,173,0,0,0, + 114,47,1,0,0,114,241,0,0,0,114,242,0,0,0,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,243, + 0,0,0,22,5,0,0,115,8,0,0,0,0,7,6,1, + 4,255,4,2,122,28,95,78,97,109,101,115,112,97,99,101, + 76,111,97,100,101,114,46,108,111,97,100,95,109,111,100,117, + 108,101,78,41,12,114,149,0,0,0,114,148,0,0,0,114, + 150,0,0,0,114,232,0,0,0,114,230,0,0,0,114,69, + 1,0,0,114,206,0,0,0,114,252,0,0,0,114,236,0, + 0,0,114,235,0,0,0,114,240,0,0,0,114,243,0,0, + 0,114,10,0,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,114,68,1,0,0,250,4,0,0,115,18, + 0,0,0,8,1,8,3,2,1,10,8,8,3,8,3,8, + 3,8,3,8,3,114,68,1,0,0,99,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,64, + 0,0,0,115,118,0,0,0,101,0,90,1,100,0,90,2, + 100,1,90,3,101,4,100,2,100,3,132,0,131,1,90,5, + 101,4,100,4,100,5,132,0,131,1,90,6,101,4,100,6, + 100,7,132,0,131,1,90,7,101,4,100,8,100,9,132,0, + 131,1,90,8,101,4,100,19,100,11,100,12,132,1,131,1, + 90,9,101,4,100,20,100,13,100,14,132,1,131,1,90,10, + 101,4,100,21,100,15,100,16,132,1,131,1,90,11,101,4, + 100,17,100,18,132,0,131,1,90,12,100,10,83,0,41,22, + 218,10,80,97,116,104,70,105,110,100,101,114,122,62,77,101, + 116,97,32,112,97,116,104,32,102,105,110,100,101,114,32,102, + 111,114,32,115,121,115,46,112,97,116,104,32,97,110,100,32, + 112,97,99,107,97,103,101,32,95,95,112,97,116,104,95,95, + 32,97,116,116,114,105,98,117,116,101,115,46,99,1,0,0, + 0,0,0,0,0,0,0,0,0,3,0,0,0,4,0,0, + 0,67,0,0,0,115,78,0,0,0,116,0,116,1,106,2, + 160,3,161,0,131,1,68,0,93,44,92,2,125,1,125,2, + 124,2,100,1,117,0,114,40,116,1,106,2,124,1,61,0, + 113,14,116,4,124,2,100,2,131,2,114,14,124,2,160,5, + 161,0,1,0,113,14,116,6,4,0,106,7,100,3,55,0, + 2,0,95,7,100,1,83,0,41,4,122,125,67,97,108,108, + 32,116,104,101,32,105,110,118,97,108,105,100,97,116,101,95, + 99,97,99,104,101,115,40,41,32,109,101,116,104,111,100,32, + 111,110,32,97,108,108,32,112,97,116,104,32,101,110,116,114, + 121,32,102,105,110,100,101,114,115,10,32,32,32,32,32,32, + 32,32,115,116,111,114,101,100,32,105,110,32,115,121,115,46, + 112,97,116,104,95,105,109,112,111,114,116,101,114,95,99,97, + 99,104,101,115,32,40,119,104,101,114,101,32,105,109,112,108, + 101,109,101,110,116,101,100,41,46,78,218,17,105,110,118,97, + 108,105,100,97,116,101,95,99,97,99,104,101,115,114,4,0, + 0,0,41,8,218,4,108,105,115,116,114,21,0,0,0,218, + 19,112,97,116,104,95,105,109,112,111,114,116,101,114,95,99, + 97,99,104,101,218,5,105,116,101,109,115,114,152,0,0,0, + 114,71,1,0,0,114,45,1,0,0,114,50,1,0,0,41, + 3,114,216,0,0,0,114,140,0,0,0,218,6,102,105,110, + 100,101,114,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,114,71,1,0,0,40,5,0,0,115,12,0,0,0, + 0,4,22,1,8,1,10,1,10,1,10,3,122,28,80,97, + 116,104,70,105,110,100,101,114,46,105,110,118,97,108,105,100, + 97,116,101,95,99,97,99,104,101,115,99,2,0,0,0,0, + 0,0,0,0,0,0,0,3,0,0,0,9,0,0,0,67, + 0,0,0,115,82,0,0,0,116,0,106,1,100,1,117,1, + 114,28,116,0,106,1,115,28,116,2,160,3,100,2,116,4, + 161,2,1,0,116,0,106,1,68,0,93,42,125,2,122,14, + 124,2,124,1,131,1,87,0,2,0,1,0,83,0,4,0, + 116,5,121,74,1,0,1,0,1,0,89,0,113,34,89,0, + 113,34,48,0,113,34,100,1,83,0,41,3,122,46,83,101, + 97,114,99,104,32,115,121,115,46,112,97,116,104,95,104,111, + 111,107,115,32,102,111,114,32,97,32,102,105,110,100,101,114, + 32,102,111,114,32,39,112,97,116,104,39,46,78,122,23,115, + 121,115,46,112,97,116,104,95,104,111,111,107,115,32,105,115, + 32,101,109,112,116,121,41,6,114,21,0,0,0,218,10,112, + 97,116,104,95,104,111,111,107,115,114,98,0,0,0,114,99, + 0,0,0,114,162,0,0,0,114,141,0,0,0,41,3,114, + 216,0,0,0,114,65,0,0,0,90,4,104,111,111,107,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,11, + 95,112,97,116,104,95,104,111,111,107,115,53,5,0,0,115, + 16,0,0,0,0,3,16,1,12,1,10,1,2,1,14,1, + 12,1,12,2,122,22,80,97,116,104,70,105,110,100,101,114, + 46,95,112,97,116,104,95,104,111,111,107,115,99,2,0,0, + 0,0,0,0,0,0,0,0,0,3,0,0,0,8,0,0, + 0,67,0,0,0,115,100,0,0,0,124,1,100,1,107,2, + 114,42,122,12,116,0,160,1,161,0,125,1,87,0,110,20, + 4,0,116,2,121,40,1,0,1,0,1,0,89,0,100,2, + 83,0,48,0,122,14,116,3,106,4,124,1,25,0,125,2, + 87,0,110,38,4,0,116,5,121,94,1,0,1,0,1,0, + 124,0,160,6,124,1,161,1,125,2,124,2,116,3,106,4, + 124,1,60,0,89,0,110,2,48,0,124,2,83,0,41,3, + 122,210,71,101,116,32,116,104,101,32,102,105,110,100,101,114, + 32,102,111,114,32,116,104,101,32,112,97,116,104,32,101,110, + 116,114,121,32,102,114,111,109,32,115,121,115,46,112,97,116, 104,95,105,109,112,111,114,116,101,114,95,99,97,99,104,101, - 46,10,32,32,32,32,32,32,32,32,78,41,7,114,21,0, - 0,0,114,65,0,0,0,114,81,1,0,0,114,164,0,0, - 0,114,202,0,0,0,114,205,0,0,0,114,45,1,0,0, - 41,6,114,216,0,0,0,114,163,0,0,0,114,65,0,0, - 0,114,225,0,0,0,114,210,0,0,0,114,80,1,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 226,0,0,0,126,5,0,0,115,26,0,0,0,0,6,8, - 1,6,1,14,1,8,1,4,1,10,1,6,1,4,3,6, - 1,16,1,4,2,6,2,122,20,80,97,116,104,70,105,110, - 100,101,114,46,102,105,110,100,95,115,112,101,99,99,3,0, - 0,0,0,0,0,0,0,0,0,0,4,0,0,0,4,0, - 0,0,67,0,0,0,115,30,0,0,0,124,0,160,0,124, - 1,124,2,161,2,125,3,124,3,100,1,117,0,114,24,100, - 1,83,0,124,3,106,1,83,0,41,2,122,170,102,105,110, - 100,32,116,104,101,32,109,111,100,117,108,101,32,111,110,32, + 46,10,10,32,32,32,32,32,32,32,32,73,102,32,116,104, + 101,32,112,97,116,104,32,101,110,116,114,121,32,105,115,32, + 110,111,116,32,105,110,32,116,104,101,32,99,97,99,104,101, + 44,32,102,105,110,100,32,116,104,101,32,97,112,112,114,111, + 112,114,105,97,116,101,32,102,105,110,100,101,114,10,32,32, + 32,32,32,32,32,32,97,110,100,32,99,97,99,104,101,32, + 105,116,46,32,73,102,32,110,111,32,102,105,110,100,101,114, + 32,105,115,32,97,118,97,105,108,97,98,108,101,44,32,115, + 116,111,114,101,32,78,111,110,101,46,10,10,32,32,32,32, + 32,32,32,32,114,14,0,0,0,78,41,7,114,24,0,0, + 0,114,81,0,0,0,114,26,1,0,0,114,21,0,0,0, + 114,73,1,0,0,218,8,75,101,121,69,114,114,111,114,114, + 77,1,0,0,41,3,114,216,0,0,0,114,65,0,0,0, + 114,75,1,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,218,20,95,112,97,116,104,95,105,109,112,111, + 114,116,101,114,95,99,97,99,104,101,66,5,0,0,115,22, + 0,0,0,0,8,8,1,2,1,12,1,12,3,8,1,2, + 1,14,1,12,1,10,1,16,1,122,31,80,97,116,104,70, + 105,110,100,101,114,46,95,112,97,116,104,95,105,109,112,111, + 114,116,101,114,95,99,97,99,104,101,99,3,0,0,0,0, + 0,0,0,0,0,0,0,6,0,0,0,4,0,0,0,67, + 0,0,0,115,82,0,0,0,116,0,124,2,100,1,131,2, + 114,26,124,2,160,1,124,1,161,1,92,2,125,3,125,4, + 110,14,124,2,160,2,124,1,161,1,125,3,103,0,125,4, + 124,3,100,0,117,1,114,60,116,3,160,4,124,1,124,3, + 161,2,83,0,116,3,160,5,124,1,100,0,161,2,125,5, + 124,4,124,5,95,6,124,5,83,0,41,2,78,114,161,0, + 0,0,41,7,114,152,0,0,0,114,161,0,0,0,114,229, + 0,0,0,114,158,0,0,0,114,224,0,0,0,114,207,0, + 0,0,114,202,0,0,0,41,6,114,216,0,0,0,114,163, + 0,0,0,114,75,1,0,0,114,164,0,0,0,114,165,0, + 0,0,114,210,0,0,0,114,10,0,0,0,114,10,0,0, + 0,114,11,0,0,0,218,16,95,108,101,103,97,99,121,95, + 103,101,116,95,115,112,101,99,88,5,0,0,115,18,0,0, + 0,0,4,10,1,16,2,10,1,4,1,8,1,12,1,12, + 1,6,1,122,27,80,97,116,104,70,105,110,100,101,114,46, + 95,108,101,103,97,99,121,95,103,101,116,95,115,112,101,99, + 78,99,4,0,0,0,0,0,0,0,0,0,0,0,9,0, + 0,0,5,0,0,0,67,0,0,0,115,166,0,0,0,103, + 0,125,4,124,2,68,0,93,134,125,5,116,0,124,5,116, + 1,116,2,102,2,131,2,115,28,113,8,124,0,160,3,124, + 5,161,1,125,6,124,6,100,1,117,1,114,8,116,4,124, + 6,100,2,131,2,114,70,124,6,160,5,124,1,124,3,161, + 2,125,7,110,12,124,0,160,6,124,1,124,6,161,2,125, + 7,124,7,100,1,117,0,114,92,113,8,124,7,106,7,100, + 1,117,1,114,110,124,7,2,0,1,0,83,0,124,7,106, + 8,125,8,124,8,100,1,117,0,114,132,116,9,100,3,131, + 1,130,1,124,4,160,10,124,8,161,1,1,0,113,8,116, + 11,160,12,124,1,100,1,161,2,125,7,124,4,124,7,95, + 8,124,7,83,0,41,4,122,63,70,105,110,100,32,116,104, + 101,32,108,111,97,100,101,114,32,111,114,32,110,97,109,101, + 115,112,97,99,101,95,112,97,116,104,32,102,111,114,32,116, + 104,105,115,32,109,111,100,117,108,101,47,112,97,99,107,97, + 103,101,32,110,97,109,101,46,78,114,226,0,0,0,122,19, + 115,112,101,99,32,109,105,115,115,105,110,103,32,108,111,97, + 100,101,114,41,13,114,185,0,0,0,114,108,0,0,0,218, + 5,98,121,116,101,115,114,79,1,0,0,114,152,0,0,0, + 114,226,0,0,0,114,80,1,0,0,114,164,0,0,0,114, + 202,0,0,0,114,141,0,0,0,114,191,0,0,0,114,158, + 0,0,0,114,207,0,0,0,41,9,114,216,0,0,0,114, + 163,0,0,0,114,65,0,0,0,114,225,0,0,0,218,14, + 110,97,109,101,115,112,97,99,101,95,112,97,116,104,90,5, + 101,110,116,114,121,114,75,1,0,0,114,210,0,0,0,114, + 165,0,0,0,114,10,0,0,0,114,10,0,0,0,114,11, + 0,0,0,218,9,95,103,101,116,95,115,112,101,99,103,5, + 0,0,115,40,0,0,0,0,5,4,1,8,1,14,1,2, + 1,10,1,8,1,10,1,14,2,12,1,8,1,2,1,10, + 1,8,1,6,1,8,1,8,5,12,2,12,1,6,1,122, + 20,80,97,116,104,70,105,110,100,101,114,46,95,103,101,116, + 95,115,112,101,99,99,4,0,0,0,0,0,0,0,0,0, + 0,0,6,0,0,0,5,0,0,0,67,0,0,0,115,100, + 0,0,0,124,2,100,1,117,0,114,14,116,0,106,1,125, + 2,124,0,160,2,124,1,124,2,124,3,161,3,125,4,124, + 4,100,1,117,0,114,40,100,1,83,0,124,4,106,3,100, + 1,117,0,114,92,124,4,106,4,125,5,124,5,114,86,100, + 1,124,4,95,5,116,6,124,1,124,5,124,0,106,2,131, + 3,124,4,95,4,124,4,83,0,100,1,83,0,110,4,124, + 4,83,0,100,1,83,0,41,2,122,141,84,114,121,32,116, + 111,32,102,105,110,100,32,97,32,115,112,101,99,32,102,111, + 114,32,39,102,117,108,108,110,97,109,101,39,32,111,110,32, 115,121,115,46,112,97,116,104,32,111,114,32,39,112,97,116, - 104,39,32,98,97,115,101,100,32,111,110,32,115,121,115,46, - 112,97,116,104,95,104,111,111,107,115,32,97,110,100,10,32, - 32,32,32,32,32,32,32,115,121,115,46,112,97,116,104,95, + 104,39,46,10,10,32,32,32,32,32,32,32,32,84,104,101, + 32,115,101,97,114,99,104,32,105,115,32,98,97,115,101,100, + 32,111,110,32,115,121,115,46,112,97,116,104,95,104,111,111, + 107,115,32,97,110,100,32,115,121,115,46,112,97,116,104,95, 105,109,112,111,114,116,101,114,95,99,97,99,104,101,46,10, - 10,32,32,32,32,32,32,32,32,84,104,105,115,32,109,101, - 116,104,111,100,32,105,115,32,100,101,112,114,101,99,97,116, - 101,100,46,32,32,85,115,101,32,102,105,110,100,95,115,112, - 101,99,40,41,32,105,110,115,116,101,97,100,46,10,10,32, - 32,32,32,32,32,32,32,78,114,227,0,0,0,114,228,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,229,0,0,0,150,5,0,0,115,8,0,0,0,0, - 8,12,1,8,1,4,1,122,22,80,97,116,104,70,105,110, - 100,101,114,46,102,105,110,100,95,109,111,100,117,108,101,99, - 1,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0, - 4,0,0,0,79,0,0,0,115,28,0,0,0,100,1,100, - 2,108,0,109,1,125,3,1,0,124,3,106,2,124,1,105, - 0,124,2,164,1,142,1,83,0,41,3,97,32,1,0,0, - 10,32,32,32,32,32,32,32,32,70,105,110,100,32,100,105, - 115,116,114,105,98,117,116,105,111,110,115,46,10,10,32,32, - 32,32,32,32,32,32,82,101,116,117,114,110,32,97,110,32, - 105,116,101,114,97,98,108,101,32,111,102,32,97,108,108,32, - 68,105,115,116,114,105,98,117,116,105,111,110,32,105,110,115, - 116,97,110,99,101,115,32,99,97,112,97,98,108,101,32,111, - 102,10,32,32,32,32,32,32,32,32,108,111,97,100,105,110, - 103,32,116,104,101,32,109,101,116,97,100,97,116,97,32,102, - 111,114,32,112,97,99,107,97,103,101,115,32,109,97,116,99, - 104,105,110,103,32,96,96,99,111,110,116,101,120,116,46,110, - 97,109,101,96,96,10,32,32,32,32,32,32,32,32,40,111, - 114,32,97,108,108,32,110,97,109,101,115,32,105,102,32,96, - 96,78,111,110,101,96,96,32,105,110,100,105,99,97,116,101, - 100,41,32,97,108,111,110,103,32,116,104,101,32,112,97,116, - 104,115,32,105,110,32,116,104,101,32,108,105,115,116,10,32, - 32,32,32,32,32,32,32,111,102,32,100,105,114,101,99,116, - 111,114,105,101,115,32,96,96,99,111,110,116,101,120,116,46, - 112,97,116,104,96,96,46,10,32,32,32,32,32,32,32,32, - 114,0,0,0,0,41,1,218,18,77,101,116,97,100,97,116, - 97,80,97,116,104,70,105,110,100,101,114,41,3,90,18,105, - 109,112,111,114,116,108,105,98,46,109,101,116,97,100,97,116, - 97,114,82,1,0,0,218,18,102,105,110,100,95,100,105,115, - 116,114,105,98,117,116,105,111,110,115,41,4,114,216,0,0, - 0,114,143,0,0,0,114,144,0,0,0,114,82,1,0,0, + 32,32,32,32,32,32,32,32,78,41,7,114,21,0,0,0, + 114,65,0,0,0,114,83,1,0,0,114,164,0,0,0,114, + 202,0,0,0,114,205,0,0,0,114,45,1,0,0,41,6, + 114,216,0,0,0,114,163,0,0,0,114,65,0,0,0,114, + 225,0,0,0,114,210,0,0,0,114,82,1,0,0,114,10, + 0,0,0,114,10,0,0,0,114,11,0,0,0,114,226,0, + 0,0,135,5,0,0,115,26,0,0,0,0,6,8,1,6, + 1,14,1,8,1,4,1,10,1,6,1,4,3,6,1,16, + 1,4,2,6,2,122,20,80,97,116,104,70,105,110,100,101, + 114,46,102,105,110,100,95,115,112,101,99,99,3,0,0,0, + 0,0,0,0,0,0,0,0,4,0,0,0,4,0,0,0, + 67,0,0,0,115,30,0,0,0,124,0,160,0,124,1,124, + 2,161,2,125,3,124,3,100,1,117,0,114,24,100,1,83, + 0,124,3,106,1,83,0,41,2,122,170,102,105,110,100,32, + 116,104,101,32,109,111,100,117,108,101,32,111,110,32,115,121, + 115,46,112,97,116,104,32,111,114,32,39,112,97,116,104,39, + 32,98,97,115,101,100,32,111,110,32,115,121,115,46,112,97, + 116,104,95,104,111,111,107,115,32,97,110,100,10,32,32,32, + 32,32,32,32,32,115,121,115,46,112,97,116,104,95,105,109, + 112,111,114,116,101,114,95,99,97,99,104,101,46,10,10,32, + 32,32,32,32,32,32,32,84,104,105,115,32,109,101,116,104, + 111,100,32,105,115,32,100,101,112,114,101,99,97,116,101,100, + 46,32,32,85,115,101,32,102,105,110,100,95,115,112,101,99, + 40,41,32,105,110,115,116,101,97,100,46,10,10,32,32,32, + 32,32,32,32,32,78,114,227,0,0,0,114,228,0,0,0, 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 83,1,0,0,163,5,0,0,115,4,0,0,0,0,10,12, - 1,122,29,80,97,116,104,70,105,110,100,101,114,46,102,105, - 110,100,95,100,105,115,116,114,105,98,117,116,105,111,110,115, - 41,1,78,41,2,78,78,41,1,78,41,13,114,149,0,0, - 0,114,148,0,0,0,114,150,0,0,0,114,151,0,0,0, - 114,230,0,0,0,114,69,1,0,0,114,75,1,0,0,114, - 77,1,0,0,114,78,1,0,0,114,81,1,0,0,114,226, - 0,0,0,114,229,0,0,0,114,83,1,0,0,114,10,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,68,1,0,0,30,5,0,0,115,34,0,0,0,8, - 2,4,2,2,1,10,9,2,1,10,12,2,1,10,21,2, - 1,10,14,2,1,12,31,2,1,12,23,2,1,12,12,2, - 1,114,68,1,0,0,99,0,0,0,0,0,0,0,0,0, - 0,0,0,0,0,0,0,3,0,0,0,64,0,0,0,115, - 90,0,0,0,101,0,90,1,100,0,90,2,100,1,90,3, - 100,2,100,3,132,0,90,4,100,4,100,5,132,0,90,5, - 101,6,90,7,100,6,100,7,132,0,90,8,100,8,100,9, - 132,0,90,9,100,19,100,11,100,12,132,1,90,10,100,13, - 100,14,132,0,90,11,101,12,100,15,100,16,132,0,131,1, - 90,13,100,17,100,18,132,0,90,14,100,10,83,0,41,20, - 218,10,70,105,108,101,70,105,110,100,101,114,122,172,70,105, - 108,101,45,98,97,115,101,100,32,102,105,110,100,101,114,46, - 10,10,32,32,32,32,73,110,116,101,114,97,99,116,105,111, - 110,115,32,119,105,116,104,32,116,104,101,32,102,105,108,101, - 32,115,121,115,116,101,109,32,97,114,101,32,99,97,99,104, - 101,100,32,102,111,114,32,112,101,114,102,111,114,109,97,110, - 99,101,44,32,98,101,105,110,103,10,32,32,32,32,114,101, - 102,114,101,115,104,101,100,32,119,104,101,110,32,116,104,101, - 32,100,105,114,101,99,116,111,114,121,32,116,104,101,32,102, - 105,110,100,101,114,32,105,115,32,104,97,110,100,108,105,110, - 103,32,104,97,115,32,98,101,101,110,32,109,111,100,105,102, - 105,101,100,46,10,10,32,32,32,32,99,2,0,0,0,0, - 0,0,0,0,0,0,0,5,0,0,0,6,0,0,0,7, - 0,0,0,115,112,0,0,0,103,0,125,3,124,2,68,0, - 93,32,92,2,137,0,125,4,124,3,160,0,135,0,102,1, - 100,1,100,2,132,8,124,4,68,0,131,1,161,1,1,0, - 113,8,124,3,124,0,95,1,124,1,112,54,100,3,124,0, - 95,2,116,3,124,0,106,2,131,1,115,86,116,4,116,5, - 160,6,161,0,124,0,106,2,131,2,124,0,95,2,100,4, - 124,0,95,7,116,8,131,0,124,0,95,9,116,8,131,0, - 124,0,95,10,100,5,83,0,41,6,122,154,73,110,105,116, - 105,97,108,105,122,101,32,119,105,116,104,32,116,104,101,32, - 112,97,116,104,32,116,111,32,115,101,97,114,99,104,32,111, - 110,32,97,110,100,32,97,32,118,97,114,105,97,98,108,101, - 32,110,117,109,98,101,114,32,111,102,10,32,32,32,32,32, - 32,32,32,50,45,116,117,112,108,101,115,32,99,111,110,116, - 97,105,110,105,110,103,32,116,104,101,32,108,111,97,100,101, - 114,32,97,110,100,32,116,104,101,32,102,105,108,101,32,115, - 117,102,102,105,120,101,115,32,116,104,101,32,108,111,97,100, - 101,114,10,32,32,32,32,32,32,32,32,114,101,99,111,103, - 110,105,122,101,115,46,99,1,0,0,0,0,0,0,0,0, - 0,0,0,2,0,0,0,3,0,0,0,51,0,0,0,115, - 22,0,0,0,124,0,93,14,125,1,124,1,136,0,102,2, - 86,0,1,0,113,2,100,0,83,0,114,69,0,0,0,114, - 10,0,0,0,114,40,1,0,0,169,1,114,164,0,0,0, - 114,10,0,0,0,114,11,0,0,0,114,12,0,0,0,192, - 5,0,0,114,13,0,0,0,122,38,70,105,108,101,70,105, - 110,100,101,114,46,95,95,105,110,105,116,95,95,46,60,108, - 111,99,97,108,115,62,46,60,103,101,110,101,120,112,114,62, - 114,96,0,0,0,114,129,0,0,0,78,41,11,114,191,0, - 0,0,218,8,95,108,111,97,100,101,114,115,114,65,0,0, - 0,114,85,0,0,0,114,67,0,0,0,114,24,0,0,0, - 114,81,0,0,0,218,11,95,112,97,116,104,95,109,116,105, - 109,101,218,3,115,101,116,218,11,95,112,97,116,104,95,99, - 97,99,104,101,218,19,95,114,101,108,97,120,101,100,95,112, - 97,116,104,95,99,97,99,104,101,41,5,114,142,0,0,0, - 114,65,0,0,0,218,14,108,111,97,100,101,114,95,100,101, - 116,97,105,108,115,90,7,108,111,97,100,101,114,115,114,212, - 0,0,0,114,10,0,0,0,114,85,1,0,0,114,11,0, - 0,0,114,232,0,0,0,186,5,0,0,115,20,0,0,0, - 0,4,4,1,12,1,26,1,6,2,10,1,10,1,18,1, - 6,1,8,1,122,19,70,105,108,101,70,105,110,100,101,114, - 46,95,95,105,110,105,116,95,95,99,1,0,0,0,0,0, - 0,0,0,0,0,0,1,0,0,0,2,0,0,0,67,0, - 0,0,115,10,0,0,0,100,1,124,0,95,0,100,2,83, - 0,41,3,122,31,73,110,118,97,108,105,100,97,116,101,32, - 116,104,101,32,100,105,114,101,99,116,111,114,121,32,109,116, - 105,109,101,46,114,129,0,0,0,78,41,1,114,87,1,0, - 0,114,13,1,0,0,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,114,69,1,0,0,202,5,0,0,115,2, - 0,0,0,0,2,122,28,70,105,108,101,70,105,110,100,101, - 114,46,105,110,118,97,108,105,100,97,116,101,95,99,97,99, - 104,101,115,99,2,0,0,0,0,0,0,0,0,0,0,0, - 3,0,0,0,3,0,0,0,67,0,0,0,115,42,0,0, - 0,124,0,160,0,124,1,161,1,125,2,124,2,100,1,117, - 0,114,26,100,1,103,0,102,2,83,0,124,2,106,1,124, - 2,106,2,112,38,103,0,102,2,83,0,41,2,122,197,84, - 114,121,32,116,111,32,102,105,110,100,32,97,32,108,111,97, - 100,101,114,32,102,111,114,32,116,104,101,32,115,112,101,99, - 105,102,105,101,100,32,109,111,100,117,108,101,44,32,111,114, - 32,116,104,101,32,110,97,109,101,115,112,97,99,101,10,32, - 32,32,32,32,32,32,32,112,97,99,107,97,103,101,32,112, - 111,114,116,105,111,110,115,46,32,82,101,116,117,114,110,115, - 32,40,108,111,97,100,101,114,44,32,108,105,115,116,45,111, - 102,45,112,111,114,116,105,111,110,115,41,46,10,10,32,32, - 32,32,32,32,32,32,84,104,105,115,32,109,101,116,104,111, - 100,32,105,115,32,100,101,112,114,101,99,97,116,101,100,46, - 32,32,85,115,101,32,102,105,110,100,95,115,112,101,99,40, - 41,32,105,110,115,116,101,97,100,46,10,10,32,32,32,32, - 32,32,32,32,78,41,3,114,226,0,0,0,114,164,0,0, - 0,114,202,0,0,0,41,3,114,142,0,0,0,114,163,0, - 0,0,114,210,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,114,161,0,0,0,208,5,0,0,115, - 8,0,0,0,0,7,10,1,8,1,8,1,122,22,70,105, - 108,101,70,105,110,100,101,114,46,102,105,110,100,95,108,111, - 97,100,101,114,99,6,0,0,0,0,0,0,0,0,0,0, - 0,7,0,0,0,6,0,0,0,67,0,0,0,115,26,0, - 0,0,124,1,124,2,124,3,131,2,125,6,116,0,124,2, - 124,3,124,6,124,4,100,1,141,4,83,0,41,2,78,114, - 201,0,0,0,41,1,114,213,0,0,0,41,7,114,142,0, - 0,0,114,211,0,0,0,114,163,0,0,0,114,65,0,0, - 0,90,4,115,109,115,108,114,225,0,0,0,114,164,0,0, - 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 114,81,1,0,0,220,5,0,0,115,8,0,0,0,0,1, - 10,1,8,1,2,255,122,20,70,105,108,101,70,105,110,100, - 101,114,46,95,103,101,116,95,115,112,101,99,78,99,3,0, - 0,0,0,0,0,0,0,0,0,0,14,0,0,0,9,0, - 0,0,67,0,0,0,115,126,1,0,0,100,1,125,3,124, - 1,160,0,100,2,161,1,100,3,25,0,125,4,122,24,116, - 1,124,0,106,2,112,34,116,3,160,4,161,0,131,1,106, - 5,125,5,87,0,110,22,4,0,116,6,121,64,1,0,1, - 0,1,0,100,4,125,5,89,0,110,2,48,0,124,5,124, - 0,106,7,107,3,114,90,124,0,160,8,161,0,1,0,124, - 5,124,0,95,7,116,9,131,0,114,112,124,0,106,10,125, - 6,124,4,160,11,161,0,125,7,110,10,124,0,106,12,125, - 6,124,4,125,7,124,7,124,6,118,0,114,216,116,13,124, - 0,106,2,124,4,131,2,125,8,124,0,106,14,68,0,93, - 58,92,2,125,9,125,10,100,5,124,9,23,0,125,11,116, - 13,124,8,124,11,131,2,125,12,116,15,124,12,131,1,114, - 148,124,0,160,16,124,10,124,1,124,12,124,8,103,1,124, - 2,161,5,2,0,1,0,83,0,113,148,116,17,124,8,131, - 1,125,3,124,0,106,14,68,0,93,112,92,2,125,9,125, - 10,122,20,116,13,124,0,106,2,124,4,124,9,23,0,131, - 2,125,12,87,0,110,24,4,0,116,18,144,1,121,18,1, - 0,1,0,1,0,89,0,1,0,100,6,83,0,48,0,116, - 19,106,20,100,7,124,12,100,3,100,8,141,3,1,0,124, - 7,124,9,23,0,124,6,118,0,114,222,116,15,124,12,131, - 1,114,222,124,0,160,16,124,10,124,1,124,12,100,6,124, - 2,161,5,2,0,1,0,83,0,113,222,124,3,144,1,114, - 122,116,19,160,20,100,9,124,8,161,2,1,0,116,19,160, - 21,124,1,100,6,161,2,125,13,124,8,103,1,124,13,95, - 22,124,13,83,0,100,6,83,0,41,10,122,111,84,114,121, - 32,116,111,32,102,105,110,100,32,97,32,115,112,101,99,32, - 102,111,114,32,116,104,101,32,115,112,101,99,105,102,105,101, - 100,32,109,111,100,117,108,101,46,10,10,32,32,32,32,32, - 32,32,32,82,101,116,117,114,110,115,32,116,104,101,32,109, - 97,116,99,104,105,110,103,32,115,112,101,99,44,32,111,114, - 32,78,111,110,101,32,105,102,32,110,111,116,32,102,111,117, - 110,100,46,10,32,32,32,32,32,32,32,32,70,114,96,0, - 0,0,114,45,0,0,0,114,129,0,0,0,114,232,0,0, - 0,78,122,9,116,114,121,105,110,103,32,123,125,41,1,90, - 9,118,101,114,98,111,115,105,116,121,122,25,112,111,115,115, - 105,98,108,101,32,110,97,109,101,115,112,97,99,101,32,102, - 111,114,32,123,125,41,23,114,103,0,0,0,114,75,0,0, - 0,114,65,0,0,0,114,24,0,0,0,114,81,0,0,0, - 114,33,1,0,0,114,76,0,0,0,114,87,1,0,0,218, - 11,95,102,105,108,108,95,99,97,99,104,101,114,27,0,0, - 0,114,90,1,0,0,114,130,0,0,0,114,89,1,0,0, - 114,67,0,0,0,114,86,1,0,0,114,80,0,0,0,114, - 81,1,0,0,114,82,0,0,0,114,110,0,0,0,114,158, - 0,0,0,114,173,0,0,0,114,207,0,0,0,114,202,0, - 0,0,41,14,114,142,0,0,0,114,163,0,0,0,114,225, - 0,0,0,90,12,105,115,95,110,97,109,101,115,112,97,99, - 101,90,11,116,97,105,108,95,109,111,100,117,108,101,114,193, - 0,0,0,90,5,99,97,99,104,101,90,12,99,97,99,104, - 101,95,109,111,100,117,108,101,90,9,98,97,115,101,95,112, - 97,116,104,114,41,1,0,0,114,211,0,0,0,90,13,105, - 110,105,116,95,102,105,108,101,110,97,109,101,90,9,102,117, - 108,108,95,112,97,116,104,114,210,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,226,0,0,0, - 225,5,0,0,115,78,0,0,0,0,5,4,1,14,1,2, - 1,24,1,12,1,10,1,10,1,8,1,6,2,6,1,6, - 1,10,2,6,1,4,2,8,1,12,1,14,1,8,1,10, - 1,8,1,26,4,8,2,14,1,2,1,20,1,14,1,10, - 1,16,1,12,1,8,1,10,1,4,255,10,2,6,1,12, - 1,12,1,8,1,4,1,122,20,70,105,108,101,70,105,110, - 100,101,114,46,102,105,110,100,95,115,112,101,99,99,1,0, - 0,0,0,0,0,0,0,0,0,0,9,0,0,0,10,0, - 0,0,67,0,0,0,115,188,0,0,0,124,0,106,0,125, - 1,122,22,116,1,160,2,124,1,112,22,116,1,160,3,161, - 0,161,1,125,2,87,0,110,28,4,0,116,4,116,5,116, - 6,102,3,121,56,1,0,1,0,1,0,103,0,125,2,89, - 0,110,2,48,0,116,7,106,8,160,9,100,1,161,1,115, - 82,116,10,124,2,131,1,124,0,95,11,110,74,116,10,131, - 0,125,3,124,2,68,0,93,56,125,4,124,4,160,12,100, - 2,161,1,92,3,125,5,125,6,125,7,124,6,114,134,100, - 3,160,13,124,5,124,7,160,14,161,0,161,2,125,8,110, - 4,124,5,125,8,124,3,160,15,124,8,161,1,1,0,113, - 92,124,3,124,0,95,11,116,7,106,8,160,9,116,16,161, - 1,114,184,100,4,100,5,132,0,124,2,68,0,131,1,124, - 0,95,17,100,6,83,0,41,7,122,68,70,105,108,108,32, - 116,104,101,32,99,97,99,104,101,32,111,102,32,112,111,116, - 101,110,116,105,97,108,32,109,111,100,117,108,101,115,32,97, - 110,100,32,112,97,99,107,97,103,101,115,32,102,111,114,32, - 116,104,105,115,32,100,105,114,101,99,116,111,114,121,46,114, - 20,0,0,0,114,96,0,0,0,114,87,0,0,0,99,1, - 0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,4, - 0,0,0,83,0,0,0,115,20,0,0,0,104,0,124,0, - 93,12,125,1,124,1,160,0,161,0,146,2,113,4,83,0, - 114,10,0,0,0,41,1,114,130,0,0,0,41,2,114,8, - 0,0,0,90,2,102,110,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,114,19,0,0,0,49,6,0,0,114, - 13,0,0,0,122,41,70,105,108,101,70,105,110,100,101,114, - 46,95,102,105,108,108,95,99,97,99,104,101,46,60,108,111, - 99,97,108,115,62,46,60,115,101,116,99,111,109,112,62,78, - 41,18,114,65,0,0,0,114,24,0,0,0,114,30,1,0, - 0,114,81,0,0,0,114,26,1,0,0,218,15,80,101,114, - 109,105,115,115,105,111,110,69,114,114,111,114,218,18,78,111, - 116,65,68,105,114,101,99,116,111,114,121,69,114,114,111,114, - 114,21,0,0,0,114,28,0,0,0,114,29,0,0,0,114, - 88,1,0,0,114,89,1,0,0,114,125,0,0,0,114,88, - 0,0,0,114,130,0,0,0,218,3,97,100,100,114,30,0, - 0,0,114,90,1,0,0,41,9,114,142,0,0,0,114,65, - 0,0,0,114,31,1,0,0,90,21,108,111,119,101,114,95, - 115,117,102,102,105,120,95,99,111,110,116,101,110,116,115,114, - 64,1,0,0,114,140,0,0,0,114,52,1,0,0,114,41, - 1,0,0,90,8,110,101,119,95,110,97,109,101,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,114,92,1,0, - 0,20,6,0,0,115,34,0,0,0,0,2,6,1,2,1, - 22,1,18,3,10,3,12,1,12,7,6,1,8,1,16,1, - 4,1,18,2,4,1,12,1,6,1,12,1,122,22,70,105, - 108,101,70,105,110,100,101,114,46,95,102,105,108,108,95,99, - 97,99,104,101,99,1,0,0,0,0,0,0,0,0,0,0, - 0,3,0,0,0,3,0,0,0,7,0,0,0,115,18,0, - 0,0,135,0,135,1,102,2,100,1,100,2,132,8,125,2, - 124,2,83,0,41,3,97,20,1,0,0,65,32,99,108,97, - 115,115,32,109,101,116,104,111,100,32,119,104,105,99,104,32, - 114,101,116,117,114,110,115,32,97,32,99,108,111,115,117,114, - 101,32,116,111,32,117,115,101,32,111,110,32,115,121,115,46, - 112,97,116,104,95,104,111,111,107,10,32,32,32,32,32,32, - 32,32,119,104,105,99,104,32,119,105,108,108,32,114,101,116, - 117,114,110,32,97,110,32,105,110,115,116,97,110,99,101,32, - 117,115,105,110,103,32,116,104,101,32,115,112,101,99,105,102, - 105,101,100,32,108,111,97,100,101,114,115,32,97,110,100,32, - 116,104,101,32,112,97,116,104,10,32,32,32,32,32,32,32, - 32,99,97,108,108,101,100,32,111,110,32,116,104,101,32,99, - 108,111,115,117,114,101,46,10,10,32,32,32,32,32,32,32, - 32,73,102,32,116,104,101,32,112,97,116,104,32,99,97,108, - 108,101,100,32,111,110,32,116,104,101,32,99,108,111,115,117, - 114,101,32,105,115,32,110,111,116,32,97,32,100,105,114,101, - 99,116,111,114,121,44,32,73,109,112,111,114,116,69,114,114, - 111,114,32,105,115,10,32,32,32,32,32,32,32,32,114,97, - 105,115,101,100,46,10,10,32,32,32,32,32,32,32,32,99, - 1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0, - 4,0,0,0,19,0,0,0,115,36,0,0,0,116,0,124, - 0,131,1,115,20,116,1,100,1,124,0,100,2,141,2,130, - 1,136,0,124,0,103,1,136,1,162,1,82,0,142,0,83, - 0,41,3,122,45,80,97,116,104,32,104,111,111,107,32,102, - 111,114,32,105,109,112,111,114,116,108,105,98,46,109,97,99, - 104,105,110,101,114,121,46,70,105,108,101,70,105,110,100,101, - 114,46,122,30,111,110,108,121,32,100,105,114,101,99,116,111, - 114,105,101,115,32,97,114,101,32,115,117,112,112,111,114,116, - 101,100,114,71,0,0,0,41,2,114,82,0,0,0,114,141, - 0,0,0,114,71,0,0,0,169,2,114,216,0,0,0,114, - 91,1,0,0,114,10,0,0,0,114,11,0,0,0,218,24, - 112,97,116,104,95,104,111,111,107,95,102,111,114,95,70,105, - 108,101,70,105,110,100,101,114,61,6,0,0,115,6,0,0, - 0,0,2,8,1,12,1,122,54,70,105,108,101,70,105,110, - 100,101,114,46,112,97,116,104,95,104,111,111,107,46,60,108, - 111,99,97,108,115,62,46,112,97,116,104,95,104,111,111,107, - 95,102,111,114,95,70,105,108,101,70,105,110,100,101,114,114, - 10,0,0,0,41,3,114,216,0,0,0,114,91,1,0,0, - 114,97,1,0,0,114,10,0,0,0,114,96,1,0,0,114, - 11,0,0,0,218,9,112,97,116,104,95,104,111,111,107,51, - 6,0,0,115,4,0,0,0,0,10,14,6,122,20,70,105, - 108,101,70,105,110,100,101,114,46,112,97,116,104,95,104,111, - 111,107,99,1,0,0,0,0,0,0,0,0,0,0,0,1, - 0,0,0,3,0,0,0,67,0,0,0,115,12,0,0,0, - 100,1,160,0,124,0,106,1,161,1,83,0,41,2,78,122, - 16,70,105,108,101,70,105,110,100,101,114,40,123,33,114,125, - 41,41,2,114,88,0,0,0,114,65,0,0,0,114,13,1, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,62,1,0,0,69,6,0,0,115,2,0,0,0,0, - 1,122,19,70,105,108,101,70,105,110,100,101,114,46,95,95, - 114,101,112,114,95,95,41,1,78,41,15,114,149,0,0,0, - 114,148,0,0,0,114,150,0,0,0,114,151,0,0,0,114, - 232,0,0,0,114,69,1,0,0,114,167,0,0,0,114,229, - 0,0,0,114,161,0,0,0,114,81,1,0,0,114,226,0, - 0,0,114,92,1,0,0,114,230,0,0,0,114,98,1,0, - 0,114,62,1,0,0,114,10,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,114,84,1,0,0,177, - 5,0,0,115,22,0,0,0,8,2,4,7,8,16,8,4, - 4,2,8,12,8,5,10,51,8,31,2,1,10,17,114,84, - 1,0,0,99,4,0,0,0,0,0,0,0,0,0,0,0, - 6,0,0,0,8,0,0,0,67,0,0,0,115,144,0,0, - 0,124,0,160,0,100,1,161,1,125,4,124,0,160,0,100, - 2,161,1,125,5,124,4,115,66,124,5,114,36,124,5,106, - 1,125,4,110,30,124,2,124,3,107,2,114,56,116,2,124, - 1,124,2,131,2,125,4,110,10,116,3,124,1,124,2,131, - 2,125,4,124,5,115,84,116,4,124,1,124,2,124,4,100, - 3,141,3,125,5,122,36,124,5,124,0,100,2,60,0,124, - 4,124,0,100,1,60,0,124,2,124,0,100,4,60,0,124, - 3,124,0,100,5,60,0,87,0,110,18,4,0,116,5,121, - 138,1,0,1,0,1,0,89,0,110,2,48,0,100,0,83, - 0,41,6,78,218,10,95,95,108,111,97,100,101,114,95,95, - 218,8,95,95,115,112,101,99,95,95,114,85,1,0,0,90, - 8,95,95,102,105,108,101,95,95,90,10,95,95,99,97,99, - 104,101,100,95,95,41,6,218,3,103,101,116,114,164,0,0, - 0,114,39,1,0,0,114,32,1,0,0,114,213,0,0,0, - 218,9,69,120,99,101,112,116,105,111,110,41,6,90,2,110, - 115,114,140,0,0,0,90,8,112,97,116,104,110,97,109,101, - 90,9,99,112,97,116,104,110,97,109,101,114,164,0,0,0, + 229,0,0,0,159,5,0,0,115,8,0,0,0,0,8,12, + 1,8,1,4,1,122,22,80,97,116,104,70,105,110,100,101, + 114,46,102,105,110,100,95,109,111,100,117,108,101,99,1,0, + 0,0,0,0,0,0,0,0,0,0,4,0,0,0,4,0, + 0,0,79,0,0,0,115,28,0,0,0,100,1,100,2,108, + 0,109,1,125,3,1,0,124,3,106,2,124,1,105,0,124, + 2,164,1,142,1,83,0,41,3,97,32,1,0,0,10,32, + 32,32,32,32,32,32,32,70,105,110,100,32,100,105,115,116, + 114,105,98,117,116,105,111,110,115,46,10,10,32,32,32,32, + 32,32,32,32,82,101,116,117,114,110,32,97,110,32,105,116, + 101,114,97,98,108,101,32,111,102,32,97,108,108,32,68,105, + 115,116,114,105,98,117,116,105,111,110,32,105,110,115,116,97, + 110,99,101,115,32,99,97,112,97,98,108,101,32,111,102,10, + 32,32,32,32,32,32,32,32,108,111,97,100,105,110,103,32, + 116,104,101,32,109,101,116,97,100,97,116,97,32,102,111,114, + 32,112,97,99,107,97,103,101,115,32,109,97,116,99,104,105, + 110,103,32,96,96,99,111,110,116,101,120,116,46,110,97,109, + 101,96,96,10,32,32,32,32,32,32,32,32,40,111,114,32, + 97,108,108,32,110,97,109,101,115,32,105,102,32,96,96,78, + 111,110,101,96,96,32,105,110,100,105,99,97,116,101,100,41, + 32,97,108,111,110,103,32,116,104,101,32,112,97,116,104,115, + 32,105,110,32,116,104,101,32,108,105,115,116,10,32,32,32, + 32,32,32,32,32,111,102,32,100,105,114,101,99,116,111,114, + 105,101,115,32,96,96,99,111,110,116,101,120,116,46,112,97, + 116,104,96,96,46,10,32,32,32,32,32,32,32,32,114,0, + 0,0,0,41,1,218,18,77,101,116,97,100,97,116,97,80, + 97,116,104,70,105,110,100,101,114,41,3,90,18,105,109,112, + 111,114,116,108,105,98,46,109,101,116,97,100,97,116,97,114, + 84,1,0,0,218,18,102,105,110,100,95,100,105,115,116,114, + 105,98,117,116,105,111,110,115,41,4,114,216,0,0,0,114, + 143,0,0,0,114,144,0,0,0,114,84,1,0,0,114,10, + 0,0,0,114,10,0,0,0,114,11,0,0,0,114,85,1, + 0,0,172,5,0,0,115,4,0,0,0,0,10,12,1,122, + 29,80,97,116,104,70,105,110,100,101,114,46,102,105,110,100, + 95,100,105,115,116,114,105,98,117,116,105,111,110,115,41,1, + 78,41,2,78,78,41,1,78,41,13,114,149,0,0,0,114, + 148,0,0,0,114,150,0,0,0,114,151,0,0,0,114,230, + 0,0,0,114,71,1,0,0,114,77,1,0,0,114,79,1, + 0,0,114,80,1,0,0,114,83,1,0,0,114,226,0,0, + 0,114,229,0,0,0,114,85,1,0,0,114,10,0,0,0, + 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, + 70,1,0,0,36,5,0,0,115,34,0,0,0,8,2,4, + 2,2,1,10,12,2,1,10,12,2,1,10,21,2,1,10, + 14,2,1,12,31,2,1,12,23,2,1,12,12,2,1,114, + 70,1,0,0,99,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,3,0,0,0,64,0,0,0,115,90,0, + 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, + 100,3,132,0,90,4,100,4,100,5,132,0,90,5,101,6, + 90,7,100,6,100,7,132,0,90,8,100,8,100,9,132,0, + 90,9,100,19,100,11,100,12,132,1,90,10,100,13,100,14, + 132,0,90,11,101,12,100,15,100,16,132,0,131,1,90,13, + 100,17,100,18,132,0,90,14,100,10,83,0,41,20,218,10, + 70,105,108,101,70,105,110,100,101,114,122,172,70,105,108,101, + 45,98,97,115,101,100,32,102,105,110,100,101,114,46,10,10, + 32,32,32,32,73,110,116,101,114,97,99,116,105,111,110,115, + 32,119,105,116,104,32,116,104,101,32,102,105,108,101,32,115, + 121,115,116,101,109,32,97,114,101,32,99,97,99,104,101,100, + 32,102,111,114,32,112,101,114,102,111,114,109,97,110,99,101, + 44,32,98,101,105,110,103,10,32,32,32,32,114,101,102,114, + 101,115,104,101,100,32,119,104,101,110,32,116,104,101,32,100, + 105,114,101,99,116,111,114,121,32,116,104,101,32,102,105,110, + 100,101,114,32,105,115,32,104,97,110,100,108,105,110,103,32, + 104,97,115,32,98,101,101,110,32,109,111,100,105,102,105,101, + 100,46,10,10,32,32,32,32,99,2,0,0,0,0,0,0, + 0,0,0,0,0,5,0,0,0,6,0,0,0,7,0,0, + 0,115,112,0,0,0,103,0,125,3,124,2,68,0,93,32, + 92,2,137,0,125,4,124,3,160,0,135,0,102,1,100,1, + 100,2,132,8,124,4,68,0,131,1,161,1,1,0,113,8, + 124,3,124,0,95,1,124,1,112,54,100,3,124,0,95,2, + 116,3,124,0,106,2,131,1,115,86,116,4,116,5,160,6, + 161,0,124,0,106,2,131,2,124,0,95,2,100,4,124,0, + 95,7,116,8,131,0,124,0,95,9,116,8,131,0,124,0, + 95,10,100,5,83,0,41,6,122,154,73,110,105,116,105,97, + 108,105,122,101,32,119,105,116,104,32,116,104,101,32,112,97, + 116,104,32,116,111,32,115,101,97,114,99,104,32,111,110,32, + 97,110,100,32,97,32,118,97,114,105,97,98,108,101,32,110, + 117,109,98,101,114,32,111,102,10,32,32,32,32,32,32,32, + 32,50,45,116,117,112,108,101,115,32,99,111,110,116,97,105, + 110,105,110,103,32,116,104,101,32,108,111,97,100,101,114,32, + 97,110,100,32,116,104,101,32,102,105,108,101,32,115,117,102, + 102,105,120,101,115,32,116,104,101,32,108,111,97,100,101,114, + 10,32,32,32,32,32,32,32,32,114,101,99,111,103,110,105, + 122,101,115,46,99,1,0,0,0,0,0,0,0,0,0,0, + 0,2,0,0,0,3,0,0,0,51,0,0,0,115,22,0, + 0,0,124,0,93,14,125,1,124,1,136,0,102,2,86,0, + 1,0,113,2,100,0,83,0,114,69,0,0,0,114,10,0, + 0,0,114,40,1,0,0,169,1,114,164,0,0,0,114,10, + 0,0,0,114,11,0,0,0,114,12,0,0,0,201,5,0, + 0,114,13,0,0,0,122,38,70,105,108,101,70,105,110,100, + 101,114,46,95,95,105,110,105,116,95,95,46,60,108,111,99, + 97,108,115,62,46,60,103,101,110,101,120,112,114,62,114,96, + 0,0,0,114,129,0,0,0,78,41,11,114,191,0,0,0, + 218,8,95,108,111,97,100,101,114,115,114,65,0,0,0,114, + 85,0,0,0,114,67,0,0,0,114,24,0,0,0,114,81, + 0,0,0,218,11,95,112,97,116,104,95,109,116,105,109,101, + 218,3,115,101,116,218,11,95,112,97,116,104,95,99,97,99, + 104,101,218,19,95,114,101,108,97,120,101,100,95,112,97,116, + 104,95,99,97,99,104,101,41,5,114,142,0,0,0,114,65, + 0,0,0,218,14,108,111,97,100,101,114,95,100,101,116,97, + 105,108,115,90,7,108,111,97,100,101,114,115,114,212,0,0, + 0,114,10,0,0,0,114,87,1,0,0,114,11,0,0,0, + 114,232,0,0,0,195,5,0,0,115,20,0,0,0,0,4, + 4,1,12,1,26,1,6,2,10,1,10,1,18,1,6,1, + 8,1,122,19,70,105,108,101,70,105,110,100,101,114,46,95, + 95,105,110,105,116,95,95,99,1,0,0,0,0,0,0,0, + 0,0,0,0,1,0,0,0,2,0,0,0,67,0,0,0, + 115,10,0,0,0,100,1,124,0,95,0,100,2,83,0,41, + 3,122,31,73,110,118,97,108,105,100,97,116,101,32,116,104, + 101,32,100,105,114,101,99,116,111,114,121,32,109,116,105,109, + 101,46,114,129,0,0,0,78,41,1,114,89,1,0,0,114, + 13,1,0,0,114,10,0,0,0,114,10,0,0,0,114,11, + 0,0,0,114,71,1,0,0,211,5,0,0,115,2,0,0, + 0,0,2,122,28,70,105,108,101,70,105,110,100,101,114,46, + 105,110,118,97,108,105,100,97,116,101,95,99,97,99,104,101, + 115,99,2,0,0,0,0,0,0,0,0,0,0,0,3,0, + 0,0,3,0,0,0,67,0,0,0,115,42,0,0,0,124, + 0,160,0,124,1,161,1,125,2,124,2,100,1,117,0,114, + 26,100,1,103,0,102,2,83,0,124,2,106,1,124,2,106, + 2,112,38,103,0,102,2,83,0,41,2,122,197,84,114,121, + 32,116,111,32,102,105,110,100,32,97,32,108,111,97,100,101, + 114,32,102,111,114,32,116,104,101,32,115,112,101,99,105,102, + 105,101,100,32,109,111,100,117,108,101,44,32,111,114,32,116, + 104,101,32,110,97,109,101,115,112,97,99,101,10,32,32,32, + 32,32,32,32,32,112,97,99,107,97,103,101,32,112,111,114, + 116,105,111,110,115,46,32,82,101,116,117,114,110,115,32,40, + 108,111,97,100,101,114,44,32,108,105,115,116,45,111,102,45, + 112,111,114,116,105,111,110,115,41,46,10,10,32,32,32,32, + 32,32,32,32,84,104,105,115,32,109,101,116,104,111,100,32, + 105,115,32,100,101,112,114,101,99,97,116,101,100,46,32,32, + 85,115,101,32,102,105,110,100,95,115,112,101,99,40,41,32, + 105,110,115,116,101,97,100,46,10,10,32,32,32,32,32,32, + 32,32,78,41,3,114,226,0,0,0,114,164,0,0,0,114, + 202,0,0,0,41,3,114,142,0,0,0,114,163,0,0,0, 114,210,0,0,0,114,10,0,0,0,114,10,0,0,0,114, - 11,0,0,0,218,14,95,102,105,120,95,117,112,95,109,111, - 100,117,108,101,75,6,0,0,115,34,0,0,0,0,2,10, - 1,10,1,4,1,4,1,8,1,8,1,12,2,10,1,4, - 1,14,1,2,1,8,1,8,1,8,1,12,1,12,2,114, - 103,1,0,0,99,0,0,0,0,0,0,0,0,0,0,0, - 0,3,0,0,0,3,0,0,0,67,0,0,0,115,38,0, - 0,0,116,0,116,1,160,2,161,0,102,2,125,0,116,3, - 116,4,102,2,125,1,116,5,116,6,102,2,125,2,124,0, - 124,1,124,2,103,3,83,0,41,1,122,95,82,101,116,117, - 114,110,115,32,97,32,108,105,115,116,32,111,102,32,102,105, - 108,101,45,98,97,115,101,100,32,109,111,100,117,108,101,32, - 108,111,97,100,101,114,115,46,10,10,32,32,32,32,69,97, - 99,104,32,105,116,101,109,32,105,115,32,97,32,116,117,112, - 108,101,32,40,108,111,97,100,101,114,44,32,115,117,102,102, - 105,120,101,115,41,46,10,32,32,32,32,41,7,114,19,1, - 0,0,114,187,0,0,0,218,18,101,120,116,101,110,115,105, - 111,110,95,115,117,102,102,105,120,101,115,114,32,1,0,0, - 114,126,0,0,0,114,39,1,0,0,114,112,0,0,0,41, - 3,90,10,101,120,116,101,110,115,105,111,110,115,90,6,115, - 111,117,114,99,101,90,8,98,121,116,101,99,111,100,101,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,208, - 0,0,0,98,6,0,0,115,8,0,0,0,0,5,12,1, - 8,1,8,1,114,208,0,0,0,99,1,0,0,0,0,0, - 0,0,0,0,0,0,10,0,0,0,9,0,0,0,67,0, - 0,0,115,132,1,0,0,124,0,97,0,116,0,106,1,97, - 1,116,0,106,2,97,2,116,1,106,3,116,4,25,0,125, - 1,100,1,100,2,103,1,102,2,100,3,100,4,100,2,103, - 2,102,2,102,2,125,2,124,2,68,0,93,108,92,2,125, - 3,125,4,116,5,100,5,100,6,132,0,124,4,68,0,131, - 1,131,1,115,82,74,0,130,1,124,4,100,7,25,0,125, - 5,124,3,116,1,106,3,118,0,114,116,116,1,106,3,124, - 3,25,0,125,6,1,0,113,170,113,52,122,20,116,0,160, - 6,124,3,161,1,125,6,87,0,1,0,113,170,87,0,113, - 52,4,0,116,7,121,158,1,0,1,0,1,0,89,0,113, - 52,89,0,113,52,48,0,113,52,116,7,100,8,131,1,130, - 1,116,8,124,1,100,9,124,6,131,3,1,0,116,8,124, - 1,100,10,124,5,131,3,1,0,116,8,124,1,100,11,100, - 12,160,9,124,4,161,1,131,3,1,0,116,8,124,1,100, - 13,100,14,100,15,132,0,124,4,68,0,131,1,131,3,1, - 0,103,0,100,16,162,1,125,7,124,3,100,3,107,2,144, - 1,114,6,124,7,160,10,100,17,161,1,1,0,124,7,68, - 0,93,52,125,8,124,8,116,1,106,3,118,1,144,1,114, - 38,116,0,160,6,124,8,161,1,125,9,110,10,116,1,106, - 3,124,8,25,0,125,9,116,8,124,1,124,8,124,9,131, - 3,1,0,144,1,113,10,116,8,124,1,100,18,116,11,131, - 0,131,3,1,0,116,12,160,13,116,2,160,14,161,0,161, - 1,1,0,124,3,100,3,107,2,144,1,114,128,116,15,160, - 10,100,19,161,1,1,0,100,20,116,12,118,0,144,1,114, - 128,100,21,116,16,95,17,100,22,83,0,41,23,122,205,83, - 101,116,117,112,32,116,104,101,32,112,97,116,104,45,98,97, - 115,101,100,32,105,109,112,111,114,116,101,114,115,32,102,111, - 114,32,105,109,112,111,114,116,108,105,98,32,98,121,32,105, - 109,112,111,114,116,105,110,103,32,110,101,101,100,101,100,10, - 32,32,32,32,98,117,105,108,116,45,105,110,32,109,111,100, - 117,108,101,115,32,97,110,100,32,105,110,106,101,99,116,105, - 110,103,32,116,104,101,109,32,105,110,116,111,32,116,104,101, - 32,103,108,111,98,97,108,32,110,97,109,101,115,112,97,99, - 101,46,10,10,32,32,32,32,79,116,104,101,114,32,99,111, - 109,112,111,110,101,110,116,115,32,97,114,101,32,101,120,116, - 114,97,99,116,101,100,32,102,114,111,109,32,116,104,101,32, - 99,111,114,101,32,98,111,111,116,115,116,114,97,112,32,109, - 111,100,117,108,101,46,10,10,32,32,32,32,218,5,112,111, - 115,105,120,114,2,0,0,0,218,2,110,116,114,1,0,0, - 0,99,1,0,0,0,0,0,0,0,0,0,0,0,2,0, - 0,0,3,0,0,0,115,0,0,0,115,26,0,0,0,124, - 0,93,18,125,1,116,0,124,1,131,1,100,0,107,2,86, - 0,1,0,113,2,100,1,83,0,114,3,0,0,0,114,5, - 0,0,0,114,7,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,114,12,0,0,0,127,6,0,0, - 114,13,0,0,0,122,25,95,115,101,116,117,112,46,60,108, - 111,99,97,108,115,62,46,60,103,101,110,101,120,112,114,62, - 114,0,0,0,0,122,30,105,109,112,111,114,116,108,105,98, - 32,114,101,113,117,105,114,101,115,32,112,111,115,105,120,32, - 111,114,32,110,116,114,24,0,0,0,114,59,0,0,0,114, - 50,0,0,0,114,14,0,0,0,218,20,95,112,97,116,104, - 115,101,112,115,95,119,105,116,104,95,99,111,108,111,110,99, + 11,0,0,0,114,161,0,0,0,217,5,0,0,115,8,0, + 0,0,0,7,10,1,8,1,8,1,122,22,70,105,108,101, + 70,105,110,100,101,114,46,102,105,110,100,95,108,111,97,100, + 101,114,99,6,0,0,0,0,0,0,0,0,0,0,0,7, + 0,0,0,6,0,0,0,67,0,0,0,115,26,0,0,0, + 124,1,124,2,124,3,131,2,125,6,116,0,124,2,124,3, + 124,6,124,4,100,1,141,4,83,0,41,2,78,114,201,0, + 0,0,41,1,114,213,0,0,0,41,7,114,142,0,0,0, + 114,211,0,0,0,114,163,0,0,0,114,65,0,0,0,90, + 4,115,109,115,108,114,225,0,0,0,114,164,0,0,0,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,83, + 1,0,0,229,5,0,0,115,8,0,0,0,0,1,10,1, + 8,1,2,255,122,20,70,105,108,101,70,105,110,100,101,114, + 46,95,103,101,116,95,115,112,101,99,78,99,3,0,0,0, + 0,0,0,0,0,0,0,0,14,0,0,0,9,0,0,0, + 67,0,0,0,115,126,1,0,0,100,1,125,3,124,1,160, + 0,100,2,161,1,100,3,25,0,125,4,122,24,116,1,124, + 0,106,2,112,34,116,3,160,4,161,0,131,1,106,5,125, + 5,87,0,110,22,4,0,116,6,121,64,1,0,1,0,1, + 0,100,4,125,5,89,0,110,2,48,0,124,5,124,0,106, + 7,107,3,114,90,124,0,160,8,161,0,1,0,124,5,124, + 0,95,7,116,9,131,0,114,112,124,0,106,10,125,6,124, + 4,160,11,161,0,125,7,110,10,124,0,106,12,125,6,124, + 4,125,7,124,7,124,6,118,0,114,216,116,13,124,0,106, + 2,124,4,131,2,125,8,124,0,106,14,68,0,93,58,92, + 2,125,9,125,10,100,5,124,9,23,0,125,11,116,13,124, + 8,124,11,131,2,125,12,116,15,124,12,131,1,114,148,124, + 0,160,16,124,10,124,1,124,12,124,8,103,1,124,2,161, + 5,2,0,1,0,83,0,113,148,116,17,124,8,131,1,125, + 3,124,0,106,14,68,0,93,112,92,2,125,9,125,10,122, + 20,116,13,124,0,106,2,124,4,124,9,23,0,131,2,125, + 12,87,0,110,24,4,0,116,18,144,1,121,18,1,0,1, + 0,1,0,89,0,1,0,100,6,83,0,48,0,116,19,106, + 20,100,7,124,12,100,3,100,8,141,3,1,0,124,7,124, + 9,23,0,124,6,118,0,114,222,116,15,124,12,131,1,114, + 222,124,0,160,16,124,10,124,1,124,12,100,6,124,2,161, + 5,2,0,1,0,83,0,113,222,124,3,144,1,114,122,116, + 19,160,20,100,9,124,8,161,2,1,0,116,19,160,21,124, + 1,100,6,161,2,125,13,124,8,103,1,124,13,95,22,124, + 13,83,0,100,6,83,0,41,10,122,111,84,114,121,32,116, + 111,32,102,105,110,100,32,97,32,115,112,101,99,32,102,111, + 114,32,116,104,101,32,115,112,101,99,105,102,105,101,100,32, + 109,111,100,117,108,101,46,10,10,32,32,32,32,32,32,32, + 32,82,101,116,117,114,110,115,32,116,104,101,32,109,97,116, + 99,104,105,110,103,32,115,112,101,99,44,32,111,114,32,78, + 111,110,101,32,105,102,32,110,111,116,32,102,111,117,110,100, + 46,10,32,32,32,32,32,32,32,32,70,114,96,0,0,0, + 114,45,0,0,0,114,129,0,0,0,114,232,0,0,0,78, + 122,9,116,114,121,105,110,103,32,123,125,41,1,90,9,118, + 101,114,98,111,115,105,116,121,122,25,112,111,115,115,105,98, + 108,101,32,110,97,109,101,115,112,97,99,101,32,102,111,114, + 32,123,125,41,23,114,103,0,0,0,114,75,0,0,0,114, + 65,0,0,0,114,24,0,0,0,114,81,0,0,0,114,33, + 1,0,0,114,76,0,0,0,114,89,1,0,0,218,11,95, + 102,105,108,108,95,99,97,99,104,101,114,27,0,0,0,114, + 92,1,0,0,114,130,0,0,0,114,91,1,0,0,114,67, + 0,0,0,114,88,1,0,0,114,80,0,0,0,114,83,1, + 0,0,114,82,0,0,0,114,110,0,0,0,114,158,0,0, + 0,114,173,0,0,0,114,207,0,0,0,114,202,0,0,0, + 41,14,114,142,0,0,0,114,163,0,0,0,114,225,0,0, + 0,90,12,105,115,95,110,97,109,101,115,112,97,99,101,90, + 11,116,97,105,108,95,109,111,100,117,108,101,114,193,0,0, + 0,90,5,99,97,99,104,101,90,12,99,97,99,104,101,95, + 109,111,100,117,108,101,90,9,98,97,115,101,95,112,97,116, + 104,114,41,1,0,0,114,211,0,0,0,90,13,105,110,105, + 116,95,102,105,108,101,110,97,109,101,90,9,102,117,108,108, + 95,112,97,116,104,114,210,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,114,226,0,0,0,234,5, + 0,0,115,78,0,0,0,0,5,4,1,14,1,2,1,24, + 1,12,1,10,1,10,1,8,1,6,2,6,1,6,1,10, + 2,6,1,4,2,8,1,12,1,14,1,8,1,10,1,8, + 1,26,4,8,2,14,1,2,1,20,1,14,1,10,1,16, + 1,12,1,8,1,10,1,4,255,10,2,6,1,12,1,12, + 1,8,1,4,1,122,20,70,105,108,101,70,105,110,100,101, + 114,46,102,105,110,100,95,115,112,101,99,99,1,0,0,0, + 0,0,0,0,0,0,0,0,9,0,0,0,10,0,0,0, + 67,0,0,0,115,188,0,0,0,124,0,106,0,125,1,122, + 22,116,1,160,2,124,1,112,22,116,1,160,3,161,0,161, + 1,125,2,87,0,110,28,4,0,116,4,116,5,116,6,102, + 3,121,56,1,0,1,0,1,0,103,0,125,2,89,0,110, + 2,48,0,116,7,106,8,160,9,100,1,161,1,115,82,116, + 10,124,2,131,1,124,0,95,11,110,74,116,10,131,0,125, + 3,124,2,68,0,93,56,125,4,124,4,160,12,100,2,161, + 1,92,3,125,5,125,6,125,7,124,6,114,134,100,3,160, + 13,124,5,124,7,160,14,161,0,161,2,125,8,110,4,124, + 5,125,8,124,3,160,15,124,8,161,1,1,0,113,92,124, + 3,124,0,95,11,116,7,106,8,160,9,116,16,161,1,114, + 184,100,4,100,5,132,0,124,2,68,0,131,1,124,0,95, + 17,100,6,83,0,41,7,122,68,70,105,108,108,32,116,104, + 101,32,99,97,99,104,101,32,111,102,32,112,111,116,101,110, + 116,105,97,108,32,109,111,100,117,108,101,115,32,97,110,100, + 32,112,97,99,107,97,103,101,115,32,102,111,114,32,116,104, + 105,115,32,100,105,114,101,99,116,111,114,121,46,114,20,0, + 0,0,114,96,0,0,0,114,87,0,0,0,99,1,0,0, + 0,0,0,0,0,0,0,0,0,2,0,0,0,4,0,0, + 0,83,0,0,0,115,20,0,0,0,104,0,124,0,93,12, + 125,1,124,1,160,0,161,0,146,2,113,4,83,0,114,10, + 0,0,0,41,1,114,130,0,0,0,41,2,114,8,0,0, + 0,90,2,102,110,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,114,19,0,0,0,58,6,0,0,114,13,0, + 0,0,122,41,70,105,108,101,70,105,110,100,101,114,46,95, + 102,105,108,108,95,99,97,99,104,101,46,60,108,111,99,97, + 108,115,62,46,60,115,101,116,99,111,109,112,62,78,41,18, + 114,65,0,0,0,114,24,0,0,0,114,30,1,0,0,114, + 81,0,0,0,114,26,1,0,0,218,15,80,101,114,109,105, + 115,115,105,111,110,69,114,114,111,114,218,18,78,111,116,65, + 68,105,114,101,99,116,111,114,121,69,114,114,111,114,114,21, + 0,0,0,114,28,0,0,0,114,29,0,0,0,114,90,1, + 0,0,114,91,1,0,0,114,125,0,0,0,114,88,0,0, + 0,114,130,0,0,0,218,3,97,100,100,114,30,0,0,0, + 114,92,1,0,0,41,9,114,142,0,0,0,114,65,0,0, + 0,114,31,1,0,0,90,21,108,111,119,101,114,95,115,117, + 102,102,105,120,95,99,111,110,116,101,110,116,115,114,66,1, + 0,0,114,140,0,0,0,114,54,1,0,0,114,41,1,0, + 0,90,8,110,101,119,95,110,97,109,101,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,114,94,1,0,0,29, + 6,0,0,115,34,0,0,0,0,2,6,1,2,1,22,1, + 18,3,10,3,12,1,12,7,6,1,8,1,16,1,4,1, + 18,2,4,1,12,1,6,1,12,1,122,22,70,105,108,101, + 70,105,110,100,101,114,46,95,102,105,108,108,95,99,97,99, + 104,101,99,1,0,0,0,0,0,0,0,0,0,0,0,3, + 0,0,0,3,0,0,0,7,0,0,0,115,18,0,0,0, + 135,0,135,1,102,2,100,1,100,2,132,8,125,2,124,2, + 83,0,41,3,97,20,1,0,0,65,32,99,108,97,115,115, + 32,109,101,116,104,111,100,32,119,104,105,99,104,32,114,101, + 116,117,114,110,115,32,97,32,99,108,111,115,117,114,101,32, + 116,111,32,117,115,101,32,111,110,32,115,121,115,46,112,97, + 116,104,95,104,111,111,107,10,32,32,32,32,32,32,32,32, + 119,104,105,99,104,32,119,105,108,108,32,114,101,116,117,114, + 110,32,97,110,32,105,110,115,116,97,110,99,101,32,117,115, + 105,110,103,32,116,104,101,32,115,112,101,99,105,102,105,101, + 100,32,108,111,97,100,101,114,115,32,97,110,100,32,116,104, + 101,32,112,97,116,104,10,32,32,32,32,32,32,32,32,99, + 97,108,108,101,100,32,111,110,32,116,104,101,32,99,108,111, + 115,117,114,101,46,10,10,32,32,32,32,32,32,32,32,73, + 102,32,116,104,101,32,112,97,116,104,32,99,97,108,108,101, + 100,32,111,110,32,116,104,101,32,99,108,111,115,117,114,101, + 32,105,115,32,110,111,116,32,97,32,100,105,114,101,99,116, + 111,114,121,44,32,73,109,112,111,114,116,69,114,114,111,114, + 32,105,115,10,32,32,32,32,32,32,32,32,114,97,105,115, + 101,100,46,10,10,32,32,32,32,32,32,32,32,99,1,0, + 0,0,0,0,0,0,0,0,0,0,1,0,0,0,4,0, + 0,0,19,0,0,0,115,36,0,0,0,116,0,124,0,131, + 1,115,20,116,1,100,1,124,0,100,2,141,2,130,1,136, + 0,124,0,103,1,136,1,162,1,82,0,142,0,83,0,41, + 3,122,45,80,97,116,104,32,104,111,111,107,32,102,111,114, + 32,105,109,112,111,114,116,108,105,98,46,109,97,99,104,105, + 110,101,114,121,46,70,105,108,101,70,105,110,100,101,114,46, + 122,30,111,110,108,121,32,100,105,114,101,99,116,111,114,105, + 101,115,32,97,114,101,32,115,117,112,112,111,114,116,101,100, + 114,71,0,0,0,41,2,114,82,0,0,0,114,141,0,0, + 0,114,71,0,0,0,169,2,114,216,0,0,0,114,93,1, + 0,0,114,10,0,0,0,114,11,0,0,0,218,24,112,97, + 116,104,95,104,111,111,107,95,102,111,114,95,70,105,108,101, + 70,105,110,100,101,114,70,6,0,0,115,6,0,0,0,0, + 2,8,1,12,1,122,54,70,105,108,101,70,105,110,100,101, + 114,46,112,97,116,104,95,104,111,111,107,46,60,108,111,99, + 97,108,115,62,46,112,97,116,104,95,104,111,111,107,95,102, + 111,114,95,70,105,108,101,70,105,110,100,101,114,114,10,0, + 0,0,41,3,114,216,0,0,0,114,93,1,0,0,114,99, + 1,0,0,114,10,0,0,0,114,98,1,0,0,114,11,0, + 0,0,218,9,112,97,116,104,95,104,111,111,107,60,6,0, + 0,115,4,0,0,0,0,10,14,6,122,20,70,105,108,101, + 70,105,110,100,101,114,46,112,97,116,104,95,104,111,111,107, + 99,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0, + 0,3,0,0,0,67,0,0,0,115,12,0,0,0,100,1, + 160,0,124,0,106,1,161,1,83,0,41,2,78,122,16,70, + 105,108,101,70,105,110,100,101,114,40,123,33,114,125,41,41, + 2,114,88,0,0,0,114,65,0,0,0,114,13,1,0,0, + 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, + 64,1,0,0,78,6,0,0,115,2,0,0,0,0,1,122, + 19,70,105,108,101,70,105,110,100,101,114,46,95,95,114,101, + 112,114,95,95,41,1,78,41,15,114,149,0,0,0,114,148, + 0,0,0,114,150,0,0,0,114,151,0,0,0,114,232,0, + 0,0,114,71,1,0,0,114,167,0,0,0,114,229,0,0, + 0,114,161,0,0,0,114,83,1,0,0,114,226,0,0,0, + 114,94,1,0,0,114,230,0,0,0,114,100,1,0,0,114, + 64,1,0,0,114,10,0,0,0,114,10,0,0,0,114,10, + 0,0,0,114,11,0,0,0,114,86,1,0,0,186,5,0, + 0,115,22,0,0,0,8,2,4,7,8,16,8,4,4,2, + 8,12,8,5,10,51,8,31,2,1,10,17,114,86,1,0, + 0,99,4,0,0,0,0,0,0,0,0,0,0,0,6,0, + 0,0,8,0,0,0,67,0,0,0,115,144,0,0,0,124, + 0,160,0,100,1,161,1,125,4,124,0,160,0,100,2,161, + 1,125,5,124,4,115,66,124,5,114,36,124,5,106,1,125, + 4,110,30,124,2,124,3,107,2,114,56,116,2,124,1,124, + 2,131,2,125,4,110,10,116,3,124,1,124,2,131,2,125, + 4,124,5,115,84,116,4,124,1,124,2,124,4,100,3,141, + 3,125,5,122,36,124,5,124,0,100,2,60,0,124,4,124, + 0,100,1,60,0,124,2,124,0,100,4,60,0,124,3,124, + 0,100,5,60,0,87,0,110,18,4,0,116,5,121,138,1, + 0,1,0,1,0,89,0,110,2,48,0,100,0,83,0,41, + 6,78,218,10,95,95,108,111,97,100,101,114,95,95,218,8, + 95,95,115,112,101,99,95,95,114,87,1,0,0,90,8,95, + 95,102,105,108,101,95,95,90,10,95,95,99,97,99,104,101, + 100,95,95,41,6,218,3,103,101,116,114,164,0,0,0,114, + 39,1,0,0,114,32,1,0,0,114,213,0,0,0,218,9, + 69,120,99,101,112,116,105,111,110,41,6,90,2,110,115,114, + 140,0,0,0,90,8,112,97,116,104,110,97,109,101,90,9, + 99,112,97,116,104,110,97,109,101,114,164,0,0,0,114,210, + 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,218,14,95,102,105,120,95,117,112,95,109,111,100,117, + 108,101,84,6,0,0,115,34,0,0,0,0,2,10,1,10, + 1,4,1,4,1,8,1,8,1,12,2,10,1,4,1,14, + 1,2,1,8,1,8,1,8,1,12,1,12,2,114,105,1, + 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,3, + 0,0,0,3,0,0,0,67,0,0,0,115,38,0,0,0, + 116,0,116,1,160,2,161,0,102,2,125,0,116,3,116,4, + 102,2,125,1,116,5,116,6,102,2,125,2,124,0,124,1, + 124,2,103,3,83,0,41,1,122,95,82,101,116,117,114,110, + 115,32,97,32,108,105,115,116,32,111,102,32,102,105,108,101, + 45,98,97,115,101,100,32,109,111,100,117,108,101,32,108,111, + 97,100,101,114,115,46,10,10,32,32,32,32,69,97,99,104, + 32,105,116,101,109,32,105,115,32,97,32,116,117,112,108,101, + 32,40,108,111,97,100,101,114,44,32,115,117,102,102,105,120, + 101,115,41,46,10,32,32,32,32,41,7,114,19,1,0,0, + 114,187,0,0,0,218,18,101,120,116,101,110,115,105,111,110, + 95,115,117,102,102,105,120,101,115,114,32,1,0,0,114,126, + 0,0,0,114,39,1,0,0,114,112,0,0,0,41,3,90, + 10,101,120,116,101,110,115,105,111,110,115,90,6,115,111,117, + 114,99,101,90,8,98,121,116,101,99,111,100,101,114,10,0, + 0,0,114,10,0,0,0,114,11,0,0,0,114,208,0,0, + 0,107,6,0,0,115,8,0,0,0,0,5,12,1,8,1, + 8,1,114,208,0,0,0,99,1,0,0,0,0,0,0,0, + 0,0,0,0,10,0,0,0,9,0,0,0,67,0,0,0, + 115,132,1,0,0,124,0,97,0,116,0,106,1,97,1,116, + 0,106,2,97,2,116,1,106,3,116,4,25,0,125,1,100, + 1,100,2,103,1,102,2,100,3,100,4,100,2,103,2,102, + 2,102,2,125,2,124,2,68,0,93,108,92,2,125,3,125, + 4,116,5,100,5,100,6,132,0,124,4,68,0,131,1,131, + 1,115,82,74,0,130,1,124,4,100,7,25,0,125,5,124, + 3,116,1,106,3,118,0,114,116,116,1,106,3,124,3,25, + 0,125,6,1,0,113,170,113,52,122,20,116,0,160,6,124, + 3,161,1,125,6,87,0,1,0,113,170,87,0,113,52,4, + 0,116,7,121,158,1,0,1,0,1,0,89,0,113,52,89, + 0,113,52,48,0,113,52,116,7,100,8,131,1,130,1,116, + 8,124,1,100,9,124,6,131,3,1,0,116,8,124,1,100, + 10,124,5,131,3,1,0,116,8,124,1,100,11,100,12,160, + 9,124,4,161,1,131,3,1,0,116,8,124,1,100,13,100, + 14,100,15,132,0,124,4,68,0,131,1,131,3,1,0,103, + 0,100,16,162,1,125,7,124,3,100,3,107,2,144,1,114, + 6,124,7,160,10,100,17,161,1,1,0,124,7,68,0,93, + 52,125,8,124,8,116,1,106,3,118,1,144,1,114,38,116, + 0,160,6,124,8,161,1,125,9,110,10,116,1,106,3,124, + 8,25,0,125,9,116,8,124,1,124,8,124,9,131,3,1, + 0,144,1,113,10,116,8,124,1,100,18,116,11,131,0,131, + 3,1,0,116,12,160,13,116,2,160,14,161,0,161,1,1, + 0,124,3,100,3,107,2,144,1,114,128,116,15,160,10,100, + 19,161,1,1,0,100,20,116,12,118,0,144,1,114,128,100, + 21,116,16,95,17,100,22,83,0,41,23,122,205,83,101,116, + 117,112,32,116,104,101,32,112,97,116,104,45,98,97,115,101, + 100,32,105,109,112,111,114,116,101,114,115,32,102,111,114,32, + 105,109,112,111,114,116,108,105,98,32,98,121,32,105,109,112, + 111,114,116,105,110,103,32,110,101,101,100,101,100,10,32,32, + 32,32,98,117,105,108,116,45,105,110,32,109,111,100,117,108, + 101,115,32,97,110,100,32,105,110,106,101,99,116,105,110,103, + 32,116,104,101,109,32,105,110,116,111,32,116,104,101,32,103, + 108,111,98,97,108,32,110,97,109,101,115,112,97,99,101,46, + 10,10,32,32,32,32,79,116,104,101,114,32,99,111,109,112, + 111,110,101,110,116,115,32,97,114,101,32,101,120,116,114,97, + 99,116,101,100,32,102,114,111,109,32,116,104,101,32,99,111, + 114,101,32,98,111,111,116,115,116,114,97,112,32,109,111,100, + 117,108,101,46,10,10,32,32,32,32,218,5,112,111,115,105, + 120,114,2,0,0,0,218,2,110,116,114,1,0,0,0,99, 1,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0, - 4,0,0,0,83,0,0,0,115,22,0,0,0,104,0,124, - 0,93,14,125,1,100,0,124,1,155,0,157,2,146,2,113, - 4,83,0,114,15,0,0,0,114,10,0,0,0,114,17,0, - 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,19,0,0,0,144,6,0,0,114,13,0,0,0,122, - 25,95,115,101,116,117,112,46,60,108,111,99,97,108,115,62, - 46,60,115,101,116,99,111,109,112,62,41,3,114,90,0,0, - 0,114,98,0,0,0,114,184,0,0,0,114,215,0,0,0, - 114,27,0,0,0,122,4,46,112,121,119,122,6,95,100,46, - 112,121,100,84,78,41,18,114,158,0,0,0,114,21,0,0, - 0,114,187,0,0,0,114,54,1,0,0,114,149,0,0,0, - 218,3,97,108,108,90,18,95,98,117,105,108,116,105,110,95, - 102,114,111,109,95,110,97,109,101,114,141,0,0,0,114,153, - 0,0,0,114,62,0,0,0,114,61,0,0,0,114,32,0, - 0,0,114,44,1,0,0,114,191,0,0,0,114,104,1,0, - 0,114,126,0,0,0,114,214,0,0,0,114,218,0,0,0, - 41,10,218,17,95,98,111,111,116,115,116,114,97,112,95,109, - 111,100,117,108,101,90,11,115,101,108,102,95,109,111,100,117, - 108,101,90,10,111,115,95,100,101,116,97,105,108,115,90,10, - 98,117,105,108,116,105,110,95,111,115,114,50,0,0,0,114, - 59,0,0,0,90,9,111,115,95,109,111,100,117,108,101,90, - 13,98,117,105,108,116,105,110,95,110,97,109,101,115,90,12, - 98,117,105,108,116,105,110,95,110,97,109,101,90,14,98,117, - 105,108,116,105,110,95,109,111,100,117,108,101,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,218,6,95,115,101, - 116,117,112,109,6,0,0,115,70,0,0,0,0,8,4,1, - 6,1,6,2,10,3,22,1,12,2,22,1,8,1,10,1, - 10,1,6,2,2,1,10,1,10,1,12,1,12,2,8,2, - 12,1,12,1,18,1,22,3,8,1,10,1,10,1,8,1, - 12,1,12,2,10,1,16,3,14,1,14,1,10,1,10,1, - 10,1,114,110,1,0,0,99,1,0,0,0,0,0,0,0, - 0,0,0,0,2,0,0,0,4,0,0,0,67,0,0,0, - 115,50,0,0,0,116,0,124,0,131,1,1,0,116,1,131, - 0,125,1,116,2,106,3,160,4,116,5,106,6,124,1,142, - 0,103,1,161,1,1,0,116,2,106,7,160,8,116,9,161, - 1,1,0,100,1,83,0,41,2,122,41,73,110,115,116,97, - 108,108,32,116,104,101,32,112,97,116,104,45,98,97,115,101, - 100,32,105,109,112,111,114,116,32,99,111,109,112,111,110,101, - 110,116,115,46,78,41,10,114,110,1,0,0,114,208,0,0, - 0,114,21,0,0,0,114,74,1,0,0,114,191,0,0,0, - 114,84,1,0,0,114,98,1,0,0,218,9,109,101,116,97, - 95,112,97,116,104,114,61,0,0,0,114,68,1,0,0,41, - 2,114,109,1,0,0,90,17,115,117,112,112,111,114,116,101, - 100,95,108,111,97,100,101,114,115,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,218,8,95,105,110,115,116,97, - 108,108,166,6,0,0,115,8,0,0,0,0,2,8,1,6, - 1,20,1,114,112,1,0,0,41,1,114,86,0,0,0,41, - 1,78,41,3,78,78,78,41,2,114,0,0,0,0,114,0, - 0,0,0,41,1,84,41,1,78,41,1,78,41,81,114,151, - 0,0,0,114,187,0,0,0,114,90,0,0,0,114,21,0, - 0,0,114,98,0,0,0,114,184,0,0,0,114,28,0,0, - 0,90,11,95,77,83,95,87,73,78,68,79,87,83,114,106, - 1,0,0,114,24,0,0,0,114,215,0,0,0,114,105,1, - 0,0,114,50,0,0,0,114,108,1,0,0,114,59,0,0, - 0,114,135,0,0,0,114,57,0,0,0,114,62,0,0,0, - 114,107,1,0,0,114,31,0,0,0,90,37,95,67,65,83, - 69,95,73,78,83,69,78,83,73,84,73,86,69,95,80,76, - 65,84,70,79,82,77,83,95,66,89,84,69,83,95,75,69, - 89,114,30,0,0,0,114,32,0,0,0,114,39,0,0,0, - 114,44,0,0,0,114,46,0,0,0,114,67,0,0,0,114, - 74,0,0,0,114,75,0,0,0,114,79,0,0,0,114,80, - 0,0,0,114,82,0,0,0,114,85,0,0,0,114,94,0, - 0,0,218,4,116,121,112,101,218,8,95,95,99,111,100,101, - 95,95,114,186,0,0,0,114,37,0,0,0,114,172,0,0, - 0,114,36,0,0,0,114,41,0,0,0,114,3,1,0,0, - 114,115,0,0,0,114,111,0,0,0,114,126,0,0,0,114, - 112,0,0,0,90,23,68,69,66,85,71,95,66,89,84,69, - 67,79,68,69,95,83,85,70,70,73,88,69,83,90,27,79, - 80,84,73,77,73,90,69,68,95,66,89,84,69,67,79,68, - 69,95,83,85,70,70,73,88,69,83,114,120,0,0,0,114, - 127,0,0,0,114,134,0,0,0,114,136,0,0,0,114,138, - 0,0,0,114,160,0,0,0,114,167,0,0,0,114,176,0, - 0,0,114,180,0,0,0,114,182,0,0,0,114,189,0,0, - 0,114,194,0,0,0,114,195,0,0,0,114,200,0,0,0, - 218,6,111,98,106,101,99,116,114,209,0,0,0,114,213,0, - 0,0,114,214,0,0,0,114,231,0,0,0,114,244,0,0, - 0,114,6,1,0,0,114,32,1,0,0,114,39,1,0,0, - 114,44,1,0,0,114,19,1,0,0,114,45,1,0,0,114, - 66,1,0,0,114,68,1,0,0,114,84,1,0,0,114,103, - 1,0,0,114,208,0,0,0,114,110,1,0,0,114,112,1, - 0,0,114,10,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,218,8,60,109,111,100,117,108,101,62, - 1,0,0,0,115,170,0,0,0,4,22,8,1,8,1,8, - 1,8,1,8,3,10,1,4,1,8,1,10,2,8,3,4, - 1,10,2,6,2,22,1,8,1,8,1,10,1,14,4,4, - 1,4,1,2,1,2,255,4,4,8,17,8,5,8,5,8, - 6,4,1,10,30,8,6,8,8,8,10,8,9,8,5,8, - 7,6,1,10,8,8,5,10,22,10,127,0,20,16,1,12, - 2,4,1,4,2,6,2,6,2,8,2,16,71,8,40,8, - 19,8,12,8,12,8,28,8,17,8,33,8,28,8,24,10, - 13,10,10,10,11,8,14,6,3,4,1,2,255,12,68,14, - 64,14,29,16,127,0,17,14,72,18,45,18,26,4,3,18, - 58,14,63,14,42,14,127,0,20,14,127,0,27,10,23,8, - 11,8,57, + 3,0,0,0,115,0,0,0,115,26,0,0,0,124,0,93, + 18,125,1,116,0,124,1,131,1,100,0,107,2,86,0,1, + 0,113,2,100,1,83,0,114,3,0,0,0,114,5,0,0, + 0,114,7,0,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,114,12,0,0,0,136,6,0,0,114,13, + 0,0,0,122,25,95,115,101,116,117,112,46,60,108,111,99, + 97,108,115,62,46,60,103,101,110,101,120,112,114,62,114,0, + 0,0,0,122,30,105,109,112,111,114,116,108,105,98,32,114, + 101,113,117,105,114,101,115,32,112,111,115,105,120,32,111,114, + 32,110,116,114,24,0,0,0,114,59,0,0,0,114,50,0, + 0,0,114,14,0,0,0,218,20,95,112,97,116,104,115,101, + 112,115,95,119,105,116,104,95,99,111,108,111,110,99,1,0, + 0,0,0,0,0,0,0,0,0,0,2,0,0,0,4,0, + 0,0,83,0,0,0,115,22,0,0,0,104,0,124,0,93, + 14,125,1,100,0,124,1,155,0,157,2,146,2,113,4,83, + 0,114,15,0,0,0,114,10,0,0,0,114,17,0,0,0, + 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, + 19,0,0,0,153,6,0,0,114,13,0,0,0,122,25,95, + 115,101,116,117,112,46,60,108,111,99,97,108,115,62,46,60, + 115,101,116,99,111,109,112,62,41,3,114,90,0,0,0,114, + 98,0,0,0,114,184,0,0,0,114,215,0,0,0,114,27, + 0,0,0,122,4,46,112,121,119,122,6,95,100,46,112,121, + 100,84,78,41,18,114,158,0,0,0,114,21,0,0,0,114, + 187,0,0,0,114,56,1,0,0,114,149,0,0,0,218,3, + 97,108,108,90,18,95,98,117,105,108,116,105,110,95,102,114, + 111,109,95,110,97,109,101,114,141,0,0,0,114,153,0,0, + 0,114,62,0,0,0,114,61,0,0,0,114,32,0,0,0, + 114,44,1,0,0,114,191,0,0,0,114,106,1,0,0,114, + 126,0,0,0,114,214,0,0,0,114,218,0,0,0,41,10, + 218,17,95,98,111,111,116,115,116,114,97,112,95,109,111,100, + 117,108,101,90,11,115,101,108,102,95,109,111,100,117,108,101, + 90,10,111,115,95,100,101,116,97,105,108,115,90,10,98,117, + 105,108,116,105,110,95,111,115,114,50,0,0,0,114,59,0, + 0,0,90,9,111,115,95,109,111,100,117,108,101,90,13,98, + 117,105,108,116,105,110,95,110,97,109,101,115,90,12,98,117, + 105,108,116,105,110,95,110,97,109,101,90,14,98,117,105,108, + 116,105,110,95,109,111,100,117,108,101,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,218,6,95,115,101,116,117, + 112,118,6,0,0,115,70,0,0,0,0,8,4,1,6,1, + 6,2,10,3,22,1,12,2,22,1,8,1,10,1,10,1, + 6,2,2,1,10,1,10,1,12,1,12,2,8,2,12,1, + 12,1,18,1,22,3,8,1,10,1,10,1,8,1,12,1, + 12,2,10,1,16,3,14,1,14,1,10,1,10,1,10,1, + 114,112,1,0,0,99,1,0,0,0,0,0,0,0,0,0, + 0,0,2,0,0,0,4,0,0,0,67,0,0,0,115,50, + 0,0,0,116,0,124,0,131,1,1,0,116,1,131,0,125, + 1,116,2,106,3,160,4,116,5,106,6,124,1,142,0,103, + 1,161,1,1,0,116,2,106,7,160,8,116,9,161,1,1, + 0,100,1,83,0,41,2,122,41,73,110,115,116,97,108,108, + 32,116,104,101,32,112,97,116,104,45,98,97,115,101,100,32, + 105,109,112,111,114,116,32,99,111,109,112,111,110,101,110,116, + 115,46,78,41,10,114,112,1,0,0,114,208,0,0,0,114, + 21,0,0,0,114,76,1,0,0,114,191,0,0,0,114,86, + 1,0,0,114,100,1,0,0,218,9,109,101,116,97,95,112, + 97,116,104,114,61,0,0,0,114,70,1,0,0,41,2,114, + 111,1,0,0,90,17,115,117,112,112,111,114,116,101,100,95, + 108,111,97,100,101,114,115,114,10,0,0,0,114,10,0,0, + 0,114,11,0,0,0,218,8,95,105,110,115,116,97,108,108, + 175,6,0,0,115,8,0,0,0,0,2,8,1,6,1,20, + 1,114,114,1,0,0,41,1,114,86,0,0,0,41,1,78, + 41,3,78,78,78,41,2,114,0,0,0,0,114,0,0,0, + 0,41,1,84,41,1,78,41,1,78,41,81,114,151,0,0, + 0,114,187,0,0,0,114,90,0,0,0,114,21,0,0,0, + 114,98,0,0,0,114,184,0,0,0,114,28,0,0,0,90, + 11,95,77,83,95,87,73,78,68,79,87,83,114,108,1,0, + 0,114,24,0,0,0,114,215,0,0,0,114,107,1,0,0, + 114,50,0,0,0,114,110,1,0,0,114,59,0,0,0,114, + 135,0,0,0,114,57,0,0,0,114,62,0,0,0,114,109, + 1,0,0,114,31,0,0,0,90,37,95,67,65,83,69,95, + 73,78,83,69,78,83,73,84,73,86,69,95,80,76,65,84, + 70,79,82,77,83,95,66,89,84,69,83,95,75,69,89,114, + 30,0,0,0,114,32,0,0,0,114,39,0,0,0,114,44, + 0,0,0,114,46,0,0,0,114,67,0,0,0,114,74,0, + 0,0,114,75,0,0,0,114,79,0,0,0,114,80,0,0, + 0,114,82,0,0,0,114,85,0,0,0,114,94,0,0,0, + 218,4,116,121,112,101,218,8,95,95,99,111,100,101,95,95, + 114,186,0,0,0,114,37,0,0,0,114,172,0,0,0,114, + 36,0,0,0,114,41,0,0,0,114,3,1,0,0,114,115, + 0,0,0,114,111,0,0,0,114,126,0,0,0,114,112,0, + 0,0,90,23,68,69,66,85,71,95,66,89,84,69,67,79, + 68,69,95,83,85,70,70,73,88,69,83,90,27,79,80,84, + 73,77,73,90,69,68,95,66,89,84,69,67,79,68,69,95, + 83,85,70,70,73,88,69,83,114,120,0,0,0,114,127,0, + 0,0,114,134,0,0,0,114,136,0,0,0,114,138,0,0, + 0,114,160,0,0,0,114,167,0,0,0,114,176,0,0,0, + 114,180,0,0,0,114,182,0,0,0,114,189,0,0,0,114, + 194,0,0,0,114,195,0,0,0,114,200,0,0,0,218,6, + 111,98,106,101,99,116,114,209,0,0,0,114,213,0,0,0, + 114,214,0,0,0,114,231,0,0,0,114,244,0,0,0,114, + 6,1,0,0,114,32,1,0,0,114,39,1,0,0,114,44, + 1,0,0,114,19,1,0,0,114,45,1,0,0,114,68,1, + 0,0,114,70,1,0,0,114,86,1,0,0,114,105,1,0, + 0,114,208,0,0,0,114,112,1,0,0,114,114,1,0,0, + 114,10,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,218,8,60,109,111,100,117,108,101,62,1,0, + 0,0,115,170,0,0,0,4,22,8,1,8,1,8,1,8, + 1,8,3,10,1,4,1,8,1,10,2,8,3,4,1,10, + 2,6,2,22,1,8,1,8,1,10,1,14,4,4,1,4, + 1,2,1,2,255,4,4,8,17,8,5,8,5,8,6,4, + 1,10,30,8,6,8,8,8,10,8,9,8,5,8,7,6, + 1,10,8,8,5,10,22,10,127,0,20,16,1,12,2,4, + 1,4,2,6,2,6,2,8,2,16,71,8,40,8,19,8, + 12,8,12,8,28,8,17,8,33,8,28,8,24,10,13,10, + 10,10,11,8,14,6,3,4,1,2,255,12,68,14,64,14, + 29,16,127,0,17,14,72,18,45,18,26,4,3,18,58,14, + 69,14,42,14,127,0,23,14,127,0,27,10,23,8,11,8, + 57, }; From webhook-mailer at python.org Wed Feb 2 10:03:19 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 15:03:19 -0000 Subject: [Python-checkins] bpo-45459: Add Py_buffer to limited API (GH-29991) Message-ID: https://github.com/python/cpython/commit/f66c857572a308822c70fd25e0197b6e0dec6e34 commit: f66c857572a308822c70fd25e0197b6e0dec6e34 branch: main author: Christian Heimes committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T07:03:10-08:00 summary: bpo-45459: Add Py_buffer to limited API (GH-29991) - [x] ``Py_buffer`` struct - [x] ``PyBuffer_*()`` API functions - [x] ``PyBUF_*`` constants - [x] ``Py_bf_getbuffer`` and ``Py_bf_releasebuffer`` type slots - [x] ``PyMemoryView_FromBuffer()`` API - [x] tests for limited API - [x] ``make regen-limited-abi`` - [x] documentation update - [ ] export ``PyPickleBuffer*()`` API ??? files: A Include/buffer.h A Misc/NEWS.d/next/C API/2021-10-18-16-54-24.bpo-45459.Y1pEZs.rst M Doc/c-api/buffer.rst M Doc/c-api/type.rst M Doc/data/stable_abi.dat M Doc/whatsnew/3.11.rst M Include/Python.h M Include/cpython/abstract.h M Include/cpython/object.h M Include/memoryobject.h M Include/typeslots.h M Lib/test/test_stable_abi_ctypes.py M Lib/test/test_xxlimited.py M Makefile.pre.in M Misc/stable_abi.txt M Modules/xxlimited.c M PC/python3dll.c diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst index 820a3a6f990ef..05e131d06b909 100644 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -499,6 +499,13 @@ Buffer-related functions This function fails if *len* != *src->len*. +.. c:function:: int PyObject_CopyData(Py_buffer *dest, Py_buffer *src) + + Copy data from *src* to *dest* buffer. Can convert between C-style and + or Fortran-style buffers. + + ``0`` is returned on success, ``-1`` on error. + .. c:function:: void PyBuffer_FillContiguousStrides(int ndims, Py_ssize_t *shape, Py_ssize_t *strides, int itemsize, char order) Fill the *strides* array with byte-strides of a :term:`contiguous` (C-style if diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index f96886985e932..97a818ab2ccd0 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -272,12 +272,6 @@ The following functions and structs are used to create * :c:member:`~PyTypeObject.tp_vectorcall_offset` (see :ref:`PyMemberDef `) - The following fields cannot be set using :c:type:`PyType_Spec` and - :c:type:`PyType_Slot` under the limited API: - - * :c:member:`~PyBufferProcs.bf_getbuffer` - * :c:member:`~PyBufferProcs.bf_releasebuffer` - Setting :c:data:`Py_tp_bases` or :c:data:`Py_tp_base` may be problematic on some platforms. To avoid issues, use the *bases* argument of @@ -287,6 +281,11 @@ The following functions and structs are used to create Slots in :c:type:`PyBufferProcs` in may be set in the unlimited API. + .. versionchanged:: 3.11 + :c:member:`~PyBufferProcs.bf_getbuffer` and + :c:member:`~PyBufferProcs.bf_releasebuffer` are now available + under limited API. + .. c:member:: void *PyType_Slot.pfunc The desired value of the slot. In most cases, this is a pointer diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index 02e54e5d7f14a..18bbf03187b30 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -10,6 +10,14 @@ function,PyArg_ValidateKeywordArguments,3.2, var,PyBaseObject_Type,3.2, function,PyBool_FromLong,3.2, var,PyBool_Type,3.2, +function,PyBuffer_FillContiguousStrides,3.11, +function,PyBuffer_FillInfo,3.11, +function,PyBuffer_FromContiguous,3.11, +function,PyBuffer_GetPointer,3.11, +function,PyBuffer_IsContiguous,3.11, +function,PyBuffer_Release,3.11, +function,PyBuffer_SizeFromFormat,3.11, +function,PyBuffer_ToContiguous,3.11, var,PyByteArrayIter_Type,3.2, function,PyByteArray_AsString,3.2, function,PyByteArray_Concat,3.2, @@ -375,6 +383,7 @@ function,PyMem_Malloc,3.2, function,PyMem_Realloc,3.2, type,PyMemberDef,3.2, var,PyMemberDescr_Type,3.2, +function,PyMemoryView_FromBuffer,3.11, function,PyMemoryView_FromMemory,3.7, function,PyMemoryView_FromObject,3.2, function,PyMemoryView_GetContiguous,3.2, @@ -476,8 +485,10 @@ function,PyObject_CallMethodObjArgs,3.2, function,PyObject_CallNoArgs,3.10, function,PyObject_CallObject,3.2, function,PyObject_Calloc,3.7, +function,PyObject_CheckBuffer,3.11, function,PyObject_CheckReadBuffer,3.2, function,PyObject_ClearWeakRefs,3.2, +function,PyObject_CopyData,3.11, function,PyObject_DelItem,3.2, function,PyObject_DelItemString,3.2, function,PyObject_Dir,3.2, @@ -495,6 +506,7 @@ function,PyObject_GenericSetDict,3.7, function,PyObject_GetAIter,3.10, function,PyObject_GetAttr,3.2, function,PyObject_GetAttrString,3.2, +function,PyObject_GetBuffer,3.11, function,PyObject_GetItem,3.2, function,PyObject_GetIter,3.2, function,PyObject_HasAttr,3.2, @@ -832,6 +844,7 @@ var,Py_UTF8Mode,3.8, function,Py_VaBuildValue,3.2, var,Py_Version,3.11, function,Py_XNewRef,3.10, +type,Py_buffer,3.11, type,Py_intptr_t,3.2, type,Py_ssize_t,3.2, type,Py_uintptr_t,3.2, diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index e7f3dab2b51db..3458ad63c9df8 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -657,6 +657,26 @@ New Features :c:macro:`PY_VERSION_HEX`. (Contributed by Gabriele N. Tornetta in :issue:`43931`.) +* :c:type:`Py_buffer` and APIs are now part of the limited API and the stable + ABI: + + * :c:func:`PyObject_CheckBuffer` + * :c:func:`PyObject_GetBuffer` + * :c:func:`PyBuffer_GetPointer` + * :c:func:`PyBuffer_SizeFromFormat` + * :c:func:`PyBuffer_ToContiguous` + * :c:func:`PyBuffer_FromContiguous` + * :c:func:`PyBuffer_CopyData` + * :c:func:`PyBuffer_IsContiguous` + * :c:func:`PyBuffer_FillContiguousStrides` + * :c:func:`PyBuffer_FillInfo` + * :c:func:`PyBuffer_Release` + * :c:func:`PyMemoryView_FromBuffer` + * :c:member:`~PyBufferProcs.bf_getbuffer` and + :c:member:`~PyBufferProcs.bf_releasebuffer` type slots + + (Contributed by Christian Heimes in :issue:`45459`.) + Porting to Python 3.11 ---------------------- diff --git a/Include/Python.h b/Include/Python.h index 7260ae5cd0b4f..5416b04e4bfb3 100644 --- a/Include/Python.h +++ b/Include/Python.h @@ -50,6 +50,7 @@ #include "longobject.h" #include "cpython/longintrepr.h" #include "boolobject.h" +#include "buffer.h" #include "floatobject.h" #include "complexobject.h" #include "rangeobject.h" diff --git a/Include/buffer.h b/Include/buffer.h new file mode 100644 index 0000000000000..6893505e66e3e --- /dev/null +++ b/Include/buffer.h @@ -0,0 +1,142 @@ +/* Public Py_buffer API */ + +#ifndef Py_BUFFER_H +#define Py_BUFFER_H +#ifdef __cplusplus +extern "C" { +#endif + +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030b0000 + +/* === New Buffer API ============================================ + * Limited API and stable ABI since Python 3.11 + * + * Py_buffer struct layout and size is now part of the stable abi3. The + * struct layout and size must not be changed in any way, as it would + * break the ABI. + * + */ + +typedef struct { + void *buf; + PyObject *obj; /* owned reference */ + Py_ssize_t len; + Py_ssize_t itemsize; /* This is Py_ssize_t so it can be + pointed to by strides in simple case.*/ + int readonly; + int ndim; + char *format; + Py_ssize_t *shape; + Py_ssize_t *strides; + Py_ssize_t *suboffsets; + void *internal; +} Py_buffer; + +/* Return 1 if the getbuffer function is available, otherwise return 0. */ +PyAPI_FUNC(int) PyObject_CheckBuffer(PyObject *obj); + +/* This is a C-API version of the getbuffer function call. It checks + to make sure object has the required function pointer and issues the + call. + + Returns -1 and raises an error on failure and returns 0 on success. */ +PyAPI_FUNC(int) PyObject_GetBuffer(PyObject *obj, Py_buffer *view, + int flags); + +/* Get the memory area pointed to by the indices for the buffer given. + Note that view->ndim is the assumed size of indices. */ +PyAPI_FUNC(void *) PyBuffer_GetPointer(const Py_buffer *view, const Py_ssize_t *indices); + +/* Return the implied itemsize of the data-format area from a + struct-style description. */ +PyAPI_FUNC(Py_ssize_t) PyBuffer_SizeFromFormat(const char *format); + +/* Implementation in memoryobject.c */ +PyAPI_FUNC(int) PyBuffer_ToContiguous(void *buf, const Py_buffer *view, + Py_ssize_t len, char order); + +PyAPI_FUNC(int) PyBuffer_FromContiguous(const Py_buffer *view, const void *buf, + Py_ssize_t len, char order); + +/* Copy len bytes of data from the contiguous chunk of memory + pointed to by buf into the buffer exported by obj. Return + 0 on success and return -1 and raise a PyBuffer_Error on + error (i.e. the object does not have a buffer interface or + it is not working). + + If fort is 'F', then if the object is multi-dimensional, + then the data will be copied into the array in + Fortran-style (first dimension varies the fastest). If + fort is 'C', then the data will be copied into the array + in C-style (last dimension varies the fastest). If fort + is 'A', then it does not matter and the copy will be made + in whatever way is more efficient. */ +PyAPI_FUNC(int) PyObject_CopyData(PyObject *dest, PyObject *src); + +/* Copy the data from the src buffer to the buffer of destination. */ +PyAPI_FUNC(int) PyBuffer_IsContiguous(const Py_buffer *view, char fort); + +/*Fill the strides array with byte-strides of a contiguous + (Fortran-style if fort is 'F' or C-style otherwise) + array of the given shape with the given number of bytes + per element. */ +PyAPI_FUNC(void) PyBuffer_FillContiguousStrides(int ndims, + Py_ssize_t *shape, + Py_ssize_t *strides, + int itemsize, + char fort); + +/* Fills in a buffer-info structure correctly for an exporter + that can only share a contiguous chunk of memory of + "unsigned bytes" of the given length. + + Returns 0 on success and -1 (with raising an error) on error. */ +PyAPI_FUNC(int) PyBuffer_FillInfo(Py_buffer *view, PyObject *o, void *buf, + Py_ssize_t len, int readonly, + int flags); + +/* Releases a Py_buffer obtained from getbuffer ParseTuple's "s*". */ +PyAPI_FUNC(void) PyBuffer_Release(Py_buffer *view); + +/* Maximum number of dimensions */ +#define PyBUF_MAX_NDIM 64 + +/* Flags for getting buffers */ +#define PyBUF_SIMPLE 0 +#define PyBUF_WRITABLE 0x0001 + +#ifndef Py_LIMITED_API +/* we used to include an E, backwards compatible alias */ +#define PyBUF_WRITEABLE PyBUF_WRITABLE +#endif + +#define PyBUF_FORMAT 0x0004 +#define PyBUF_ND 0x0008 +#define PyBUF_STRIDES (0x0010 | PyBUF_ND) +#define PyBUF_C_CONTIGUOUS (0x0020 | PyBUF_STRIDES) +#define PyBUF_F_CONTIGUOUS (0x0040 | PyBUF_STRIDES) +#define PyBUF_ANY_CONTIGUOUS (0x0080 | PyBUF_STRIDES) +#define PyBUF_INDIRECT (0x0100 | PyBUF_STRIDES) + +#define PyBUF_CONTIG (PyBUF_ND | PyBUF_WRITABLE) +#define PyBUF_CONTIG_RO (PyBUF_ND) + +#define PyBUF_STRIDED (PyBUF_STRIDES | PyBUF_WRITABLE) +#define PyBUF_STRIDED_RO (PyBUF_STRIDES) + +#define PyBUF_RECORDS (PyBUF_STRIDES | PyBUF_WRITABLE | PyBUF_FORMAT) +#define PyBUF_RECORDS_RO (PyBUF_STRIDES | PyBUF_FORMAT) + +#define PyBUF_FULL (PyBUF_INDIRECT | PyBUF_WRITABLE | PyBUF_FORMAT) +#define PyBUF_FULL_RO (PyBUF_INDIRECT | PyBUF_FORMAT) + + +#define PyBUF_READ 0x100 +#define PyBUF_WRITE 0x200 + +#endif /* !Py_LIMITED_API || Py_LIMITED_API >= 3.11 */ + +#ifdef __cplusplus +} +#endif +#endif /* Py_BUFFER_H */ diff --git a/Include/cpython/abstract.h b/Include/cpython/abstract.h index 2876a7bb84f52..b5a31392f2ed0 100644 --- a/Include/cpython/abstract.h +++ b/Include/cpython/abstract.h @@ -168,74 +168,6 @@ PyAPI_FUNC(int) _PyObject_HasLen(PyObject *o); value. If one of the calls fails, this function returns -1. */ PyAPI_FUNC(Py_ssize_t) PyObject_LengthHint(PyObject *o, Py_ssize_t); -/* === New Buffer API ============================================ */ - -/* Return 1 if the getbuffer function is available, otherwise return 0. */ -PyAPI_FUNC(int) PyObject_CheckBuffer(PyObject *obj); - -/* This is a C-API version of the getbuffer function call. It checks - to make sure object has the required function pointer and issues the - call. - - Returns -1 and raises an error on failure and returns 0 on success. */ -PyAPI_FUNC(int) PyObject_GetBuffer(PyObject *obj, Py_buffer *view, - int flags); - -/* Get the memory area pointed to by the indices for the buffer given. - Note that view->ndim is the assumed size of indices. */ -PyAPI_FUNC(void *) PyBuffer_GetPointer(const Py_buffer *view, const Py_ssize_t *indices); - -/* Return the implied itemsize of the data-format area from a - struct-style description. */ -PyAPI_FUNC(Py_ssize_t) PyBuffer_SizeFromFormat(const char *format); - -/* Implementation in memoryobject.c */ -PyAPI_FUNC(int) PyBuffer_ToContiguous(void *buf, const Py_buffer *view, - Py_ssize_t len, char order); - -PyAPI_FUNC(int) PyBuffer_FromContiguous(const Py_buffer *view, const void *buf, - Py_ssize_t len, char order); - -/* Copy len bytes of data from the contiguous chunk of memory - pointed to by buf into the buffer exported by obj. Return - 0 on success and return -1 and raise a PyBuffer_Error on - error (i.e. the object does not have a buffer interface or - it is not working). - - If fort is 'F', then if the object is multi-dimensional, - then the data will be copied into the array in - Fortran-style (first dimension varies the fastest). If - fort is 'C', then the data will be copied into the array - in C-style (last dimension varies the fastest). If fort - is 'A', then it does not matter and the copy will be made - in whatever way is more efficient. */ -PyAPI_FUNC(int) PyObject_CopyData(PyObject *dest, PyObject *src); - -/* Copy the data from the src buffer to the buffer of destination. */ -PyAPI_FUNC(int) PyBuffer_IsContiguous(const Py_buffer *view, char fort); - -/*Fill the strides array with byte-strides of a contiguous - (Fortran-style if fort is 'F' or C-style otherwise) - array of the given shape with the given number of bytes - per element. */ -PyAPI_FUNC(void) PyBuffer_FillContiguousStrides(int ndims, - Py_ssize_t *shape, - Py_ssize_t *strides, - int itemsize, - char fort); - -/* Fills in a buffer-info structure correctly for an exporter - that can only share a contiguous chunk of memory of - "unsigned bytes" of the given length. - - Returns 0 on success and -1 (with raising an error) on error. */ -PyAPI_FUNC(int) PyBuffer_FillInfo(Py_buffer *view, PyObject *o, void *buf, - Py_ssize_t len, int readonly, - int flags); - -/* Releases a Py_buffer obtained from getbuffer ParseTuple's "s*". */ -PyAPI_FUNC(void) PyBuffer_Release(Py_buffer *view); - /* === Sequence protocol ================================================ */ /* Assume tp_as_sequence and sq_item exist and that 'i' does not diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 7b9f3acbc439d..1554ac8aef1c4 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -2,6 +2,8 @@ # error "this header file must not be included directly" #endif +#include "buffer.h" // for Py_buffer, included after PyObject has been defined + PyAPI_FUNC(void) _Py_NewReference(PyObject *op); #ifdef Py_TRACE_REFS @@ -45,61 +47,12 @@ typedef struct _Py_Identifier { #define _Py_static_string(varname, value) static _Py_Identifier varname = _Py_static_string_init(value) #define _Py_IDENTIFIER(varname) _Py_static_string(PyId_##varname, #varname) -/* buffer interface */ -typedef struct bufferinfo { - void *buf; - PyObject *obj; /* owned reference */ - Py_ssize_t len; - Py_ssize_t itemsize; /* This is Py_ssize_t so it can be - pointed to by strides in simple case.*/ - int readonly; - int ndim; - char *format; - Py_ssize_t *shape; - Py_ssize_t *strides; - Py_ssize_t *suboffsets; - void *internal; -} Py_buffer; - typedef int (*getbufferproc)(PyObject *, Py_buffer *, int); typedef void (*releasebufferproc)(PyObject *, Py_buffer *); typedef PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames); -/* Maximum number of dimensions */ -#define PyBUF_MAX_NDIM 64 - -/* Flags for getting buffers */ -#define PyBUF_SIMPLE 0 -#define PyBUF_WRITABLE 0x0001 -/* we used to include an E, backwards compatible alias */ -#define PyBUF_WRITEABLE PyBUF_WRITABLE -#define PyBUF_FORMAT 0x0004 -#define PyBUF_ND 0x0008 -#define PyBUF_STRIDES (0x0010 | PyBUF_ND) -#define PyBUF_C_CONTIGUOUS (0x0020 | PyBUF_STRIDES) -#define PyBUF_F_CONTIGUOUS (0x0040 | PyBUF_STRIDES) -#define PyBUF_ANY_CONTIGUOUS (0x0080 | PyBUF_STRIDES) -#define PyBUF_INDIRECT (0x0100 | PyBUF_STRIDES) - -#define PyBUF_CONTIG (PyBUF_ND | PyBUF_WRITABLE) -#define PyBUF_CONTIG_RO (PyBUF_ND) - -#define PyBUF_STRIDED (PyBUF_STRIDES | PyBUF_WRITABLE) -#define PyBUF_STRIDED_RO (PyBUF_STRIDES) - -#define PyBUF_RECORDS (PyBUF_STRIDES | PyBUF_WRITABLE | PyBUF_FORMAT) -#define PyBUF_RECORDS_RO (PyBUF_STRIDES | PyBUF_FORMAT) - -#define PyBUF_FULL (PyBUF_INDIRECT | PyBUF_WRITABLE | PyBUF_FORMAT) -#define PyBUF_FULL_RO (PyBUF_INDIRECT | PyBUF_FORMAT) - - -#define PyBUF_READ 0x100 -#define PyBUF_WRITE 0x200 -/* End buffer interface */ - typedef struct { /* Number implementations must check *both* diff --git a/Include/memoryobject.h b/Include/memoryobject.h index 0298cc9373068..154397ce1e56d 100644 --- a/Include/memoryobject.h +++ b/Include/memoryobject.h @@ -25,7 +25,7 @@ PyAPI_FUNC(PyObject *) PyMemoryView_FromObject(PyObject *base); PyAPI_FUNC(PyObject *) PyMemoryView_FromMemory(char *mem, Py_ssize_t size, int flags); #endif -#ifndef Py_LIMITED_API +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030b0000 PyAPI_FUNC(PyObject *) PyMemoryView_FromBuffer(const Py_buffer *info); #endif PyAPI_FUNC(PyObject *) PyMemoryView_GetContiguous(PyObject *base, diff --git a/Include/typeslots.h b/Include/typeslots.h index 5800d0158bc92..506b05580de14 100644 --- a/Include/typeslots.h +++ b/Include/typeslots.h @@ -1,12 +1,6 @@ /* Do not renumber the file; these numbers are part of the stable ABI. */ -#if defined(Py_LIMITED_API) -/* Disabled, see #10181 */ -#undef Py_bf_getbuffer -#undef Py_bf_releasebuffer -#else #define Py_bf_getbuffer 1 #define Py_bf_releasebuffer 2 -#endif #define Py_mp_ass_subscript 3 #define Py_mp_length 4 #define Py_mp_subscript 5 diff --git a/Lib/test/test_stable_abi_ctypes.py b/Lib/test/test_stable_abi_ctypes.py index 9fd6b14b0232a..a49235b81c1b0 100644 --- a/Lib/test/test_stable_abi_ctypes.py +++ b/Lib/test/test_stable_abi_ctypes.py @@ -28,6 +28,14 @@ def test_available_symbols(self): "PyBaseObject_Type", "PyBool_FromLong", "PyBool_Type", + "PyBuffer_FillContiguousStrides", + "PyBuffer_FillInfo", + "PyBuffer_FromContiguous", + "PyBuffer_GetPointer", + "PyBuffer_IsContiguous", + "PyBuffer_Release", + "PyBuffer_SizeFromFormat", + "PyBuffer_ToContiguous", "PyByteArrayIter_Type", "PyByteArray_AsString", "PyByteArray_Concat", @@ -381,6 +389,7 @@ def test_available_symbols(self): "PyMemberDescr_Type", "PyMember_GetOne", "PyMember_SetOne", + "PyMemoryView_FromBuffer", "PyMemoryView_FromMemory", "PyMemoryView_FromObject", "PyMemoryView_GetContiguous", @@ -470,8 +479,10 @@ def test_available_symbols(self): "PyObject_CallNoArgs", "PyObject_CallObject", "PyObject_Calloc", + "PyObject_CheckBuffer", "PyObject_CheckReadBuffer", "PyObject_ClearWeakRefs", + "PyObject_CopyData", "PyObject_DelItem", "PyObject_DelItemString", "PyObject_Dir", @@ -489,6 +500,7 @@ def test_available_symbols(self): "PyObject_GetAIter", "PyObject_GetAttr", "PyObject_GetAttrString", + "PyObject_GetBuffer", "PyObject_GetItem", "PyObject_GetIter", "PyObject_HasAttr", diff --git a/Lib/test/test_xxlimited.py b/Lib/test/test_xxlimited.py index e3f521d9b040d..6dbfb3f439393 100644 --- a/Lib/test/test_xxlimited.py +++ b/Lib/test/test_xxlimited.py @@ -58,6 +58,17 @@ def test_error(self): with self.assertRaises(self.module.Error): raise self.module.Error + def test_buffer(self): + xxo = self.module.Xxo() + self.assertEqual(xxo.x_exports, 0) + b1 = memoryview(xxo) + self.assertEqual(xxo.x_exports, 1) + b2 = memoryview(xxo) + self.assertEqual(xxo.x_exports, 2) + b1[0] = 1 + self.assertEqual(b1[0], 1) + self.assertEqual(b2[0], 1) + class TestXXLimited35(CommonTests, unittest.TestCase): module = xxlimited_35 diff --git a/Makefile.pre.in b/Makefile.pre.in index edc5fc3b6802f..4dcedd684aa6d 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1439,6 +1439,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/abstract.h \ $(srcdir)/Include/bltinmodule.h \ $(srcdir)/Include/boolobject.h \ + $(srcdir)/Include/buffer.h \ $(srcdir)/Include/bytearrayobject.h \ $(srcdir)/Include/bytesobject.h \ $(srcdir)/Include/ceval.h \ diff --git a/Misc/NEWS.d/next/C API/2021-10-18-16-54-24.bpo-45459.Y1pEZs.rst b/Misc/NEWS.d/next/C API/2021-10-18-16-54-24.bpo-45459.Y1pEZs.rst new file mode 100644 index 0000000000000..a8d93227817c4 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2021-10-18-16-54-24.bpo-45459.Y1pEZs.rst @@ -0,0 +1,2 @@ +:c:type:`Py_buffer` and various ``Py_buffer`` related functions are now +part of the limited API and stable ABI. diff --git a/Misc/stable_abi.txt b/Misc/stable_abi.txt index c4f5318712a54..cc3cc56d472d9 100644 --- a/Misc/stable_abi.txt +++ b/Misc/stable_abi.txt @@ -2191,6 +2191,34 @@ function PyType_GetQualName data PyStructSequence_UnnamedField added 3.11 +# Add stable Py_buffer API in Python 3.11 (https://bugs.python.org/issue45459) +struct Py_buffer + added 3.11 +function PyObject_CheckBuffer + added 3.11 +function PyObject_GetBuffer + added 3.11 +function PyBuffer_GetPointer + added 3.11 +function PyBuffer_SizeFromFormat + added 3.11 +function PyBuffer_ToContiguous + added 3.11 +function PyBuffer_FromContiguous + added 3.11 +function PyObject_CopyData + added 3.11 +function PyBuffer_IsContiguous + added 3.11 +function PyBuffer_FillContiguousStrides + added 3.11 +function PyBuffer_FillInfo + added 3.11 +function PyBuffer_Release + added 3.11 +function PyMemoryView_FromBuffer + added 3.11 + # (Detailed comments aren't really needed for further entries: from here on # we can use version control logs.) diff --git a/Modules/xxlimited.c b/Modules/xxlimited.c index 93895c4f1214c..16d1b8311c62c 100644 --- a/Modules/xxlimited.c +++ b/Modules/xxlimited.c @@ -19,6 +19,7 @@ def __init__(self): # In the C class, "_x_attr" is not accessible from Python code self._x_attr = {} + self._x_exports = 0 def __getattr__(self, name): return self._x_attr[name] @@ -29,6 +30,13 @@ def __delattr__(self, name): del self._x_attr[name] + @property + def x_exports(self): + """Return the number of times an internal buffer is exported.""" + # Each Xxo instance has a 10-byte buffer that can be + # accessed via the buffer interface (e.g. `memoryview`). + return self._x_exports + def demo(o, /): if isinstance(o, str): return o @@ -57,6 +65,9 @@ #define Py_LIMITED_API 0x030b0000 #include "Python.h" +#include + +#define BUFSIZE 10 // Module state typedef struct { @@ -70,7 +81,9 @@ typedef struct { // Instance state typedef struct { PyObject_HEAD - PyObject *x_attr; /* Attributes dictionary */ + PyObject *x_attr; /* Attributes dictionary */ + char x_buffer[BUFSIZE]; /* buffer for Py_buffer */ + Py_ssize_t x_exports; /* how many buffer are exported */ } XxoObject; // XXX: no good way to do this yet @@ -89,6 +102,8 @@ newXxoObject(PyObject *module) return NULL; } self->x_attr = NULL; + memset(self->x_buffer, 0, BUFSIZE); + self->x_exports = 0; return self; } @@ -212,11 +227,43 @@ static PyMethodDef Xxo_methods[] = { {NULL, NULL} /* sentinel */ }; +/* Xxo buffer interface */ + +static int +Xxo_getbuffer(XxoObject *self, Py_buffer *view, int flags) +{ + int res = PyBuffer_FillInfo(view, (PyObject*)self, + (void *)self->x_buffer, BUFSIZE, + 0, flags); + if (res == 0) { + self->x_exports++; + } + return res; +} + +static void +Xxo_releasebuffer(XxoObject *self, Py_buffer *view) +{ + self->x_exports--; +} + +static PyObject * +Xxo_get_x_exports(XxoObject *self, void *c) +{ + return PyLong_FromSsize_t(self->x_exports); +} + /* Xxo type definition */ PyDoc_STRVAR(Xxo_doc, "A class that explicitly stores attributes in an internal dict"); +static PyGetSetDef Xxo_getsetlist[] = { + {"x_exports", (getter) Xxo_get_x_exports, NULL, NULL}, + {NULL}, +}; + + static PyType_Slot Xxo_Type_slots[] = { {Py_tp_doc, (char *)Xxo_doc}, {Py_tp_traverse, Xxo_traverse}, @@ -226,6 +273,9 @@ static PyType_Slot Xxo_Type_slots[] = { {Py_tp_getattro, Xxo_getattro}, {Py_tp_setattro, Xxo_setattro}, {Py_tp_methods, Xxo_methods}, + {Py_bf_getbuffer, Xxo_getbuffer}, + {Py_bf_releasebuffer, Xxo_releasebuffer}, + {Py_tp_getset, Xxo_getsetlist}, {0, 0}, /* sentinel */ }; diff --git a/PC/python3dll.c b/PC/python3dll.c index b2bb1706c4a2e..70f11dc190554 100755 --- a/PC/python3dll.c +++ b/PC/python3dll.c @@ -94,6 +94,14 @@ EXPORT_FUNC(PyArg_ValidateKeywordArguments) EXPORT_FUNC(PyArg_VaParse) EXPORT_FUNC(PyArg_VaParseTupleAndKeywords) EXPORT_FUNC(PyBool_FromLong) +EXPORT_FUNC(PyBuffer_FillContiguousStrides) +EXPORT_FUNC(PyBuffer_FillInfo) +EXPORT_FUNC(PyBuffer_FromContiguous) +EXPORT_FUNC(PyBuffer_GetPointer) +EXPORT_FUNC(PyBuffer_IsContiguous) +EXPORT_FUNC(PyBuffer_Release) +EXPORT_FUNC(PyBuffer_SizeFromFormat) +EXPORT_FUNC(PyBuffer_ToContiguous) EXPORT_FUNC(PyByteArray_AsString) EXPORT_FUNC(PyByteArray_Concat) EXPORT_FUNC(PyByteArray_FromObject) @@ -352,6 +360,7 @@ EXPORT_FUNC(PyMem_Malloc) EXPORT_FUNC(PyMem_Realloc) EXPORT_FUNC(PyMember_GetOne) EXPORT_FUNC(PyMember_SetOne) +EXPORT_FUNC(PyMemoryView_FromBuffer) EXPORT_FUNC(PyMemoryView_FromMemory) EXPORT_FUNC(PyMemoryView_FromObject) EXPORT_FUNC(PyMemoryView_GetContiguous) @@ -426,8 +435,10 @@ EXPORT_FUNC(PyObject_CallMethodObjArgs) EXPORT_FUNC(PyObject_CallNoArgs) EXPORT_FUNC(PyObject_CallObject) EXPORT_FUNC(PyObject_Calloc) +EXPORT_FUNC(PyObject_CheckBuffer) EXPORT_FUNC(PyObject_CheckReadBuffer) EXPORT_FUNC(PyObject_ClearWeakRefs) +EXPORT_FUNC(PyObject_CopyData) EXPORT_FUNC(PyObject_DelItem) EXPORT_FUNC(PyObject_DelItemString) EXPORT_FUNC(PyObject_Dir) @@ -445,6 +456,7 @@ EXPORT_FUNC(PyObject_GenericSetDict) EXPORT_FUNC(PyObject_GetAIter) EXPORT_FUNC(PyObject_GetAttr) EXPORT_FUNC(PyObject_GetAttrString) +EXPORT_FUNC(PyObject_GetBuffer) EXPORT_FUNC(PyObject_GetItem) EXPORT_FUNC(PyObject_GetIter) EXPORT_FUNC(PyObject_HasAttr) From webhook-mailer at python.org Wed Feb 2 10:56:57 2022 From: webhook-mailer at python.org (markshannon) Date: Wed, 02 Feb 2022 15:56:57 -0000 Subject: [Python-checkins] Add specialization stats for FOR_ITER. (GH-31079) Message-ID: https://github.com/python/cpython/commit/0d05da1fbf39f32d2c22a1f3702f40f916997b60 commit: 0d05da1fbf39f32d2c22a1f3702f40f916997b60 branch: main author: Mark Shannon committer: markshannon date: 2022-02-02T15:56:47Z summary: Add specialization stats for FOR_ITER. (GH-31079) files: M Python/ceval.c M Python/specialize.c M Tools/scripts/summarize_stats.py diff --git a/Python/ceval.c b/Python/ceval.c index 70748e8911f9f..3c52c5824b44a 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4212,6 +4212,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PREDICTED(FOR_ITER); /* before: [iter]; after: [iter, iter()] *or* [] */ PyObject *iter = TOP(); +#ifdef Py_STATS + extern int _PySpecialization_ClassifyIterator(PyObject *); + _py_stats.opcode_stats[FOR_ITER].specialization.failure++; + _py_stats.opcode_stats[FOR_ITER].specialization.failure_kinds[_PySpecialization_ClassifyIterator(iter)]++; +#endif PyObject *next = (*Py_TYPE(iter)->tp_iternext)(iter); if (next != NULL) { PUSH(next); diff --git a/Python/specialize.c b/Python/specialize.c index 9290fbe823956..d90d7da8f3072 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -548,6 +548,23 @@ initial_counter_value(void) { #define SPEC_FAIL_NOT_FOLLOWED_BY_COND_JUMP 14 #define SPEC_FAIL_BIG_INT 15 +/* FOR_ITER */ +#define SPEC_FAIL_ITER_GENERATOR 10 +#define SPEC_FAIL_ITER_COROUTINE 11 +#define SPEC_FAIL_ITER_ASYNC_GENERATOR 12 +#define SPEC_FAIL_ITER_LIST 13 +#define SPEC_FAIL_ITER_TUPLE 14 +#define SPEC_FAIL_ITER_SET 15 +#define SPEC_FAIL_ITER_STRING 16 +#define SPEC_FAIL_ITER_BYTES 17 +#define SPEC_FAIL_ITER_RANGE 18 +#define SPEC_FAIL_ITER_ITERTOOLS 19 +#define SPEC_FAIL_ITER_DICT_KEYS 20 +#define SPEC_FAIL_ITER_DICT_ITEMS 21 +#define SPEC_FAIL_ITER_DICT_VALUES 22 +#define SPEC_FAIL_ITER_ENUMERATE 23 + + static int specialize_module_load_attr( PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, @@ -1817,3 +1834,54 @@ _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, STAT_INC(COMPARE_OP, success); adaptive->counter = initial_counter_value(); } + + +int + _PySpecialization_ClassifyIterator(PyObject *iter) +{ + if (PyGen_CheckExact(iter)) { + return SPEC_FAIL_ITER_GENERATOR; + } + if (PyCoro_CheckExact(iter)) { + return SPEC_FAIL_ITER_COROUTINE; + } + if (PyAsyncGen_CheckExact(iter)) { + return SPEC_FAIL_ITER_ASYNC_GENERATOR; + } + PyTypeObject *t = _Py_TYPE(iter); + if (t == &PyListIter_Type) { + return SPEC_FAIL_ITER_LIST; + } + if (t == &PyTupleIter_Type) { + return SPEC_FAIL_ITER_TUPLE; + } + if (t == &PyDictIterKey_Type) { + return SPEC_FAIL_ITER_DICT_KEYS; + } + if (t == &PyDictIterValue_Type) { + return SPEC_FAIL_ITER_DICT_VALUES; + } + if (t == &PyDictIterItem_Type) { + return SPEC_FAIL_ITER_DICT_ITEMS; + } + if (t == &PySetIter_Type) { + return SPEC_FAIL_ITER_SET; + } + if (t == &PyUnicodeIter_Type) { + return SPEC_FAIL_ITER_STRING; + } + if (t == &PyBytesIter_Type) { + return SPEC_FAIL_ITER_BYTES; + } + if (t == &PyRangeIter_Type) { + return SPEC_FAIL_ITER_RANGE; + } + if (t == &PyEnum_Type) { + return SPEC_FAIL_ITER_ENUMERATE; + } + + if (strncmp(t->tp_name, "itertools", 8) == 0) { + return SPEC_FAIL_ITER_ITERTOOLS; + } + return SPEC_FAIL_OTHER; +} diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index 319b251c854a8..f67a35a04a9dd 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -26,7 +26,7 @@ TOTAL = "specialization.deferred", "specialization.hit", "specialization.miss", "execution_count" def print_specialization_stats(name, family_stats): - if "specialization.deferred" not in family_stats: + if "specialization.failure" not in family_stats: return total = sum(family_stats.get(kind, 0) for kind in TOTAL) if total == 0: From webhook-mailer at python.org Wed Feb 2 10:57:55 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 15:57:55 -0000 Subject: [Python-checkins] bpo-46433: _PyType_GetModuleByDef: handle static types in MRO (GH-30696) Message-ID: https://github.com/python/cpython/commit/0ef08530124c5ca13a9394f4ac18bee8e6c66409 commit: 0ef08530124c5ca13a9394f4ac18bee8e6c66409 branch: main author: Petr Viktorin committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T07:57:51-08:00 summary: bpo-46433: _PyType_GetModuleByDef: handle static types in MRO (GH-30696) Automerge-Triggered-By: GH:encukou files: A Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst M Lib/test/test_capi.py M Modules/_testmultiphase.c M Modules/clinic/_testmultiphase.c.h M Objects/typeobject.c diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py index a5db8a11c5f67..ccf8ceda49831 100644 --- a/Lib/test/test_capi.py +++ b/Lib/test/test_capi.py @@ -1068,6 +1068,22 @@ def test_state_access(self): with self.assertRaises(TypeError): increment_count(1, 2, 3) + def test_get_module_bad_def(self): + # _PyType_GetModuleByDef fails gracefully if it doesn't + # find what it's looking for. + # see bpo-46433 + instance = self.module.StateAccessType() + with self.assertRaises(TypeError): + instance.getmodulebydef_bad_def() + + def test_get_module_static_in_mro(self): + # Here, the class _PyType_GetModuleByDef is looking for + # appears in the MRO after a static type (Exception). + # see bpo-46433 + class Subclass(BaseException, self.module.StateAccessType): + pass + self.assertIs(Subclass().get_defining_module(), self.module) + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst b/Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst new file mode 100644 index 0000000000000..e1987c4536b5c --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst @@ -0,0 +1,2 @@ +The internal function _PyType_GetModuleByDef now correctly handles +inheritance patterns involving static types. diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c index ee69c42336170..f7bde9895eb09 100644 --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -126,6 +126,8 @@ static PyType_Spec Example_Type_spec = { static PyModuleDef def_meth_state_access; +static PyModuleDef def_nonmodule; +static PyModuleDef def_nonmodule_with_methods; /*[clinic input] _testmultiphase.StateAccessType.get_defining_module @@ -153,6 +155,24 @@ _testmultiphase_StateAccessType_get_defining_module_impl(StateAccessTypeObject * return retval; } +/*[clinic input] +_testmultiphase.StateAccessType.getmodulebydef_bad_def + + cls: defining_class + +Test that result of _PyType_GetModuleByDef with a bad def is NULL. +[clinic start generated code]*/ + +static PyObject * +_testmultiphase_StateAccessType_getmodulebydef_bad_def_impl(StateAccessTypeObject *self, + PyTypeObject *cls) +/*[clinic end generated code: output=64509074dfcdbd31 input=906047715ee293cd]*/ +{ + _PyType_GetModuleByDef(Py_TYPE(self), &def_nonmodule); // should raise + assert(PyErr_Occurred()); + return NULL; +} + /*[clinic input] _testmultiphase.StateAccessType.increment_count_clinic @@ -249,6 +269,7 @@ _testmultiphase_StateAccessType_get_count_impl(StateAccessTypeObject *self, static PyMethodDef StateAccessType_methods[] = { _TESTMULTIPHASE_STATEACCESSTYPE_GET_DEFINING_MODULE_METHODDEF + _TESTMULTIPHASE_STATEACCESSTYPE_GETMODULEBYDEF_BAD_DEF_METHODDEF _TESTMULTIPHASE_STATEACCESSTYPE_GET_COUNT_METHODDEF _TESTMULTIPHASE_STATEACCESSTYPE_INCREMENT_COUNT_CLINIC_METHODDEF { @@ -437,9 +458,6 @@ PyInit__testmultiphase(PyObject *spec) /**** Importing a non-module object ****/ -static PyModuleDef def_nonmodule; -static PyModuleDef def_nonmodule_with_methods; - /* Create a SimpleNamespace(three=3) */ static PyObject* createfunc_nonmodule(PyObject *spec, PyModuleDef *def) diff --git a/Modules/clinic/_testmultiphase.c.h b/Modules/clinic/_testmultiphase.c.h index 55f934be8c6c1..17c28d54ce575 100644 --- a/Modules/clinic/_testmultiphase.c.h +++ b/Modules/clinic/_testmultiphase.c.h @@ -35,6 +35,36 @@ _testmultiphase_StateAccessType_get_defining_module(StateAccessTypeObject *self, return return_value; } +PyDoc_STRVAR(_testmultiphase_StateAccessType_getmodulebydef_bad_def__doc__, +"getmodulebydef_bad_def($self, /)\n" +"--\n" +"\n" +"Test that result of _PyType_GetModuleByDef with a bad def is NULL."); + +#define _TESTMULTIPHASE_STATEACCESSTYPE_GETMODULEBYDEF_BAD_DEF_METHODDEF \ + {"getmodulebydef_bad_def", (PyCFunction)(void(*)(void))_testmultiphase_StateAccessType_getmodulebydef_bad_def, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _testmultiphase_StateAccessType_getmodulebydef_bad_def__doc__}, + +static PyObject * +_testmultiphase_StateAccessType_getmodulebydef_bad_def_impl(StateAccessTypeObject *self, + PyTypeObject *cls); + +static PyObject * +_testmultiphase_StateAccessType_getmodulebydef_bad_def(StateAccessTypeObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":getmodulebydef_bad_def", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = _testmultiphase_StateAccessType_getmodulebydef_bad_def_impl(self, cls); + +exit: + return return_value; +} + PyDoc_STRVAR(_testmultiphase_StateAccessType_increment_count_clinic__doc__, "increment_count_clinic($self, /, n=1, *, twice=False)\n" "--\n" @@ -101,4 +131,4 @@ _testmultiphase_StateAccessType_get_count(StateAccessTypeObject *self, PyTypeObj exit: return return_value; } -/*[clinic end generated code: output=f01137bb3b373e14 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=eb1b8c2ee6290be3 input=a9049054013a1b77]*/ diff --git a/Objects/typeobject.c b/Objects/typeobject.c index f7e0775e2225b..0c893eaa75b6f 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3756,11 +3756,10 @@ _PyType_GetModuleByDef(PyTypeObject *type, struct PyModuleDef *def) Py_ssize_t n = PyTuple_GET_SIZE(mro); for (Py_ssize_t i = 0; i < n; i++) { PyObject *super = PyTuple_GET_ITEM(mro, i); - // _PyType_GetModuleByDef() must only be called on a heap type created - // by PyType_FromModuleAndSpec() or on its subclasses. - // type_ready_mro() ensures that a static type cannot inherit from a - // heap type. - assert(_PyType_HasFeature(type, Py_TPFLAGS_HEAPTYPE)); + if(!_PyType_HasFeature((PyTypeObject *)super, Py_TPFLAGS_HEAPTYPE)) { + // Static types in the MRO need to be skipped + continue; + } PyHeapTypeObject *ht = (PyHeapTypeObject*)super; PyObject *module = ht->ht_module; From webhook-mailer at python.org Wed Feb 2 11:23:47 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 16:23:47 -0000 Subject: [Python-checkins] bpo-37705: Remove orphaned PC/errmap.mak (GH-29724) Message-ID: https://github.com/python/cpython/commit/38e0b9efdf164be656782db0f969fb536cfbcaf1 commit: 38e0b9efdf164be656782db0f969fb536cfbcaf1 branch: main author: Oleg Iarygin committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T08:23:30-08:00 summary: bpo-37705: Remove orphaned PC/errmap.mak (GH-29724) After GH-15623 deleted `generrmap.c`, a related mak-file stopped working. The mak contains generrmap-related rules only so it should be removed altogether. Further search for `errmap\.mak|generrmap` regex through content of CPython files shows no dangling reference left. Since generrmap is already effectively removed, this pull request contains no blurp. files: D PC/errmap.mak diff --git a/PC/errmap.mak b/PC/errmap.mak deleted file mode 100644 index 646bcd0a2096f..0000000000000 --- a/PC/errmap.mak +++ /dev/null @@ -1,5 +0,0 @@ -errmap.h: generrmap.exe - .\generrmap.exe > errmap.h - -genermap.exe: generrmap.c - cl generrmap.c From webhook-mailer at python.org Wed Feb 2 11:59:48 2022 From: webhook-mailer at python.org (gpshead) Date: Wed, 02 Feb 2022 16:59:48 -0000 Subject: [Python-checkins] bpo-45173: Keep configparser deprecations until Python 3.12 (GH-30952) Message-ID: https://github.com/python/cpython/commit/e8659b47dece5a272111c0af5e340c364a9f807b commit: e8659b47dece5a272111c0af5e340c364a9f807b branch: main author: Hugo van Kemenade committer: gpshead date: 2022-02-02T08:59:39-08:00 summary: bpo-45173: Keep configparser deprecations until Python 3.12 (GH-30952) * Revert "bpo-45173 Remove configparser deprecations" This reverts commit df2284bc416dcccba1125b12af4499c45baabe4f. * bpo-45173: Note these configparser deprecations will be removed in 3.12 files: A Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst M Doc/library/configparser.rst M Doc/whatsnew/3.11.rst M Lib/configparser.py M Lib/test/test_configparser.py diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index d31452edb974f..1ebda53ecda0f 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -1201,6 +1201,28 @@ ConfigParser Objects names is stripped before :meth:`optionxform` is called. + .. method:: readfp(fp, filename=None) + + .. deprecated:: 3.2 + Use :meth:`read_file` instead. + + .. versionchanged:: 3.2 + :meth:`readfp` now iterates on *fp* instead of calling ``fp.readline()``. + + For existing code calling :meth:`readfp` with arguments which don't + support iteration, the following generator may be used as a wrapper + around the file-like object:: + + def readline_generator(fp): + line = fp.readline() + while line: + yield line + line = fp.readline() + + Instead of ``parser.readfp(fp)`` use + ``parser.read_file(readline_generator(fp))``. + + .. data:: MAX_INTERPOLATION_DEPTH The maximum depth for recursive interpolation for :meth:`get` when the *raw* @@ -1338,9 +1360,6 @@ Exceptions The ``filename`` attribute and :meth:`__init__` argument were renamed to ``source`` for consistency. - .. versionchanged:: 3.11 - The deprecated ``filename`` attribute was removed. - .. rubric:: Footnotes diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 3458ad63c9df8..33f39e5775269 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -458,6 +458,16 @@ Deprecated as deprecated, its docstring is now corrected). (Contributed by Hugo van Kemenade in :issue:`45837`.) +* The following have been deprecated in :mod:`configparser` since Python 3.2. + Their deprecation warnings have now been updated to note they will removed in + Python 3.12: + + * the :class:`configparser.SafeConfigParser` class + * the :attr:`configparser.ParsingError.filename` property + * the :meth:`configparser.ParsingError.readfp` method + + (Contributed by Hugo van Kemenade in :issue:`45173`.) + Removed ======= @@ -502,13 +512,6 @@ Removed the ``l*gettext()`` functions. (Contributed by Dong-hee Na and Serhiy Storchaka in :issue:`44235`.) -* Removed from the :mod:`configparser` module: - the :class:`SafeConfigParser` class, - the :attr:`filename` property of the :class:`ParsingError` class, - the :meth:`readfp` method of the :class:`ConfigParser` class, - deprecated since Python 3.2. - (Contributed by Hugo van Kemenade in :issue:`45173`.) - * The :func:`@asyncio.coroutine ` :term:`decorator` enabling legacy generator-based coroutines to be compatible with async/await code. The function has been deprecated since Python 3.8 and the removal was diff --git a/Lib/configparser.py b/Lib/configparser.py index c10309acf1ab4..3470624e63f61 100644 --- a/Lib/configparser.py +++ b/Lib/configparser.py @@ -146,12 +146,13 @@ import os import re import sys +import warnings __all__ = ["NoSectionError", "DuplicateOptionError", "DuplicateSectionError", "NoOptionError", "InterpolationError", "InterpolationDepthError", "InterpolationMissingOptionError", "InterpolationSyntaxError", "ParsingError", "MissingSectionHeaderError", - "ConfigParser", "RawConfigParser", + "ConfigParser", "SafeConfigParser", "RawConfigParser", "Interpolation", "BasicInterpolation", "ExtendedInterpolation", "LegacyInterpolation", "SectionProxy", "ConverterMapping", "DEFAULTSECT", "MAX_INTERPOLATION_DEPTH"] @@ -311,6 +312,26 @@ def __init__(self, source=None, filename=None): self.errors = [] self.args = (source, ) + @property + def filename(self): + """Deprecated, use `source'.""" + warnings.warn( + "The 'filename' attribute will be removed in Python 3.12. " + "Use 'source' instead.", + DeprecationWarning, stacklevel=2 + ) + return self.source + + @filename.setter + def filename(self, value): + """Deprecated, user `source'.""" + warnings.warn( + "The 'filename' attribute will be removed in Python 3.12. " + "Use 'source' instead.", + DeprecationWarning, stacklevel=2 + ) + self.source = value + def append(self, lineno, line): self.errors.append((lineno, line)) self.message += '\n\t[line %2d]: %s' % (lineno, line) @@ -733,6 +754,15 @@ def read_dict(self, dictionary, source=''): elements_added.add((section, key)) self.set(section, key, value) + def readfp(self, fp, filename=None): + """Deprecated, use read_file instead.""" + warnings.warn( + "This method will be removed in Python 3.12. " + "Use 'parser.read_file()' instead.", + DeprecationWarning, stacklevel=2 + ) + self.read_file(fp, source=filename) + def get(self, section, option, *, raw=False, vars=None, fallback=_UNSET): """Get an option value for a given section. @@ -1195,6 +1225,19 @@ def _read_defaults(self, defaults): self._interpolation = hold_interpolation +class SafeConfigParser(ConfigParser): + """ConfigParser alias for backwards compatibility purposes.""" + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + warnings.warn( + "The SafeConfigParser class has been renamed to ConfigParser " + "in Python 3.2. This alias will be removed in Python 3.12." + " Use ConfigParser directly instead.", + DeprecationWarning, stacklevel=2 + ) + + class SectionProxy(MutableMapping): """A proxy for a single section from a parser.""" diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py index cedf505122d13..e9b03e6c62ef1 100644 --- a/Lib/test/test_configparser.py +++ b/Lib/test/test_configparser.py @@ -1612,6 +1612,13 @@ def test_parsing_error(self): "and `source'. Use `source'.") error = configparser.ParsingError(filename='source') self.assertEqual(error.source, 'source') + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always", DeprecationWarning) + self.assertEqual(error.filename, 'source') + error.filename = 'filename' + self.assertEqual(error.source, 'filename') + for warning in w: + self.assertTrue(warning.category is DeprecationWarning) def test_interpolation_validation(self): parser = configparser.ConfigParser() @@ -1630,6 +1637,27 @@ def test_interpolation_validation(self): self.assertEqual(str(cm.exception), "bad interpolation variable " "reference '%(()'") + def test_readfp_deprecation(self): + sio = io.StringIO(""" + [section] + option = value + """) + parser = configparser.ConfigParser() + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always", DeprecationWarning) + parser.readfp(sio, filename='StringIO') + for warning in w: + self.assertTrue(warning.category is DeprecationWarning) + self.assertEqual(len(parser), 2) + self.assertEqual(parser['section']['option'], 'value') + + def test_safeconfigparser_deprecation(self): + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always", DeprecationWarning) + parser = configparser.SafeConfigParser() + for warning in w: + self.assertTrue(warning.category is DeprecationWarning) + def test_sectionproxy_repr(self): parser = configparser.ConfigParser() parser.read_string(""" diff --git a/Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst b/Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst new file mode 100644 index 0000000000000..ee5a88f621498 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst @@ -0,0 +1 @@ +Note the configparser deprecations will be removed in Python 3.12. From webhook-mailer at python.org Wed Feb 2 13:41:39 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 18:41:39 -0000 Subject: [Python-checkins] [3.10] bpo-45173: Note configparser deprecations will be removed in 3.12 (GH-31084) Message-ID: https://github.com/python/cpython/commit/ba4d79af32b6bb8378cb7003f67d6c4d413fbe89 commit: ba4d79af32b6bb8378cb7003f67d6c4d413fbe89 branch: 3.10 author: Hugo van Kemenade committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T10:41:30-08:00 summary: [3.10] bpo-45173: Note configparser deprecations will be removed in 3.12 (GH-31084) Cherry-pick of [`b06e9ba`](https://github.com/python/cpython/pull/30952/commits/b06e9ba398fafe39028c3b2dc3943266a16b1416) from https://github.com/python/cpython/pull/30952. files: A Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst M Lib/configparser.py diff --git a/Lib/configparser.py b/Lib/configparser.py index 042a5c74b696f..3470624e63f61 100644 --- a/Lib/configparser.py +++ b/Lib/configparser.py @@ -316,7 +316,7 @@ def __init__(self, source=None, filename=None): def filename(self): """Deprecated, use `source'.""" warnings.warn( - "The 'filename' attribute will be removed in future versions. " + "The 'filename' attribute will be removed in Python 3.12. " "Use 'source' instead.", DeprecationWarning, stacklevel=2 ) @@ -326,7 +326,7 @@ def filename(self): def filename(self, value): """Deprecated, user `source'.""" warnings.warn( - "The 'filename' attribute will be removed in future versions. " + "The 'filename' attribute will be removed in Python 3.12. " "Use 'source' instead.", DeprecationWarning, stacklevel=2 ) @@ -757,7 +757,7 @@ def read_dict(self, dictionary, source=''): def readfp(self, fp, filename=None): """Deprecated, use read_file instead.""" warnings.warn( - "This method will be removed in future versions. " + "This method will be removed in Python 3.12. " "Use 'parser.read_file()' instead.", DeprecationWarning, stacklevel=2 ) @@ -1232,7 +1232,7 @@ def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) warnings.warn( "The SafeConfigParser class has been renamed to ConfigParser " - "in Python 3.2. This alias will be removed in future versions." + "in Python 3.2. This alias will be removed in Python 3.12." " Use ConfigParser directly instead.", DeprecationWarning, stacklevel=2 ) diff --git a/Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst b/Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst new file mode 100644 index 0000000000000..ee5a88f621498 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst @@ -0,0 +1 @@ +Note the configparser deprecations will be removed in Python 3.12. From webhook-mailer at python.org Wed Feb 2 14:54:32 2022 From: webhook-mailer at python.org (zooba) Date: Wed, 02 Feb 2022 19:54:32 -0000 Subject: [Python-checkins] bpo-46616: Ensures test_importlib.test_windows cleans up registry keys after completion (GH-31086) Message-ID: https://github.com/python/cpython/commit/89a0a90c2e0e685bc70206fc45e4413c4f4411ed commit: 89a0a90c2e0e685bc70206fc45e4413c4f4411ed branch: main author: Steve Dower committer: zooba date: 2022-02-02T19:54:27Z summary: bpo-46616: Ensures test_importlib.test_windows cleans up registry keys after completion (GH-31086) files: A Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst M Lib/test/test_importlib/test_windows.py diff --git a/Lib/test/test_importlib/test_windows.py b/Lib/test/test_importlib/test_windows.py index b3e8e7e6d63fc..b7dfe865a03a9 100644 --- a/Lib/test/test_importlib/test_windows.py +++ b/Lib/test/test_importlib/test_windows.py @@ -60,17 +60,28 @@ def setup_module(machinery, name, path=None): root = machinery.WindowsRegistryFinder.REGISTRY_KEY key = root.format(fullname=name, sys_version='%d.%d' % sys.version_info[:2]) + base_key = "Software\\Python\\PythonCore\\{}.{}".format( + sys.version_info.major, sys.version_info.minor) + assert key.casefold().startswith(base_key.casefold()), ( + "expected key '{}' to start with '{}'".format(key, base_key)) try: with temp_module(name, "a = 1") as location: + try: + OpenKey(HKEY_CURRENT_USER, base_key) + if machinery.WindowsRegistryFinder.DEBUG_BUILD: + delete_key = os.path.dirname(key) + else: + delete_key = key + except OSError: + delete_key = base_key subkey = CreateKey(HKEY_CURRENT_USER, key) if path is None: path = location + ".py" SetValue(subkey, "", REG_SZ, path) yield finally: - if machinery.WindowsRegistryFinder.DEBUG_BUILD: - key = os.path.dirname(key) - delete_registry_tree(HKEY_CURRENT_USER, key) + if delete_key: + delete_registry_tree(HKEY_CURRENT_USER, delete_key) @unittest.skipUnless(sys.platform.startswith('win'), 'requires Windows') diff --git a/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst b/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst new file mode 100644 index 0000000000000..31c63c3d8f181 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst @@ -0,0 +1,2 @@ +Ensures ``test_importlib.test_windows`` cleans up registry keys after +completion. From webhook-mailer at python.org Wed Feb 2 15:15:26 2022 From: webhook-mailer at python.org (gpshead) Date: Wed, 02 Feb 2022 20:15:26 -0000 Subject: [Python-checkins] bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (#31015) Message-ID: https://github.com/python/cpython/commit/164a017e13ee96bd1ea1ae79f5ac9e25fe83994e commit: 164a017e13ee96bd1ea1ae79f5ac9e25fe83994e branch: main author: Gregory P. Smith committer: gpshead date: 2022-02-02T12:15:16-08:00 summary: bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (#31015) Disable compiler optimization within test_peg_generator. This speed up test_peg_generator by always disabling compiler optimizations by using -O0 or equivalent when the test is building its own C extensions. A build not using --with-pydebug in order to speed up test execution winds up with this test taking a very long time as it would do repeated compilation of parser C code using the same optimization flags as CPython was built with. This speeds the test up 6-8x on gps-raspbian. Also incorporate's #31017's win32 conditional and flags. Co-authored-by: Kumar Aditya kumaraditya303 files: A Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst M Tools/peg_generator/pegen/build.py diff --git a/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst b/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst new file mode 100644 index 0000000000000..be50fc8cbe0a5 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst @@ -0,0 +1,3 @@ +test_peg_generator now disables compiler optimization when testing +compilation of its own C extensions to significantly speed up the +testing on non-debug builds of CPython. diff --git a/Tools/peg_generator/pegen/build.py b/Tools/peg_generator/pegen/build.py index c69e5c9a5f26a..78789b94df2e4 100644 --- a/Tools/peg_generator/pegen/build.py +++ b/Tools/peg_generator/pegen/build.py @@ -1,6 +1,7 @@ import itertools import pathlib import shutil +import sys import sysconfig import tempfile import tokenize @@ -32,6 +33,7 @@ def compile_c_extension( build_dir: Optional[str] = None, verbose: bool = False, keep_asserts: bool = True, + disable_optimization: bool = True, # Significant test_peg_generator speedup. ) -> str: """Compile the generated source for a parser generator into an extension module. @@ -61,6 +63,14 @@ def compile_c_extension( extra_link_args = get_extra_flags("LDFLAGS", "PY_LDFLAGS_NODIST") if keep_asserts: extra_compile_args.append("-UNDEBUG") + if disable_optimization: + if sys.platform == 'win32': + extra_compile_args.append("/Od") + extra_link_args.append("/LTCG:OFF") + else: + extra_compile_args.append("-O0") + if sysconfig.get_config_var("GNULD") == "yes": + extra_link_args.append("-fno-lto") extension = [ Extension( extension_name, From webhook-mailer at python.org Wed Feb 2 15:23:59 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 20:23:59 -0000 Subject: [Python-checkins] bpo-46616: Ensures test_importlib.test_windows cleans up registry keys after completion (GH-31086) Message-ID: https://github.com/python/cpython/commit/3c6173ca67c019f3eb7a2fc34c5bfc426f99c5b2 commit: 3c6173ca67c019f3eb7a2fc34c5bfc426f99c5b2 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T12:23:49-08:00 summary: bpo-46616: Ensures test_importlib.test_windows cleans up registry keys after completion (GH-31086) (cherry picked from commit 89a0a90c2e0e685bc70206fc45e4413c4f4411ed) Co-authored-by: Steve Dower files: A Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst M Lib/test/test_importlib/test_windows.py diff --git a/Lib/test/test_importlib/test_windows.py b/Lib/test/test_importlib/test_windows.py index 9a53e8dbee56f..1c0abd3a872a6 100644 --- a/Lib/test/test_importlib/test_windows.py +++ b/Lib/test/test_importlib/test_windows.py @@ -41,17 +41,28 @@ def setup_module(machinery, name, path=None): root = machinery.WindowsRegistryFinder.REGISTRY_KEY key = root.format(fullname=name, sys_version='%d.%d' % sys.version_info[:2]) + base_key = "Software\\Python\\PythonCore\\{}.{}".format( + sys.version_info.major, sys.version_info.minor) + assert key.casefold().startswith(base_key.casefold()), ( + "expected key '{}' to start with '{}'".format(key, base_key)) try: with temp_module(name, "a = 1") as location: + try: + OpenKey(HKEY_CURRENT_USER, base_key) + if machinery.WindowsRegistryFinder.DEBUG_BUILD: + delete_key = os.path.dirname(key) + else: + delete_key = key + except OSError: + delete_key = base_key subkey = CreateKey(HKEY_CURRENT_USER, key) if path is None: path = location + ".py" SetValue(subkey, "", REG_SZ, path) yield finally: - if machinery.WindowsRegistryFinder.DEBUG_BUILD: - key = os.path.dirname(key) - delete_registry_tree(HKEY_CURRENT_USER, key) + if delete_key: + delete_registry_tree(HKEY_CURRENT_USER, delete_key) @unittest.skipUnless(sys.platform.startswith('win'), 'requires Windows') diff --git a/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst b/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst new file mode 100644 index 0000000000000..31c63c3d8f181 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst @@ -0,0 +1,2 @@ +Ensures ``test_importlib.test_windows`` cleans up registry keys after +completion. From webhook-mailer at python.org Wed Feb 2 15:25:50 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 02 Feb 2022 20:25:50 -0000 Subject: [Python-checkins] bpo-46616: Ensures test_importlib.test_windows cleans up registry keys after completion (GH-31086) Message-ID: https://github.com/python/cpython/commit/5765eaa13654e5f812a286700da7d6b8e144da0e commit: 5765eaa13654e5f812a286700da7d6b8e144da0e branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T12:25:37-08:00 summary: bpo-46616: Ensures test_importlib.test_windows cleans up registry keys after completion (GH-31086) (cherry picked from commit 89a0a90c2e0e685bc70206fc45e4413c4f4411ed) Co-authored-by: Steve Dower files: A Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst M Lib/test/test_importlib/test_windows.py diff --git a/Lib/test/test_importlib/test_windows.py b/Lib/test/test_importlib/test_windows.py index 6f09c5a7a5a46..1b9e185c19545 100644 --- a/Lib/test/test_importlib/test_windows.py +++ b/Lib/test/test_importlib/test_windows.py @@ -61,17 +61,28 @@ def setup_module(machinery, name, path=None): root = machinery.WindowsRegistryFinder.REGISTRY_KEY key = root.format(fullname=name, sys_version='%d.%d' % sys.version_info[:2]) + base_key = "Software\\Python\\PythonCore\\{}.{}".format( + sys.version_info.major, sys.version_info.minor) + assert key.casefold().startswith(base_key.casefold()), ( + "expected key '{}' to start with '{}'".format(key, base_key)) try: with temp_module(name, "a = 1") as location: + try: + OpenKey(HKEY_CURRENT_USER, base_key) + if machinery.WindowsRegistryFinder.DEBUG_BUILD: + delete_key = os.path.dirname(key) + else: + delete_key = key + except OSError: + delete_key = base_key subkey = CreateKey(HKEY_CURRENT_USER, key) if path is None: path = location + ".py" SetValue(subkey, "", REG_SZ, path) yield finally: - if machinery.WindowsRegistryFinder.DEBUG_BUILD: - key = os.path.dirname(key) - delete_registry_tree(HKEY_CURRENT_USER, key) + if delete_key: + delete_registry_tree(HKEY_CURRENT_USER, delete_key) @unittest.skipUnless(sys.platform.startswith('win'), 'requires Windows') diff --git a/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst b/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst new file mode 100644 index 0000000000000..31c63c3d8f181 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst @@ -0,0 +1,2 @@ +Ensures ``test_importlib.test_windows`` cleans up registry keys after +completion. From webhook-mailer at python.org Wed Feb 2 20:59:32 2022 From: webhook-mailer at python.org (terryjreedy) Date: Thu, 03 Feb 2022 01:59:32 -0000 Subject: [Python-checkins] bpo-45975: Use walrus operator for some idlelib while loops (GH-31083) Message-ID: https://github.com/python/cpython/commit/51a95be1d035a717ab29e98056b8831a98e61125 commit: 51a95be1d035a717ab29e98056b8831a98e61125 branch: main author: Nick Drozd committer: terryjreedy date: 2022-02-02T20:59:24-05:00 summary: bpo-45975: Use walrus operator for some idlelib while loops (GH-31083) files: M Lib/idlelib/idle_test/test_sidebar.py M Lib/idlelib/pyparse.py M Lib/idlelib/replace.py M Lib/idlelib/run.py M Lib/idlelib/sidebar.py diff --git a/Lib/idlelib/idle_test/test_sidebar.py b/Lib/idlelib/idle_test/test_sidebar.py index 53ac3eb27335d..01fd6a04d0de3 100644 --- a/Lib/idlelib/idle_test/test_sidebar.py +++ b/Lib/idlelib/idle_test/test_sidebar.py @@ -474,10 +474,7 @@ def get_shell_line_y_coords(self): index = text.index("@0,0") if index.split('.', 1)[1] != '0': index = text.index(f"{index} +1line linestart") - while True: - lineinfo = text.dlineinfo(index) - if lineinfo is None: - break + while (lineinfo := text.dlineinfo(index)) is not None: y_coords.append(lineinfo[1]) index = text.index(f"{index} +1line") return y_coords diff --git a/Lib/idlelib/pyparse.py b/Lib/idlelib/pyparse.py index d34872b4396e1..a94327533d865 100644 --- a/Lib/idlelib/pyparse.py +++ b/Lib/idlelib/pyparse.py @@ -179,14 +179,10 @@ def find_good_parse_start(self, is_char_in_string): # Peeking back worked; look forward until _synchre no longer # matches. i = pos + 1 - while 1: - m = _synchre(code, i) - if m: - s, i = m.span() - if not is_char_in_string(s): - pos = s - else: - break + while (m := _synchre(code, i)): + s, i = m.span() + if not is_char_in_string(s): + pos = s return pos def set_lo(self, lo): diff --git a/Lib/idlelib/replace.py b/Lib/idlelib/replace.py index 2f9ca231a05e4..ac04ed94dd475 100644 --- a/Lib/idlelib/replace.py +++ b/Lib/idlelib/replace.py @@ -158,11 +158,8 @@ def replace_all(self, event=None): first = last = None # XXX ought to replace circular instead of top-to-bottom when wrapping text.undo_block_start() - while True: - res = self.engine.search_forward(text, prog, line, col, - wrap=False, ok=ok) - if not res: - break + while (res := self.engine.search_forward( + text, prog, line, col, wrap=False, ok=ok)): line, m = res chars = text.get("%d.0" % line, "%d.0" % (line+1)) orig = m.group() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py index 47c4cbdcb8c3f..01f8d65426abc 100644 --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -482,9 +482,7 @@ def read(self, size=-1): result = self._line_buffer self._line_buffer = '' if size < 0: - while True: - line = self.shell.readline() - if not line: break + while (line := self.shell.readline()): result += line else: while len(result) < size: diff --git a/Lib/idlelib/sidebar.py b/Lib/idlelib/sidebar.py index 018c368f421c6..fb1084dbf3f18 100644 --- a/Lib/idlelib/sidebar.py +++ b/Lib/idlelib/sidebar.py @@ -471,10 +471,7 @@ def update_sidebar(self): index = text.index("@0,0") if index.split('.', 1)[1] != '0': index = text.index(f'{index}+1line linestart') - while True: - lineinfo = text.dlineinfo(index) - if lineinfo is None: - break + while (lineinfo := text.dlineinfo(index)) is not None: y = lineinfo[1] prev_newline_tagnames = text_tagnames(f"{index} linestart -1c") prompt = ( From webhook-mailer at python.org Wed Feb 2 21:28:57 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 03 Feb 2022 02:28:57 -0000 Subject: [Python-checkins] bpo-45975: Use walrus operator for some idlelib while loops (GH-31083) Message-ID: https://github.com/python/cpython/commit/2ddc278875f789de622262ee8ff5a1c3788f031c commit: 2ddc278875f789de622262ee8ff5a1c3788f031c branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T18:28:52-08:00 summary: bpo-45975: Use walrus operator for some idlelib while loops (GH-31083) (cherry picked from commit 51a95be1d035a717ab29e98056b8831a98e61125) Co-authored-by: Nick Drozd files: M Lib/idlelib/idle_test/test_sidebar.py M Lib/idlelib/pyparse.py M Lib/idlelib/replace.py M Lib/idlelib/run.py M Lib/idlelib/sidebar.py diff --git a/Lib/idlelib/idle_test/test_sidebar.py b/Lib/idlelib/idle_test/test_sidebar.py index 53ac3eb27335d..01fd6a04d0de3 100644 --- a/Lib/idlelib/idle_test/test_sidebar.py +++ b/Lib/idlelib/idle_test/test_sidebar.py @@ -474,10 +474,7 @@ def get_shell_line_y_coords(self): index = text.index("@0,0") if index.split('.', 1)[1] != '0': index = text.index(f"{index} +1line linestart") - while True: - lineinfo = text.dlineinfo(index) - if lineinfo is None: - break + while (lineinfo := text.dlineinfo(index)) is not None: y_coords.append(lineinfo[1]) index = text.index(f"{index} +1line") return y_coords diff --git a/Lib/idlelib/pyparse.py b/Lib/idlelib/pyparse.py index d34872b4396e1..a94327533d865 100644 --- a/Lib/idlelib/pyparse.py +++ b/Lib/idlelib/pyparse.py @@ -179,14 +179,10 @@ def find_good_parse_start(self, is_char_in_string): # Peeking back worked; look forward until _synchre no longer # matches. i = pos + 1 - while 1: - m = _synchre(code, i) - if m: - s, i = m.span() - if not is_char_in_string(s): - pos = s - else: - break + while (m := _synchre(code, i)): + s, i = m.span() + if not is_char_in_string(s): + pos = s return pos def set_lo(self, lo): diff --git a/Lib/idlelib/replace.py b/Lib/idlelib/replace.py index 2f9ca231a05e4..ac04ed94dd475 100644 --- a/Lib/idlelib/replace.py +++ b/Lib/idlelib/replace.py @@ -158,11 +158,8 @@ def replace_all(self, event=None): first = last = None # XXX ought to replace circular instead of top-to-bottom when wrapping text.undo_block_start() - while True: - res = self.engine.search_forward(text, prog, line, col, - wrap=False, ok=ok) - if not res: - break + while (res := self.engine.search_forward( + text, prog, line, col, wrap=False, ok=ok)): line, m = res chars = text.get("%d.0" % line, "%d.0" % (line+1)) orig = m.group() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py index 47c4cbdcb8c3f..01f8d65426abc 100644 --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -482,9 +482,7 @@ def read(self, size=-1): result = self._line_buffer self._line_buffer = '' if size < 0: - while True: - line = self.shell.readline() - if not line: break + while (line := self.shell.readline()): result += line else: while len(result) < size: diff --git a/Lib/idlelib/sidebar.py b/Lib/idlelib/sidebar.py index 018c368f421c6..fb1084dbf3f18 100644 --- a/Lib/idlelib/sidebar.py +++ b/Lib/idlelib/sidebar.py @@ -471,10 +471,7 @@ def update_sidebar(self): index = text.index("@0,0") if index.split('.', 1)[1] != '0': index = text.index(f'{index}+1line linestart') - while True: - lineinfo = text.dlineinfo(index) - if lineinfo is None: - break + while (lineinfo := text.dlineinfo(index)) is not None: y = lineinfo[1] prev_newline_tagnames = text_tagnames(f"{index} linestart -1c") prompt = ( From webhook-mailer at python.org Wed Feb 2 22:12:48 2022 From: webhook-mailer at python.org (terryjreedy) Date: Thu, 03 Feb 2022 03:12:48 -0000 Subject: [Python-checkins] [3.9] bpo-45975: Use walrus operator for some idlelib while loops (GH-31083) Message-ID: https://github.com/python/cpython/commit/fafd2dadf63973a04f5693e5be19f3e7521c10d4 commit: fafd2dadf63973a04f5693e5be19f3e7521c10d4 branch: 3.9 author: Terry Jan Reedy committer: terryjreedy date: 2022-02-02T22:12:38-05:00 summary: [3.9] bpo-45975: Use walrus operator for some idlelib while loops (GH-31083) co-authored by Nick Drozd cherrypicked from 51a95be1d035a717ab29e98056b8831a98e61125 files: M Lib/idlelib/pyparse.py M Lib/idlelib/replace.py M Lib/idlelib/run.py diff --git a/Lib/idlelib/pyparse.py b/Lib/idlelib/pyparse.py index d34872b4396e1..a94327533d865 100644 --- a/Lib/idlelib/pyparse.py +++ b/Lib/idlelib/pyparse.py @@ -179,14 +179,10 @@ def find_good_parse_start(self, is_char_in_string): # Peeking back worked; look forward until _synchre no longer # matches. i = pos + 1 - while 1: - m = _synchre(code, i) - if m: - s, i = m.span() - if not is_char_in_string(s): - pos = s - else: - break + while (m := _synchre(code, i)): + s, i = m.span() + if not is_char_in_string(s): + pos = s return pos def set_lo(self, lo): diff --git a/Lib/idlelib/replace.py b/Lib/idlelib/replace.py index 6be034af9626b..70d761db12630 100644 --- a/Lib/idlelib/replace.py +++ b/Lib/idlelib/replace.py @@ -156,11 +156,8 @@ def replace_all(self, event=None): first = last = None # XXX ought to replace circular instead of top-to-bottom when wrapping text.undo_block_start() - while True: - res = self.engine.search_forward(text, prog, line, col, - wrap=False, ok=ok) - if not res: - break + while (res := self.engine.search_forward( + text, prog, line, col, wrap=False, ok=ok)): line, m = res chars = text.get("%d.0" % line, "%d.0" % (line+1)) orig = m.group() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py index dda9711dcf7ae..4246f497cb382 100644 --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -468,9 +468,7 @@ def read(self, size=-1): result = self._line_buffer self._line_buffer = '' if size < 0: - while True: - line = self.shell.readline() - if not line: break + while (line := self.shell.readline()): result += line else: while len(result) < size: From webhook-mailer at python.org Wed Feb 2 23:03:08 2022 From: webhook-mailer at python.org (gpshead) Date: Thu, 03 Feb 2022 04:03:08 -0000 Subject: [Python-checkins] [3.10] bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (GH-31015) (GH-31089) Message-ID: https://github.com/python/cpython/commit/f5ebec4d3e1199ec38b88920cfde8e460e5120dd commit: f5ebec4d3e1199ec38b88920cfde8e460e5120dd branch: 3.10 author: Gregory P. Smith committer: gpshead date: 2022-02-02T20:02:59-08:00 summary: [3.10] bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (GH-31015) (GH-31089) Disable compiler optimization within test_peg_generator. This speed up test_peg_generator by always disabling compiler optimizations by using -O0 or equivalent when the test is building its own C extensions. A build not using --with-pydebug in order to speed up test execution winds up with this test taking a very long time as it would do repeated compilation of parser C code using the same optimization flags as CPython was built with. This speeds the test up 6-8x on gps-raspbian. Also incorporate's GH-31017's win32 conditional and flags. Co-authored-by: Kumar Aditya kumaraditya303. (cherry picked from commit 164a017e13ee96bd1ea1ae79f5ac9e25fe83994e) Co-authored-by: Gregory P. Smith files: A Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst M Tools/peg_generator/pegen/build.py diff --git a/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst b/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst new file mode 100644 index 0000000000000..be50fc8cbe0a5 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst @@ -0,0 +1,3 @@ +test_peg_generator now disables compiler optimization when testing +compilation of its own C extensions to significantly speed up the +testing on non-debug builds of CPython. diff --git a/Tools/peg_generator/pegen/build.py b/Tools/peg_generator/pegen/build.py index b80fc85423919..61a48d5714519 100644 --- a/Tools/peg_generator/pegen/build.py +++ b/Tools/peg_generator/pegen/build.py @@ -1,6 +1,7 @@ import pathlib import shutil import tokenize +import sys import sysconfig import tempfile import itertools @@ -33,6 +34,7 @@ def compile_c_extension( build_dir: Optional[str] = None, verbose: bool = False, keep_asserts: bool = True, + disable_optimization: bool = True, # Significant test_peg_generator speedup. ) -> str: """Compile the generated source for a parser generator into an extension module. @@ -62,6 +64,14 @@ def compile_c_extension( extra_link_args = get_extra_flags("LDFLAGS", "PY_LDFLAGS_NODIST") if keep_asserts: extra_compile_args.append("-UNDEBUG") + if disable_optimization: + if sys.platform == 'win32': + extra_compile_args.append("/Od") + extra_link_args.append("/LTCG:OFF") + else: + extra_compile_args.append("-O0") + if sysconfig.get_config_var("GNULD") == "yes": + extra_link_args.append("-fno-lto") extension = [ Extension( extension_name, From webhook-mailer at python.org Wed Feb 2 23:33:04 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 03 Feb 2022 04:33:04 -0000 Subject: [Python-checkins] [3.9] [3.10] bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (GH-31015) (GH-31089) (GH-31093) Message-ID: https://github.com/python/cpython/commit/e8258608c28c65680253d0ca6167430e34c2fd87 commit: e8258608c28c65680253d0ca6167430e34c2fd87 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-02T20:32:54-08:00 summary: [3.9] [3.10] bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (GH-31015) (GH-31089) (GH-31093) Disable compiler optimization within test_peg_generator. This speed up test_peg_generator by always disabling compiler optimizations by using -O0 or equivalent when the test is building its own C extensions. A build not using --with-pydebug in order to speed up test execution winds up with this test taking a very long time as it would do repeated compilation of parser C code using the same optimization flags as CPython was built with. This speeds the test up 6-8x on gps-raspbian. Also incorporate's GH-31017's win32 conditional and flags. Co-authored-by: Kumar Aditya kumaraditya303. (cherry picked from commit 164a017e13ee96bd1ea1ae79f5ac9e25fe83994e) Co-authored-by: Gregory P. Smith (cherry picked from commit f5ebec4d3e1199ec38b88920cfde8e460e5120dd) Co-authored-by: Gregory P. Smith Automerge-Triggered-By: GH:gpshead files: A Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst M Tools/peg_generator/pegen/build.py diff --git a/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst b/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst new file mode 100644 index 0000000000000..be50fc8cbe0a5 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst @@ -0,0 +1,3 @@ +test_peg_generator now disables compiler optimization when testing +compilation of its own C extensions to significantly speed up the +testing on non-debug builds of CPython. diff --git a/Tools/peg_generator/pegen/build.py b/Tools/peg_generator/pegen/build.py index 931ffc787523b..109fb9ecc7adc 100644 --- a/Tools/peg_generator/pegen/build.py +++ b/Tools/peg_generator/pegen/build.py @@ -1,6 +1,7 @@ import pathlib import shutil import tokenize +import sys import sysconfig import tempfile import itertools @@ -33,6 +34,7 @@ def compile_c_extension( build_dir: Optional[str] = None, verbose: bool = False, keep_asserts: bool = True, + disable_optimization: bool = True, # Significant test_peg_generator speedup. ) -> str: """Compile the generated source for a parser generator into an extension module. @@ -59,6 +61,14 @@ def compile_c_extension( extra_link_args = get_extra_flags("LDFLAGS", "PY_LDFLAGS_NODIST") if keep_asserts: extra_compile_args.append("-UNDEBUG") + if disable_optimization: + if sys.platform == 'win32': + extra_compile_args.append("/Od") + extra_link_args.append("/LTCG:OFF") + else: + extra_compile_args.append("-O0") + if sysconfig.get_config_var("GNULD") == "yes": + extra_link_args.append("-fno-lto") extension = [ Extension( extension_name, From webhook-mailer at python.org Thu Feb 3 00:36:26 2022 From: webhook-mailer at python.org (gpshead) Date: Thu, 03 Feb 2022 05:36:26 -0000 Subject: [Python-checkins] bpo-45629: Improve test.support.skip_if_buildbot (GH-31094) Message-ID: https://github.com/python/cpython/commit/8726067ace98a27557e9fdf1a8e1c509c37cfcfc commit: 8726067ace98a27557e9fdf1a8e1c509c37cfcfc branch: main author: Gregory P. Smith committer: gpshead date: 2022-02-02T21:36:16-08:00 summary: bpo-45629: Improve test.support.skip_if_buildbot (GH-31094) It was added as part of #29222 to avoid running freeze tool tests on the buildbots but the logic was wrong so it did not skip tests on typical posix setup buildbots where the worker is launched from cron via an @reboot task and thus have no USER environment variable. This uses the canonical `getpass.getuser()` API rather than rolling its own attempt. files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index d71cfe5ee44fd..e5eb66e9068e7 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -5,6 +5,7 @@ import contextlib import functools +import getpass import os import re import stat @@ -378,10 +379,11 @@ def skip_if_buildbot(reason=None): """Decorator raising SkipTest if running on a buildbot.""" if not reason: reason = 'not suitable for buildbots' - if sys.platform == 'win32': - isbuildbot = os.environ.get('USERNAME') == 'Buildbot' - else: - isbuildbot = os.environ.get('USER') == 'buildbot' + try: + isbuildbot = getpass.getuser().lower() == 'buildbot' + except (KeyError, EnvironmentError) as err: + warnings.warn(f'getpass.getuser() failed {err}.', RuntimeWarning) + isbuildbot = False return unittest.skipIf(isbuildbot, reason) def check_sanitizer(*, address=False, memory=False, ub=False): From webhook-mailer at python.org Thu Feb 3 03:12:18 2022 From: webhook-mailer at python.org (rhettinger) Date: Thu, 03 Feb 2022 08:12:18 -0000 Subject: [Python-checkins] Add recipe for subslices (GH-31095) Message-ID: https://github.com/python/cpython/commit/a77de58108a89ada49a3af7613e84df436fd147c commit: a77de58108a89ada49a3af7613e84df436fd147c branch: 3.10 author: Raymond Hettinger committer: rhettinger date: 2022-02-03T02:12:08-06:00 summary: Add recipe for subslices (GH-31095) files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 34667561c3cfe..6e1ba3c440124 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -893,6 +893,12 @@ which incur interpreter overhead. yield from it return true_iterator(), remainder_iterator() + def subslices(seq): + "Return all contiguous non-empty subslices of a sequence" + # subslices('ABCD') --> A AB ABC ABCD B BC BCD C CD D + slices = starmap(slice, combinations(range(len(seq) + 1), 2)) + return map(operator.getitem, repeat(seq), slices) + def powerset(iterable): "powerset([1,2,3]) --> () (1,) (2,) (3,) (1,2) (1,3) (2,3) (1,2,3)" s = list(iterable) From webhook-mailer at python.org Thu Feb 3 03:41:47 2022 From: webhook-mailer at python.org (gpshead) Date: Thu, 03 Feb 2022 08:41:47 -0000 Subject: [Python-checkins] Restrict use of Mock objects as specs (GH-31090) Message-ID: https://github.com/python/cpython/commit/6394e981adaca2c0daa36c8701611e250d74024c commit: 6394e981adaca2c0daa36c8701611e250d74024c branch: main author: Matthew Suozzo committer: gpshead date: 2022-02-03T00:41:19-08:00 summary: Restrict use of Mock objects as specs (GH-31090) Follow-on to https://github.com/python/cpython/pull/25326 This covers cases where mock objects are passed directly to spec. files: A Misc/NEWS.d/next/Tests/2022-02-03-00-21-32.bpo-43478.0nfcam.rst M Lib/unittest/mock.py M Lib/unittest/test/testmock/testmock.py M Lib/unittest/test/testmock/testwith.py diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py index 9137501930000..2719f74d6fca5 100644 --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -489,6 +489,9 @@ def mock_add_spec(self, spec, spec_set=False): def _mock_add_spec(self, spec, spec_set, _spec_as_instance=False, _eat_self=False): + if _is_instance_mock(spec): + raise InvalidSpecError(f'Cannot spec a Mock object. [object={spec!r}]') + _spec_class = None _spec_signature = None _spec_asyncs = [] @@ -2789,6 +2792,7 @@ def __init__(self, spec, spec_set=False, parent=None, file_spec = None +open_spec = None def _to_stream(read_data): @@ -2845,8 +2849,12 @@ def _next_side_effect(): import _io file_spec = list(set(dir(_io.TextIOWrapper)).union(set(dir(_io.BytesIO)))) + global open_spec + if open_spec is None: + import _io + open_spec = list(set(dir(_io.open))) if mock is None: - mock = MagicMock(name='open', spec=open) + mock = MagicMock(name='open', spec=open_spec) handle = MagicMock(spec=file_spec) handle.__enter__.return_value = handle diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py index fdba543b53511..c99098dc4ea86 100644 --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -226,6 +226,14 @@ class B(object): with self.assertRaisesRegex(InvalidSpecError, "Cannot spec attr 'B' as the spec_set "): mock.patch.object(A, 'B', spec_set=A.B).start() + with self.assertRaisesRegex(InvalidSpecError, + "Cannot spec attr 'B' as the spec_set "): + mock.patch.object(A, 'B', spec_set=A.B).start() + with self.assertRaisesRegex(InvalidSpecError, "Cannot spec a Mock object."): + mock.Mock(A.B) + with mock.patch('builtins.open', mock.mock_open()): + mock.mock_open() # should still be valid with open() mocked + def test_reset_mock(self): parent = Mock() diff --git a/Lib/unittest/test/testmock/testwith.py b/Lib/unittest/test/testmock/testwith.py index 42ebf3898c89e..c74d49a63c892 100644 --- a/Lib/unittest/test/testmock/testwith.py +++ b/Lib/unittest/test/testmock/testwith.py @@ -130,8 +130,8 @@ def f(self): pass c = C() - with patch.object(c, 'f', autospec=True) as patch1: - with patch.object(c, 'f', autospec=True) as patch2: + with patch.object(c, 'f') as patch1: + with patch.object(c, 'f') as patch2: c.f() self.assertEqual(patch2.call_count, 1) self.assertEqual(patch1.call_count, 0) diff --git a/Misc/NEWS.d/next/Tests/2022-02-03-00-21-32.bpo-43478.0nfcam.rst b/Misc/NEWS.d/next/Tests/2022-02-03-00-21-32.bpo-43478.0nfcam.rst new file mode 100644 index 0000000000000..7c8fc47cfc751 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-03-00-21-32.bpo-43478.0nfcam.rst @@ -0,0 +1 @@ +Mocks can no longer be provided as the specs for other Mocks. As a result, an already-mocked object cannot be passed to `mock.Mock()`. This can uncover bugs in tests since these Mock-derived Mocks will always pass certain tests (e.g. isinstance) and builtin assert functions (e.g. assert_called_once_with) will unconditionally pass. From webhook-mailer at python.org Thu Feb 3 04:20:23 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 03 Feb 2022 09:20:23 -0000 Subject: [Python-checkins] bpo-46565: `del` loop vars that are leaking into module namespaces (GH-30993) Message-ID: https://github.com/python/cpython/commit/0cbdd2131195b0d313762968f604e80a3e65ca9f commit: 0cbdd2131195b0d313762968f604e80a3e65ca9f branch: main author: Nikita Sobolev committer: serhiy-storchaka date: 2022-02-03T11:20:08+02:00 summary: bpo-46565: `del` loop vars that are leaking into module namespaces (GH-30993) files: A Misc/NEWS.d/next/Library/2022-01-28-19-48-31.bpo-46565.bpZXO4.rst M Lib/_compat_pickle.py M Lib/email/contentmanager.py M Lib/email/quoprimime.py M Lib/http/cookiejar.py M Lib/inspect.py M Lib/json/encoder.py M Lib/lib2to3/pgen2/grammar.py M Lib/locale.py M Lib/multiprocessing/managers.py M Lib/multiprocessing/process.py M Lib/sysconfig.py M Lib/test/test_inspect.py M Lib/tokenize.py diff --git a/Lib/_compat_pickle.py b/Lib/_compat_pickle.py index f68496ae639f5..65a94b6b1bdfd 100644 --- a/Lib/_compat_pickle.py +++ b/Lib/_compat_pickle.py @@ -249,3 +249,4 @@ for excname in PYTHON3_IMPORTERROR_EXCEPTIONS: REVERSE_NAME_MAPPING[('builtins', excname)] = ('exceptions', 'ImportError') +del excname diff --git a/Lib/email/contentmanager.py b/Lib/email/contentmanager.py index fcf278dbccbac..b4f5830beada4 100644 --- a/Lib/email/contentmanager.py +++ b/Lib/email/contentmanager.py @@ -72,12 +72,14 @@ def get_non_text_content(msg): return msg.get_payload(decode=True) for maintype in 'audio image video application'.split(): raw_data_manager.add_get_handler(maintype, get_non_text_content) +del maintype def get_message_content(msg): return msg.get_payload(0) for subtype in 'rfc822 external-body'.split(): raw_data_manager.add_get_handler('message/'+subtype, get_message_content) +del subtype def get_and_fixup_unknown_message_content(msg): @@ -246,3 +248,4 @@ def set_bytes_content(msg, data, maintype, subtype, cte='base64', _finalize_set(msg, disposition, filename, cid, params) for typ in (bytes, bytearray, memoryview): raw_data_manager.add_set_handler(typ, set_bytes_content) +del typ diff --git a/Lib/email/quoprimime.py b/Lib/email/quoprimime.py index 94534f7ee1e33..27fcbb5a26e3a 100644 --- a/Lib/email/quoprimime.py +++ b/Lib/email/quoprimime.py @@ -148,6 +148,7 @@ def header_encode(header_bytes, charset='iso-8859-1'): _QUOPRI_BODY_ENCODE_MAP = _QUOPRI_BODY_MAP[:] for c in b'\r\n': _QUOPRI_BODY_ENCODE_MAP[c] = chr(c) +del c def body_encode(body, maxlinelen=76, eol=NL): """Encode with quoted-printable, wrapping at maxlinelen characters. diff --git a/Lib/http/cookiejar.py b/Lib/http/cookiejar.py index eaa76c26b9c59..ee433c0f78f4a 100644 --- a/Lib/http/cookiejar.py +++ b/Lib/http/cookiejar.py @@ -89,8 +89,7 @@ def _timegm(tt): DAYS = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"] MONTHS = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"] -MONTHS_LOWER = [] -for month in MONTHS: MONTHS_LOWER.append(month.lower()) +MONTHS_LOWER = [month.lower() for month in MONTHS] def time2isoz(t=None): """Return a string representing time in seconds since epoch, t. diff --git a/Lib/inspect.py b/Lib/inspect.py index d47f5b717471c..eb45f81aa2d95 100644 --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -156,6 +156,7 @@ mod_dict = globals() for k, v in dis.COMPILER_FLAG_NAMES.items(): mod_dict["CO_" + v] = k +del k, v, mod_dict # See Include/object.h TPFLAGS_IS_ABSTRACT = 1 << 20 diff --git a/Lib/json/encoder.py b/Lib/json/encoder.py index 21bff2c1a1fca..864f46d9dbb72 100644 --- a/Lib/json/encoder.py +++ b/Lib/json/encoder.py @@ -30,6 +30,7 @@ for i in range(0x20): ESCAPE_DCT.setdefault(chr(i), '\\u{0:04x}'.format(i)) #ESCAPE_DCT.setdefault(chr(i), '\\u%04x' % (i,)) +del i INFINITY = float('inf') diff --git a/Lib/lib2to3/pgen2/grammar.py b/Lib/lib2to3/pgen2/grammar.py index 6a4d575ac2ccf..5d550aeb65e8d 100644 --- a/Lib/lib2to3/pgen2/grammar.py +++ b/Lib/lib2to3/pgen2/grammar.py @@ -186,3 +186,4 @@ def report(self): if line: op, name = line.split() opmap[op] = getattr(token, name) +del line, op, name diff --git a/Lib/locale.py b/Lib/locale.py index 6d4f51929923f..4bd31c9fa2cdf 100644 --- a/Lib/locale.py +++ b/Lib/locale.py @@ -746,6 +746,7 @@ def getpreferredencoding(do_setlocale=True): for k, v in sorted(locale_encoding_alias.items()): k = k.replace('_', '') locale_encoding_alias.setdefault(k, v) +del k, v # # The locale_alias table maps lowercase alias names to C locale names diff --git a/Lib/multiprocessing/managers.py b/Lib/multiprocessing/managers.py index cf637c6cbbe8d..d97381926d47b 100644 --- a/Lib/multiprocessing/managers.py +++ b/Lib/multiprocessing/managers.py @@ -49,11 +49,11 @@ def reduce_array(a): reduction.register(array.array, reduce_array) view_types = [type(getattr({}, name)()) for name in ('items','keys','values')] -if view_types[0] is not list: # only needed in Py3.0 - def rebuild_as_list(obj): - return list, (list(obj),) - for view_type in view_types: - reduction.register(view_type, rebuild_as_list) +def rebuild_as_list(obj): + return list, (list(obj),) +for view_type in view_types: + reduction.register(view_type, rebuild_as_list) +del view_type, view_types # # Type for identifying shared objects diff --git a/Lib/multiprocessing/process.py b/Lib/multiprocessing/process.py index 0b2e0b45b2397..3917d2e4fa63e 100644 --- a/Lib/multiprocessing/process.py +++ b/Lib/multiprocessing/process.py @@ -427,6 +427,7 @@ def close(self): for name, signum in list(signal.__dict__.items()): if name[:3]=='SIG' and '_' not in name: _exitcode_to_name[-signum] = f'-{name}' +del name, signum # For debug and leak testing _dangling = WeakSet() diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py index ef335c69421f5..d4a8a680286c1 100644 --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -192,6 +192,7 @@ def is_python_build(check_home=False): scheme['headers'] = scheme['include'] scheme['include'] = '{srcdir}/Include' scheme['platinclude'] = '{projectbase}/.' + del scheme def _subst_vars(s, local_vars): diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py index 76fa6f7e2dab8..a553431bdccfb 100644 --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -108,7 +108,7 @@ def istest(self, predicate, exp): self.assertFalse(other(obj), 'not %s(%s)' % (other.__name__, exp)) def test__all__(self): - support.check__all__(self, inspect, not_exported=("k", "v", "mod_dict", "modulesbyfile")) + support.check__all__(self, inspect, not_exported=("modulesbyfile",)) def generator_function_example(self): for i in range(2): diff --git a/Lib/tokenize.py b/Lib/tokenize.py index 0b9e238310049..46d2224f5cc08 100644 --- a/Lib/tokenize.py +++ b/Lib/tokenize.py @@ -143,6 +143,7 @@ def _compile(expr): endpats[_prefix + '"'] = Double endpats[_prefix + "'''"] = Single3 endpats[_prefix + '"""'] = Double3 +del _prefix # A set of all of the single and triple quoted string prefixes, # including the opening quotes. @@ -153,6 +154,7 @@ def _compile(expr): single_quoted.add(u) for u in (t + '"""', t + "'''"): triple_quoted.add(u) +del t, u tabsize = 8 diff --git a/Misc/NEWS.d/next/Library/2022-01-28-19-48-31.bpo-46565.bpZXO4.rst b/Misc/NEWS.d/next/Library/2022-01-28-19-48-31.bpo-46565.bpZXO4.rst new file mode 100644 index 0000000000000..9b0969ea5897b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-28-19-48-31.bpo-46565.bpZXO4.rst @@ -0,0 +1 @@ +Remove loop variables that are leaking into modules' namespaces. From webhook-mailer at python.org Thu Feb 3 04:25:27 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 03 Feb 2022 09:25:27 -0000 Subject: [Python-checkins] bpo-46483: Remove `__class_getitem__` from `pathlib.PurePath` (GH-30848) Message-ID: https://github.com/python/cpython/commit/7ffe7ba30fc051014977c6f393c51e57e71a6648 commit: 7ffe7ba30fc051014977c6f393c51e57e71a6648 branch: main author: Nikita Sobolev committer: serhiy-storchaka date: 2022-02-03T11:25:10+02:00 summary: bpo-46483: Remove `__class_getitem__` from `pathlib.PurePath` (GH-30848) Co-authored-by: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-01-24-13-00-09.bpo-46483.9XnmKp.rst D Misc/NEWS.d/next/Library/2022-01-23-11-17-48.bpo-46483.j7qwWb.rst M Doc/whatsnew/3.11.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 33f39e5775269..acb21d3ccaf17 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -545,6 +545,9 @@ Removed Python 3.4 but has been broken since Python 3.7. (Contributed by Inada Naoki in :issue:`23882`.) +* Remove ``__class_getitem__`` method from :class:`pathlib.PurePath`, + because it was not used and added by mistake in previous versions. + (Contributed by Nikita Sobolev in :issue:`46483`.) Porting to Python 3.11 ====================== diff --git a/Lib/pathlib.py b/Lib/pathlib.py index 920e1f425a0c0..7f4210e2b80c9 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -12,7 +12,6 @@ from operator import attrgetter from stat import S_ISDIR, S_ISLNK, S_ISREG, S_ISSOCK, S_ISBLK, S_ISCHR, S_ISFIFO from urllib.parse import quote_from_bytes as urlquote_from_bytes -from types import GenericAlias __all__ = [ @@ -604,8 +603,6 @@ def __ge__(self, other): return NotImplemented return self._cparts >= other._cparts - __class_getitem__ = classmethod(GenericAlias) - drive = property(attrgetter('_drv'), doc="""The drive prefix (letter or UNC path), if any.""") diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index 5e46b4ffeae10..ec2baca18fd81 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -2450,15 +2450,6 @@ def test_complex_symlinks_relative(self): def test_complex_symlinks_relative_dot_dot(self): self._check_complex_symlinks(os.path.join('dirA', '..')) - def test_class_getitem(self): - from types import GenericAlias - - alias = self.cls[str] - self.assertIsInstance(alias, GenericAlias) - self.assertIs(alias.__origin__, self.cls) - self.assertEqual(alias.__args__, (str,)) - self.assertEqual(alias.__parameters__, ()) - class PathTest(_BasePathTest, unittest.TestCase): cls = pathlib.Path diff --git a/Misc/NEWS.d/next/Library/2022-01-23-11-17-48.bpo-46483.j7qwWb.rst b/Misc/NEWS.d/next/Library/2022-01-23-11-17-48.bpo-46483.j7qwWb.rst deleted file mode 100644 index a84503d299a10..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-23-11-17-48.bpo-46483.j7qwWb.rst +++ /dev/null @@ -1,2 +0,0 @@ -Change :meth:`pathlib.PurePath.__class_getitem__` to return -:class:`types.GenericAlias`. diff --git a/Misc/NEWS.d/next/Library/2022-01-24-13-00-09.bpo-46483.9XnmKp.rst b/Misc/NEWS.d/next/Library/2022-01-24-13-00-09.bpo-46483.9XnmKp.rst new file mode 100644 index 0000000000000..89cc818a9c59d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-24-13-00-09.bpo-46483.9XnmKp.rst @@ -0,0 +1 @@ +Remove :meth:`~object.__class_getitem__` from :class:`pathlib.PurePath` as this class was not supposed to be generic. From webhook-mailer at python.org Thu Feb 3 04:43:46 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Thu, 03 Feb 2022 09:43:46 -0000 Subject: [Python-checkins] bpo-44977: Deprecate delegation of int to __trunc__ (GH-31031) Message-ID: https://github.com/python/cpython/commit/b4bd1e1422997de61faf506b4916e83013bc7d21 commit: b4bd1e1422997de61faf506b4916e83013bc7d21 branch: main author: Zackery Spytz committer: serhiy-storchaka date: 2022-02-03T11:43:25+02:00 summary: bpo-44977: Deprecate delegation of int to __trunc__ (GH-31031) Calling int(a) when type(a) implements __trunc__ but not __int__ or __index__ now raises a DeprecationWarning. files: A Misc/NEWS.d/next/Core and Builtins/2022-01-30-18-23-08.bpo-44977.BQV_zS.rst M Doc/library/functions.rst M Doc/reference/datamodel.rst M Doc/whatsnew/3.11.rst M Lib/test/test_descr.py M Lib/test/test_int.py M Lib/test/test_long.py M Objects/abstract.c diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 9c061bcd8252a..eaa4d482ce3fc 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -891,6 +891,9 @@ are always available. They are listed here in alphabetical order. .. versionchanged:: 3.8 Falls back to :meth:`__index__` if :meth:`__int__` is not defined. + .. versionchanged:: 3.11 + The delegation to :meth:`__trunc__` is deprecated. + .. function:: isinstance(object, classinfo) diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 48c54d729424c..a773b73ce3243 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2760,6 +2760,9 @@ left undefined. The built-in function :func:`int` falls back to :meth:`__trunc__` if neither :meth:`__int__` nor :meth:`__index__` is defined. + .. versionchanged:: 3.11 + The delegation of :func:`int` to :meth:`__trunc__` is deprecated. + .. _context-managers: diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index acb21d3ccaf17..18d4652fb4293 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -458,6 +458,11 @@ Deprecated as deprecated, its docstring is now corrected). (Contributed by Hugo van Kemenade in :issue:`45837`.) +* The delegation of :func:`int` to :meth:`__trunc__` is now deprecated. Calling + ``int(a)`` when ``type(a)`` implements :meth:`__trunc__` but not + :meth:`__int__` or :meth:`__index__` now raises a :exc:`DeprecationWarning`. + (Contributed by Zackery Spytz in :issue:`44977`.) + * The following have been deprecated in :mod:`configparser` since Python 3.2. Their deprecation warnings have now been updated to note they will removed in Python 3.12: @@ -468,6 +473,7 @@ Deprecated (Contributed by Hugo van Kemenade in :issue:`45173`.) + Removed ======= diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py index 791cf714d46a3..a0942d6a85b81 100644 --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -2061,7 +2061,6 @@ def format_impl(self, spec): ("__format__", format, format_impl, set(), {}), ("__floor__", math.floor, zero, set(), {}), ("__trunc__", math.trunc, zero, set(), {}), - ("__trunc__", int, zero, set(), {}), ("__ceil__", math.ceil, zero, set(), {}), ("__dir__", dir, empty_seq, set(), {}), ("__round__", round, zero, set(), {}), diff --git a/Lib/test/test_int.py b/Lib/test/test_int.py index d6be64e7c18a0..a72699cc7506a 100644 --- a/Lib/test/test_int.py +++ b/Lib/test/test_int.py @@ -369,12 +369,14 @@ def __trunc__(self): class JustTrunc(base): def __trunc__(self): return 42 - self.assertEqual(int(JustTrunc()), 42) + with self.assertWarns(DeprecationWarning): + self.assertEqual(int(JustTrunc()), 42) class ExceptionalTrunc(base): def __trunc__(self): 1 / 0 - with self.assertRaises(ZeroDivisionError): + with self.assertRaises(ZeroDivisionError), \ + self.assertWarns(DeprecationWarning): int(ExceptionalTrunc()) for trunc_result_base in (object, Classic): @@ -385,7 +387,8 @@ def __index__(self): class TruncReturnsNonInt(base): def __trunc__(self): return Index() - self.assertEqual(int(TruncReturnsNonInt()), 42) + with self.assertWarns(DeprecationWarning): + self.assertEqual(int(TruncReturnsNonInt()), 42) class Intable(trunc_result_base): def __int__(self): @@ -394,7 +397,8 @@ def __int__(self): class TruncReturnsNonIndex(base): def __trunc__(self): return Intable() - self.assertEqual(int(TruncReturnsNonInt()), 42) + with self.assertWarns(DeprecationWarning): + self.assertEqual(int(TruncReturnsNonInt()), 42) class NonIntegral(trunc_result_base): def __trunc__(self): @@ -405,7 +409,8 @@ class TruncReturnsNonIntegral(base): def __trunc__(self): return NonIntegral() try: - int(TruncReturnsNonIntegral()) + with self.assertWarns(DeprecationWarning): + int(TruncReturnsNonIntegral()) except TypeError as e: self.assertEqual(str(e), "__trunc__ returned non-Integral" @@ -423,7 +428,8 @@ class TruncReturnsBadInt(base): def __trunc__(self): return BadInt() - with self.assertRaises(TypeError): + with self.assertRaises(TypeError), \ + self.assertWarns(DeprecationWarning): int(TruncReturnsBadInt()) def test_int_subclass_with_index(self): @@ -517,13 +523,16 @@ def __trunc__(self): self.assertIs(type(n), int) bad_int = TruncReturnsBadInt() - self.assertRaises(TypeError, int, bad_int) + with self.assertWarns(DeprecationWarning): + self.assertRaises(TypeError, int, bad_int) good_int = TruncReturnsIntSubclass() - n = int(good_int) + with self.assertWarns(DeprecationWarning): + n = int(good_int) self.assertEqual(n, 1) self.assertIs(type(n), int) - n = IntSubclass(good_int) + with self.assertWarns(DeprecationWarning): + n = IntSubclass(good_int) self.assertEqual(n, 1) self.assertIs(type(n), IntSubclass) diff --git a/Lib/test/test_long.py b/Lib/test/test_long.py index c7dd0b274d1e3..e68dfb4c542ea 100644 --- a/Lib/test/test_long.py +++ b/Lib/test/test_long.py @@ -392,7 +392,8 @@ def __long__(self): return 42 def __trunc__(self): return 1729 - self.assertEqual(int(LongTrunc()), 1729) + with self.assertWarns(DeprecationWarning): + self.assertEqual(int(LongTrunc()), 1729) def check_float_conversion(self, n): # Check that int -> float conversion behaviour matches diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-30-18-23-08.bpo-44977.BQV_zS.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-30-18-23-08.bpo-44977.BQV_zS.rst new file mode 100644 index 0000000000000..84c1191bdbd31 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-01-30-18-23-08.bpo-44977.BQV_zS.rst @@ -0,0 +1,3 @@ +The delegation of :func:`int` to :meth:`__trunc__` is now deprecated. +Calling ``int(a)`` when ``type(a)`` implements :meth:`__trunc__` but not +:meth:`__int__` or :meth:`__index__` now raises a :exc:`DeprecationWarning`. diff --git a/Objects/abstract.c b/Objects/abstract.c index 6a2d5eda14079..ed99fd6c5aafb 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -1564,6 +1564,11 @@ PyNumber_Long(PyObject *o) } trunc_func = _PyObject_LookupSpecial(o, &PyId___trunc__); if (trunc_func) { + if (PyErr_WarnEx(PyExc_DeprecationWarning, + "The delegation of int() to __trunc__ is deprecated.", 1)) { + Py_DECREF(trunc_func); + return NULL; + } result = _PyObject_CallNoArgs(trunc_func); Py_DECREF(trunc_func); if (result == NULL || PyLong_CheckExact(result)) { From webhook-mailer at python.org Thu Feb 3 05:14:48 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 03 Feb 2022 10:14:48 -0000 Subject: [Python-checkins] bpo-45773: Remove invalid peephole optimizations (GH-31066) Message-ID: https://github.com/python/cpython/commit/e0433c1e70254d4d0357a9e14596929a04bdf769 commit: e0433c1e70254d4d0357a9e14596929a04bdf769 branch: main author: Brandt Bucher committer: markshannon date: 2022-02-03T10:14:44Z summary: bpo-45773: Remove invalid peephole optimizations (GH-31066) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst M Python/compile.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst new file mode 100644 index 0000000000000..45da5116fc940 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst @@ -0,0 +1 @@ +Remove two invalid "peephole" optimizations from the bytecode compiler. diff --git a/Python/compile.c b/Python/compile.c index 1d2d567e85e93..cecd6a5c42660 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8757,7 +8757,6 @@ optimize_basic_block(struct compiler *c, basicblock *bb, PyObject *consts) switch (target->i_opcode) { case JUMP_ABSOLUTE: case JUMP_FORWARD: - case JUMP_IF_FALSE_OR_POP: i -= jump_thread(inst, target, POP_JUMP_IF_FALSE); } break; @@ -8765,7 +8764,6 @@ optimize_basic_block(struct compiler *c, basicblock *bb, PyObject *consts) switch (target->i_opcode) { case JUMP_ABSOLUTE: case JUMP_FORWARD: - case JUMP_IF_TRUE_OR_POP: i -= jump_thread(inst, target, POP_JUMP_IF_TRUE); } break; From webhook-mailer at python.org Thu Feb 3 08:33:05 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 03 Feb 2022 13:33:05 -0000 Subject: [Python-checkins] bpo-45885: Add more stats for COMPARE_OP in specialize.c (GH-31040) Message-ID: https://github.com/python/cpython/commit/674ab66ebdf06f187e193a3d7bde13b71ba0f9af commit: 674ab66ebdf06f187e193a3d7bde13b71ba0f9af branch: main author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: markshannon date: 2022-02-03T13:32:52Z summary: bpo-45885: Add more stats for COMPARE_OP in specialize.c (GH-31040) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-01-01-17-28.bpo-45885.CjyNf_.rst M Python/specialize.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-01-17-28.bpo-45885.CjyNf_.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-01-17-28.bpo-45885.CjyNf_.rst new file mode 100644 index 0000000000000..6395bd1f18d5e --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-01-17-28.bpo-45885.CjyNf_.rst @@ -0,0 +1 @@ +Added more fined-grained specialization failure stats regarding the ``COMPARE_OP`` bytecode. \ No newline at end of file diff --git a/Python/specialize.c b/Python/specialize.c index d90d7da8f3072..214f29751f388 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -547,6 +547,14 @@ initial_counter_value(void) { #define SPEC_FAIL_STRING_COMPARE 13 #define SPEC_FAIL_NOT_FOLLOWED_BY_COND_JUMP 14 #define SPEC_FAIL_BIG_INT 15 +#define SPEC_FAIL_COMPARE_BYTES 16 +#define SPEC_FAIL_COMPARE_TUPLE 17 +#define SPEC_FAIL_COMPARE_LIST 18 +#define SPEC_FAIL_COMPARE_SET 19 +#define SPEC_FAIL_COMPARE_BOOL 20 +#define SPEC_FAIL_COMPARE_BASEOBJECT 21 +#define SPEC_FAIL_COMPARE_FLOAT_LONG 22 +#define SPEC_FAIL_COMPARE_LONG_FLOAT 23 /* FOR_ITER */ #define SPEC_FAIL_ITER_GENERATOR 10 @@ -1764,6 +1772,43 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, adaptive->counter = initial_counter_value(); } + +#ifdef Py_STATS +static int +compare_op_fail_kind(PyObject *lhs, PyObject *rhs) +{ + if (Py_TYPE(lhs) != Py_TYPE(rhs)) { + if (PyFloat_CheckExact(lhs) && PyLong_CheckExact(rhs)) { + return SPEC_FAIL_COMPARE_FLOAT_LONG; + } + if (PyLong_CheckExact(lhs) && PyFloat_CheckExact(rhs)) { + return SPEC_FAIL_COMPARE_LONG_FLOAT; + } + return SPEC_FAIL_DIFFERENT_TYPES; + } + if (PyBytes_CheckExact(lhs)) { + return SPEC_FAIL_COMPARE_BYTES; + } + if (PyTuple_CheckExact(lhs)) { + return SPEC_FAIL_COMPARE_TUPLE; + } + if (PyList_CheckExact(lhs)) { + return SPEC_FAIL_COMPARE_LIST; + } + if (PySet_CheckExact(lhs) || PyFrozenSet_CheckExact(lhs)) { + return SPEC_FAIL_COMPARE_SET; + } + if (PyBool_Check(lhs)) { + return SPEC_FAIL_COMPARE_BOOL; + } + if (Py_TYPE(lhs)->tp_richcompare == PyBaseObject_Type.tp_richcompare) { + return SPEC_FAIL_COMPARE_BASEOBJECT; + } + return SPEC_FAIL_OTHER; +} +#endif + + static int compare_masks[] = { // 1-bit: jump if less than // 2-bit: jump if equal @@ -1795,7 +1840,7 @@ _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, when_to_jump_mask = (1 | 2 | 4) & ~when_to_jump_mask; } if (Py_TYPE(lhs) != Py_TYPE(rhs)) { - SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_DIFFERENT_TYPES); + SPECIALIZATION_FAIL(COMPARE_OP, compare_op_fail_kind(lhs, rhs)); goto failure; } if (PyFloat_CheckExact(lhs)) { @@ -1825,7 +1870,7 @@ _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, goto success; } } - SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_OTHER); + SPECIALIZATION_FAIL(COMPARE_OP, compare_op_fail_kind(lhs, rhs)); failure: STAT_INC(COMPARE_OP, failure); cache_backoff(adaptive); From webhook-mailer at python.org Thu Feb 3 08:48:22 2022 From: webhook-mailer at python.org (rhettinger) Date: Thu, 03 Feb 2022 13:48:22 -0000 Subject: [Python-checkins] bpo-46624: Defer to 3.12: "Remove deprecated support for non-integer values" (GH-31098) Message-ID: https://github.com/python/cpython/commit/6baa98e538b2e26f16eaaf462f99496e98d2cfb1 commit: 6baa98e538b2e26f16eaaf462f99496e98d2cfb1 branch: main author: Miro Hron?ok committer: rhettinger date: 2022-02-03T07:48:13-06:00 summary: bpo-46624: Defer to 3.12: "Remove deprecated support for non-integer values" (GH-31098) files: A Misc/NEWS.d/next/Library/2022-02-03-12-07-41.bpo-46624.f_Qqh0.rst M Doc/library/random.rst M Lib/random.py M Lib/test/test_random.py diff --git a/Doc/library/random.rst b/Doc/library/random.rst index 36f232dc319e4..da4a4f61e4567 100644 --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -135,10 +135,15 @@ Functions for integers values. Formerly it used a style like ``int(random()*n)`` which could produce slightly uneven distributions. - .. versionchanged:: 3.11 - Automatic conversion of non-integer types is no longer supported. - Calls such as ``randrange(10.0)`` and ``randrange(Fraction(10, 1))`` - now raise a :exc:`TypeError`. + .. deprecated:: 3.10 + The automatic conversion of non-integer types to equivalent integers is + deprecated. Currently ``randrange(10.0)`` is losslessly converted to + ``randrange(10)``. In the future, this will raise a :exc:`TypeError`. + + .. deprecated:: 3.10 + The exception raised for non-integral values such as ``randrange(10.5)`` + or ``randrange('10')`` will be changed from :exc:`ValueError` to + :exc:`TypeError`. .. function:: randint(a, b) diff --git a/Lib/random.py b/Lib/random.py index e8bc9416fcd94..6d7b617e33a30 100644 --- a/Lib/random.py +++ b/Lib/random.py @@ -282,17 +282,27 @@ def randbytes(self, n): ## -------------------- integer methods ------------------- def randrange(self, start, stop=None, step=_ONE): - """Choose a random item from range(stop) or range(start, stop[, step]). + """Choose a random item from range(start, stop[, step]). - Roughly equivalent to ``choice(range(start, stop, step))`` - but supports arbitrarily large ranges and is optimized - for common cases. + This fixes the problem with randint() which includes the + endpoint; in Python this is usually not what you want. """ # This code is a bit messy to make it fast for the # common case while still doing adequate error checking. - istart = _index(start) + try: + istart = _index(start) + except TypeError: + istart = int(start) + if istart != start: + _warn('randrange() will raise TypeError in the future', + DeprecationWarning, 2) + raise ValueError("non-integer arg 1 for randrange()") + _warn('non-integer arguments to randrange() have been deprecated ' + 'since Python 3.10 and will be removed in a subsequent ' + 'version', + DeprecationWarning, 2) if stop is None: # We don't check for "step != 1" because it hasn't been # type checked and converted to an integer yet. @@ -302,15 +312,37 @@ def randrange(self, start, stop=None, step=_ONE): return self._randbelow(istart) raise ValueError("empty range for randrange()") - # Stop argument supplied. - istop = _index(stop) + # stop argument supplied. + try: + istop = _index(stop) + except TypeError: + istop = int(stop) + if istop != stop: + _warn('randrange() will raise TypeError in the future', + DeprecationWarning, 2) + raise ValueError("non-integer stop for randrange()") + _warn('non-integer arguments to randrange() have been deprecated ' + 'since Python 3.10 and will be removed in a subsequent ' + 'version', + DeprecationWarning, 2) width = istop - istart - istep = _index(step) + try: + istep = _index(step) + except TypeError: + istep = int(step) + if istep != step: + _warn('randrange() will raise TypeError in the future', + DeprecationWarning, 2) + raise ValueError("non-integer step for randrange()") + _warn('non-integer arguments to randrange() have been deprecated ' + 'since Python 3.10 and will be removed in a subsequent ' + 'version', + DeprecationWarning, 2) # Fast path. if istep == 1: if width > 0: return istart + self._randbelow(width) - raise ValueError(f"empty range in randrange({start}, {stop}, {step})") + raise ValueError("empty range for randrange() (%d, %d, %d)" % (istart, istop, width)) # Non-unit step argument supplied. if istep > 0: @@ -320,7 +352,7 @@ def randrange(self, start, stop=None, step=_ONE): else: raise ValueError("zero step for randrange()") if n <= 0: - raise ValueError(f"empty range in randrange({start}, {stop}, {step})") + raise ValueError("empty range for randrange()") return istart + istep * self._randbelow(n) def randint(self, a, b): diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py index f980c5b8df0d2..5b066d23dd3fd 100644 --- a/Lib/test/test_random.py +++ b/Lib/test/test_random.py @@ -481,44 +481,50 @@ def test_randrange_nonunit_step(self): self.assertEqual(rint, 0) def test_randrange_errors(self): - raises_value_error = partial(self.assertRaises, ValueError, self.gen.randrange) - raises_type_error = partial(self.assertRaises, TypeError, self.gen.randrange) - + raises = partial(self.assertRaises, ValueError, self.gen.randrange) # Empty range - raises_value_error(3, 3) - raises_value_error(-721) - raises_value_error(0, 100, -12) - - # Zero step - raises_value_error(0, 42, 0) - raises_type_error(0, 42, 0.0) - raises_type_error(0, 0, 0.0) - - # Non-integer stop - raises_type_error(3.14159) - raises_type_error(3.0) - raises_type_error(Fraction(3, 1)) - raises_type_error('3') - raises_type_error(0, 2.71827) - raises_type_error(0, 2.0) - raises_type_error(0, Fraction(2, 1)) - raises_type_error(0, '2') - raises_type_error(0, 2.71827, 2) - - # Non-integer start - raises_type_error(2.71827, 5) - raises_type_error(2.0, 5) - raises_type_error(Fraction(2, 1), 5) - raises_type_error('2', 5) - raises_type_error(2.71827, 5, 2) - - # Non-integer step - raises_type_error(0, 42, 3.14159) - raises_type_error(0, 42, 3.0) - raises_type_error(0, 42, Fraction(3, 1)) - raises_type_error(0, 42, '3') - raises_type_error(0, 42, 1.0) - raises_type_error(0, 0, 1.0) + raises(3, 3) + raises(-721) + raises(0, 100, -12) + # Non-integer start/stop + self.assertWarns(DeprecationWarning, raises, 3.14159) + self.assertWarns(DeprecationWarning, self.gen.randrange, 3.0) + self.assertWarns(DeprecationWarning, self.gen.randrange, Fraction(3, 1)) + self.assertWarns(DeprecationWarning, raises, '3') + self.assertWarns(DeprecationWarning, raises, 0, 2.71828) + self.assertWarns(DeprecationWarning, self.gen.randrange, 0, 2.0) + self.assertWarns(DeprecationWarning, self.gen.randrange, 0, Fraction(2, 1)) + self.assertWarns(DeprecationWarning, raises, 0, '2') + # Zero and non-integer step + raises(0, 42, 0) + self.assertWarns(DeprecationWarning, raises, 0, 42, 0.0) + self.assertWarns(DeprecationWarning, raises, 0, 0, 0.0) + self.assertWarns(DeprecationWarning, raises, 0, 42, 3.14159) + self.assertWarns(DeprecationWarning, self.gen.randrange, 0, 42, 3.0) + self.assertWarns(DeprecationWarning, self.gen.randrange, 0, 42, Fraction(3, 1)) + self.assertWarns(DeprecationWarning, raises, 0, 42, '3') + self.assertWarns(DeprecationWarning, self.gen.randrange, 0, 42, 1.0) + self.assertWarns(DeprecationWarning, raises, 0, 0, 1.0) + + def test_randrange_argument_handling(self): + randrange = self.gen.randrange + with self.assertWarns(DeprecationWarning): + randrange(10.0, 20, 2) + with self.assertWarns(DeprecationWarning): + randrange(10, 20.0, 2) + with self.assertWarns(DeprecationWarning): + randrange(10, 20, 1.0) + with self.assertWarns(DeprecationWarning): + randrange(10, 20, 2.0) + with self.assertWarns(DeprecationWarning): + with self.assertRaises(ValueError): + randrange(10.5) + with self.assertWarns(DeprecationWarning): + with self.assertRaises(ValueError): + randrange(10, 20.5) + with self.assertWarns(DeprecationWarning): + with self.assertRaises(ValueError): + randrange(10, 20, 1.5) def test_randrange_step(self): # bpo-42772: When stop is None, the step argument was being ignored. diff --git a/Misc/NEWS.d/next/Library/2022-02-03-12-07-41.bpo-46624.f_Qqh0.rst b/Misc/NEWS.d/next/Library/2022-02-03-12-07-41.bpo-46624.f_Qqh0.rst new file mode 100644 index 0000000000000..b0203b9a8b50c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-03-12-07-41.bpo-46624.f_Qqh0.rst @@ -0,0 +1 @@ +Restore support for non-integer arguments of :func:`random.randrange` and :func:`random.randint`. From webhook-mailer at python.org Thu Feb 3 10:22:54 2022 From: webhook-mailer at python.org (ethanfurman) Date: Thu, 03 Feb 2022 15:22:54 -0000 Subject: [Python-checkins] bpo-46569: [Enum] fix typo in `StrEnum` docs (GH-31007) Message-ID: https://github.com/python/cpython/commit/734b1f119be6f0dcd6845c78a9e0a71d88a90b59 commit: 734b1f119be6f0dcd6845c78a9e0a71d88a90b59 branch: main author: Nikita Sobolev committer: ethanfurman date: 2022-02-03T07:22:41-08:00 summary: bpo-46569: [Enum] fix typo in `StrEnum` docs (GH-31007) files: M Doc/library/enum.rst diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index a37c9d4506241..672e256c77c0d 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -420,7 +420,7 @@ Data Types .. note:: :meth:`__str__` is :func:`str.__str__` to better support the *replacement of existing constants* use-case. :meth:`__format__` is likewise - :func:`int.__format__` for that same reason. + :func:`str.__format__` for that same reason. .. versionadded:: 3.11 From webhook-mailer at python.org Thu Feb 3 10:51:14 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 03 Feb 2022 15:51:14 -0000 Subject: [Python-checkins] bpo-46436: Fix command-line option -d/--directory in module http.server (GH-30701) Message-ID: https://github.com/python/cpython/commit/2d080347d74078a55c47715d232d1ab8dc8cd603 commit: 2d080347d74078a55c47715d232d1ab8dc8cd603 branch: main author: G?ry Ogam committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-03T07:51:05-08:00 summary: bpo-46436: Fix command-line option -d/--directory in module http.server (GH-30701) Fix command-line option -d/--directory in http.server main function that was ignored when combined with --cgi. Automerge-Triggered-By: GH:merwok files: A Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst M Lib/http/server.py diff --git a/Lib/http/server.py b/Lib/http/server.py index 4f9b8a16d45fe..194a503c45f75 100644 --- a/Lib/http/server.py +++ b/Lib/http/server.py @@ -103,8 +103,6 @@ import sys import time import urllib.parse -import contextlib -from functools import partial from http import HTTPStatus @@ -1239,7 +1237,6 @@ def test(HandlerClass=BaseHTTPRequestHandler, """ ServerClass.address_family, addr = _get_best_family(bind, port) - HandlerClass.protocol_version = protocol with ServerClass(addr, HandlerClass) as httpd: host, port = httpd.socket.getsockname()[:2] @@ -1256,29 +1253,29 @@ def test(HandlerClass=BaseHTTPRequestHandler, if __name__ == '__main__': import argparse + import contextlib parser = argparse.ArgumentParser() parser.add_argument('--cgi', action='store_true', - help='Run as CGI Server') + help='run as CGI server') parser.add_argument('--bind', '-b', metavar='ADDRESS', - help='Specify alternate bind address ' - '[default: all interfaces]') + help='specify alternate bind address ' + '(default: all interfaces)') parser.add_argument('--directory', '-d', default=os.getcwd(), - help='Specify alternative directory ' - '[default:current directory]') - parser.add_argument('port', action='store', - default=8000, type=int, + help='specify alternate directory ' + '(default: current directory)') + parser.add_argument('port', action='store', default=8000, type=int, nargs='?', - help='Specify alternate port [default: 8000]') + help='specify alternate port (default: 8000)') args = parser.parse_args() if args.cgi: handler_class = CGIHTTPRequestHandler else: - handler_class = partial(SimpleHTTPRequestHandler, - directory=args.directory) + handler_class = SimpleHTTPRequestHandler # ensure dual-stack is not disabled; ref #38907 class DualStackServer(ThreadingHTTPServer): + def server_bind(self): # suppress exception when protocol is IPv4 with contextlib.suppress(Exception): @@ -1286,6 +1283,10 @@ def server_bind(self): socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0) return super().server_bind() + def finish_request(self, request, client_address): + self.RequestHandlerClass(request, client_address, self, + directory=args.directory) + test( HandlerClass=handler_class, ServerClass=DualStackServer, diff --git a/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst b/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst new file mode 100644 index 0000000000000..ccfd949506443 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst @@ -0,0 +1,3 @@ +Fix command-line option ``-d``/``--directory`` in module :mod:`http.server` +which is ignored when combined with command-line option ``--cgi``. Patch by +G?ry Ogam. From webhook-mailer at python.org Thu Feb 3 10:55:00 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 03 Feb 2022 15:55:00 -0000 Subject: [Python-checkins] bpo-45773: Remove invalid peephole optimizations (GH-31066) Message-ID: https://github.com/python/cpython/commit/ff6948b1286c854ee77dfc0b23b9d828b36873e4 commit: ff6948b1286c854ee77dfc0b23b9d828b36873e4 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-03T07:54:51-08:00 summary: bpo-45773: Remove invalid peephole optimizations (GH-31066) (cherry picked from commit e0433c1e70254d4d0357a9e14596929a04bdf769) Co-authored-by: Brandt Bucher files: A Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst M Python/compile.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst new file mode 100644 index 0000000000000..45da5116fc940 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst @@ -0,0 +1 @@ +Remove two invalid "peephole" optimizations from the bytecode compiler. diff --git a/Python/compile.c b/Python/compile.c index 3eb34d89cc55f..a7f62bbcbb33e 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -7449,7 +7449,6 @@ optimize_basic_block(struct compiler *c, basicblock *bb, PyObject *consts) switch (target->i_opcode) { case JUMP_ABSOLUTE: case JUMP_FORWARD: - case JUMP_IF_FALSE_OR_POP: i -= jump_thread(inst, target, POP_JUMP_IF_FALSE); } break; @@ -7457,7 +7456,6 @@ optimize_basic_block(struct compiler *c, basicblock *bb, PyObject *consts) switch (target->i_opcode) { case JUMP_ABSOLUTE: case JUMP_FORWARD: - case JUMP_IF_TRUE_OR_POP: i -= jump_thread(inst, target, POP_JUMP_IF_TRUE); } break; From webhook-mailer at python.org Thu Feb 3 13:37:00 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 03 Feb 2022 18:37:00 -0000 Subject: [Python-checkins] Pass reference to func, as well as args, when pushing frame. (GH-31100) Message-ID: https://github.com/python/cpython/commit/da4d4ec1851714bb56a5e8d0c1cd5bf9842b4cee commit: da4d4ec1851714bb56a5e8d0c1cd5bf9842b4cee branch: main author: Mark Shannon committer: markshannon date: 2022-02-03T18:36:28Z summary: Pass reference to func, as well as args, when pushing frame. (GH-31100) files: M Include/internal/pycore_frame.h M Objects/frameobject.c M Python/ceval.c M Python/frame.c M Python/pystate.c diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h index 85b9cf0f77bcb..1ad156290a55e 100644 --- a/Include/internal/pycore_frame.h +++ b/Include/internal/pycore_frame.h @@ -87,12 +87,12 @@ static inline void _PyFrame_StackPush(InterpreterFrame *f, PyObject *value) { void _PyFrame_Copy(InterpreterFrame *src, InterpreterFrame *dest); +/* Consumes reference to func */ static inline void _PyFrame_InitializeSpecials( InterpreterFrame *frame, PyFunctionObject *func, PyObject *locals, int nlocalsplus) { - Py_INCREF(func); frame->f_func = func; frame->f_code = (PyCodeObject *)Py_NewRef(func->func_code); frame->f_builtins = func->func_builtins; @@ -166,9 +166,6 @@ _PyFrame_FastToLocalsWithError(InterpreterFrame *frame); void _PyFrame_LocalsToFast(InterpreterFrame *frame, int clear); -InterpreterFrame *_PyThreadState_PushFrame( - PyThreadState *tstate, PyFunctionObject *func, PyObject *locals); - extern InterpreterFrame * _PyThreadState_BumpFramePointerSlow(PyThreadState *tstate, size_t size); @@ -189,6 +186,7 @@ _PyThreadState_BumpFramePointer(PyThreadState *tstate, size_t size) void _PyThreadState_PopFrame(PyThreadState *tstate, InterpreterFrame *frame); +/* Consume reference to func */ InterpreterFrame * _PyFrame_Push(PyThreadState *tstate, PyFunctionObject *func); diff --git a/Objects/frameobject.c b/Objects/frameobject.c index 15da1325d1480..78f3894111bc3 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -784,6 +784,8 @@ _Py_IDENTIFIER(__builtins__); static void init_frame(InterpreterFrame *frame, PyFunctionObject *func, PyObject *locals) { + /* _PyFrame_InitializeSpecials consumes reference to func */ + Py_INCREF(func); PyCodeObject *code = (PyCodeObject *)func->func_code; _PyFrame_InitializeSpecials(frame, func, locals, code->co_nlocalsplus); for (Py_ssize_t i = 0; i < code->co_nlocalsplus; i++) { diff --git a/Python/ceval.c b/Python/ceval.c index 3c52c5824b44a..3197fe8eebb56 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2243,6 +2243,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr goto error; } CALL_STAT_INC(frames_pushed); + Py_INCREF(getitem); _PyFrame_InitializeSpecials(new_frame, getitem, NULL, code->co_nlocalsplus); STACK_SHRINK(2); @@ -4590,7 +4591,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr STACK_SHRINK(call_shape.postcall_shrink); // The frame has stolen all the arguments from the stack, // so there is no need to clean them up. - Py_DECREF(function); if (new_frame == NULL) { goto error; } @@ -4675,7 +4675,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr new_frame->localsplus[i] = NULL; } STACK_SHRINK(call_shape.postcall_shrink); - Py_DECREF(func); _PyFrame_SetStackPointer(frame, stack_pointer); new_frame->previous = frame; frame = cframe.current_frame = new_frame; @@ -4712,7 +4711,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr new_frame->localsplus[i] = NULL; } STACK_SHRINK(call_shape.postcall_shrink); - Py_DECREF(func); _PyFrame_SetStackPointer(frame, stack_pointer); new_frame->previous = frame; frame = cframe.current_frame = new_frame; @@ -6077,7 +6075,7 @@ initialize_locals(PyThreadState *tstate, PyFunctionObject *func, return -1; } -/* Consumes all the references to the args */ +/* Consumes references to func and all the args */ static InterpreterFrame * _PyEvalFramePushAndInit(PyThreadState *tstate, PyFunctionObject *func, PyObject *locals, PyObject* const* args, @@ -6131,7 +6129,9 @@ _PyEval_Vector(PyThreadState *tstate, PyFunctionObject *func, PyObject* const* args, size_t argcount, PyObject *kwnames) { - /* _PyEvalFramePushAndInit consumes all the references to its arguments */ + /* _PyEvalFramePushAndInit consumes the references + * to func and all its arguments */ + Py_INCREF(func); for (size_t i = 0; i < argcount; i++) { Py_INCREF(args[i]); } diff --git a/Python/frame.c b/Python/frame.c index ca7c5f9c94e07..76697cfa08313 100644 --- a/Python/frame.c +++ b/Python/frame.c @@ -109,6 +109,7 @@ _PyFrame_Clear(InterpreterFrame *frame) Py_DECREF(frame->f_code); } +/* Consumes reference to func */ InterpreterFrame * _PyFrame_Push(PyThreadState *tstate, PyFunctionObject *func) { @@ -117,6 +118,7 @@ _PyFrame_Push(PyThreadState *tstate, PyFunctionObject *func) CALL_STAT_INC(frames_pushed); InterpreterFrame *new_frame = _PyThreadState_BumpFramePointer(tstate, size); if (new_frame == NULL) { + Py_DECREF(func); return NULL; } _PyFrame_InitializeSpecials(new_frame, func, NULL, code->co_nlocalsplus); diff --git a/Python/pystate.c b/Python/pystate.c index 77467944e2afb..a85460c15103d 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -2212,26 +2212,6 @@ _PyThreadState_BumpFramePointerSlow(PyThreadState *tstate, size_t size) return (InterpreterFrame *)base; } - -InterpreterFrame * -_PyThreadState_PushFrame(PyThreadState *tstate, PyFunctionObject *func, PyObject *locals) -{ - PyCodeObject *code = (PyCodeObject *)func->func_code; - int nlocalsplus = code->co_nlocalsplus; - size_t size = nlocalsplus + code->co_stacksize + - FRAME_SPECIALS_SIZE; - CALL_STAT_INC(frames_pushed); - InterpreterFrame *frame = _PyThreadState_BumpFramePointer(tstate, size); - if (frame == NULL) { - return NULL; - } - _PyFrame_InitializeSpecials(frame, func, locals, nlocalsplus); - for (int i=0; i < nlocalsplus; i++) { - frame->localsplus[i] = NULL; - } - return frame; -} - void _PyThreadState_PopFrame(PyThreadState *tstate, InterpreterFrame * frame) { From webhook-mailer at python.org Thu Feb 3 13:41:00 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 03 Feb 2022 18:41:00 -0000 Subject: [Python-checkins] Collect stats for UNPACK_SEQUENCE. (GH-31105) Message-ID: https://github.com/python/cpython/commit/a0401d83720d93cd95ddf25f86874bfbee528722 commit: a0401d83720d93cd95ddf25f86874bfbee528722 branch: main author: Mark Shannon committer: markshannon date: 2022-02-03T18:40:56Z summary: Collect stats for UNPACK_SEQUENCE. (GH-31105) files: M Python/ceval.c M Python/specialize.c diff --git a/Python/ceval.c b/Python/ceval.c index 3197fe8eebb56..b4029d1081dcb 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2792,6 +2792,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(UNPACK_SEQUENCE) { PREDICTED(UNPACK_SEQUENCE); PyObject *seq = POP(), *item, **items; +#ifdef Py_STATS + extern int _PySpecialization_ClassifySequence(PyObject *); + _py_stats.opcode_stats[UNPACK_SEQUENCE].specialization.failure++; + _py_stats.opcode_stats[UNPACK_SEQUENCE].specialization. + failure_kinds[_PySpecialization_ClassifySequence(seq)]++; +#endif if (PyTuple_CheckExact(seq) && PyTuple_GET_SIZE(seq) == oparg) { items = ((PyTupleObject *)seq)->ob_item; diff --git a/Python/specialize.c b/Python/specialize.c index 214f29751f388..4070d6a6a0be4 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -572,6 +572,10 @@ initial_counter_value(void) { #define SPEC_FAIL_ITER_DICT_VALUES 22 #define SPEC_FAIL_ITER_ENUMERATE 23 +/* UNPACK_SEQUENCE */ +#define SPEC_FAIL_TUPLE 10 +#define SPEC_FAIL_LIST 11 + static int specialize_module_load_attr( @@ -1880,7 +1884,6 @@ _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, adaptive->counter = initial_counter_value(); } - int _PySpecialization_ClassifyIterator(PyObject *iter) { @@ -1930,3 +1933,15 @@ int } return SPEC_FAIL_OTHER; } + +int +_PySpecialization_ClassifySequence(PyObject *seq) +{ + if (PyTuple_CheckExact(seq)) { + return SPEC_FAIL_TUPLE; + } + if (PyList_CheckExact(seq)) { + return SPEC_FAIL_LIST; + } + return SPEC_FAIL_OTHER; +} From webhook-mailer at python.org Thu Feb 3 13:46:59 2022 From: webhook-mailer at python.org (benjaminp) Date: Thu, 03 Feb 2022 18:46:59 -0000 Subject: [Python-checkins] closes bpo-46626: Expose IP_BIND_ADDRESS_NO_PORT socket option. (GH-31106) Message-ID: https://github.com/python/cpython/commit/1aa6be06c4cb7f04a340adb1c7b16b89803ef254 commit: 1aa6be06c4cb7f04a340adb1c7b16b89803ef254 branch: main author: Benjamin Peterson committer: benjaminp date: 2022-02-03T10:46:50-08:00 summary: closes bpo-46626: Expose IP_BIND_ADDRESS_NO_PORT socket option. (GH-31106) files: A Misc/NEWS.d/next/Library/2022-02-03-10-22-42.bpo-46626.r2e-n_.rst M Modules/socketmodule.c diff --git a/Misc/NEWS.d/next/Library/2022-02-03-10-22-42.bpo-46626.r2e-n_.rst b/Misc/NEWS.d/next/Library/2022-02-03-10-22-42.bpo-46626.r2e-n_.rst new file mode 100644 index 0000000000000..aaca73d36cdc6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-03-10-22-42.bpo-46626.r2e-n_.rst @@ -0,0 +1 @@ +Expose Linux's ``IP_BIND_ADDRESS_NO_PORT`` option in :mod:`socket`. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 1c8ef1eb3b5b3..3fca9f68512e8 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -8072,6 +8072,9 @@ PyInit__socket(void) #ifdef IP_TRANSPARENT PyModule_AddIntMacro(m, IP_TRANSPARENT); #endif +#ifdef IP_BIND_ADDRESS_NO_PORT + PyModule_AddIntMacro(m, IP_BIND_ADDRESS_NO_PORT); +#endif /* IPv6 [gs]etsockopt options, defined in RFC2553 */ #ifdef IPV6_JOIN_GROUP From webhook-mailer at python.org Thu Feb 3 14:44:40 2022 From: webhook-mailer at python.org (terryjreedy) Date: Thu, 03 Feb 2022 19:44:40 -0000 Subject: [Python-checkins] bpo-45975: IDLE - Remove extraneous parens (GH-31107) Message-ID: https://github.com/python/cpython/commit/916d0d822c79933f4c420f7a36f16f3eb788646b commit: 916d0d822c79933f4c420f7a36f16f3eb788646b branch: main author: Terry Jan Reedy committer: terryjreedy date: 2022-02-03T14:44:35-05:00 summary: bpo-45975: IDLE - Remove extraneous parens (GH-31107) mistakenly included in 3 files in previous PR and backported both to 3.10 and 3.9. files: M Lib/idlelib/pyparse.py M Lib/idlelib/replace.py M Lib/idlelib/run.py diff --git a/Lib/idlelib/pyparse.py b/Lib/idlelib/pyparse.py index a94327533d865..8545c63e1435d 100644 --- a/Lib/idlelib/pyparse.py +++ b/Lib/idlelib/pyparse.py @@ -179,7 +179,7 @@ def find_good_parse_start(self, is_char_in_string): # Peeking back worked; look forward until _synchre no longer # matches. i = pos + 1 - while (m := _synchre(code, i)): + while m := _synchre(code, i): s, i = m.span() if not is_char_in_string(s): pos = s diff --git a/Lib/idlelib/replace.py b/Lib/idlelib/replace.py index ac04ed94dd475..ca83173877ad1 100644 --- a/Lib/idlelib/replace.py +++ b/Lib/idlelib/replace.py @@ -158,8 +158,8 @@ def replace_all(self, event=None): first = last = None # XXX ought to replace circular instead of top-to-bottom when wrapping text.undo_block_start() - while (res := self.engine.search_forward( - text, prog, line, col, wrap=False, ok=ok)): + while res := self.engine.search_forward( + text, prog, line, col, wrap=False, ok=ok): line, m = res chars = text.get("%d.0" % line, "%d.0" % (line+1)) orig = m.group() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py index 01f8d65426abc..aaa9b5ce8d181 100644 --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -482,7 +482,7 @@ def read(self, size=-1): result = self._line_buffer self._line_buffer = '' if size < 0: - while (line := self.shell.readline()): + while line := self.shell.readline(): result += line else: while len(result) < size: From webhook-mailer at python.org Thu Feb 3 15:44:20 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 03 Feb 2022 20:44:20 -0000 Subject: [Python-checkins] bpo-45975: IDLE - Remove extraneous parens (GH-31107) Message-ID: https://github.com/python/cpython/commit/63523e7b2a631f28134b25a8063d50e08c741db6 commit: 63523e7b2a631f28134b25a8063d50e08c741db6 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-03T12:44:11-08:00 summary: bpo-45975: IDLE - Remove extraneous parens (GH-31107) mistakenly included in 3 files in previous PR and backported both to 3.10 and 3.9. (cherry picked from commit 916d0d822c79933f4c420f7a36f16f3eb788646b) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/pyparse.py M Lib/idlelib/replace.py M Lib/idlelib/run.py diff --git a/Lib/idlelib/pyparse.py b/Lib/idlelib/pyparse.py index a94327533d865..8545c63e1435d 100644 --- a/Lib/idlelib/pyparse.py +++ b/Lib/idlelib/pyparse.py @@ -179,7 +179,7 @@ def find_good_parse_start(self, is_char_in_string): # Peeking back worked; look forward until _synchre no longer # matches. i = pos + 1 - while (m := _synchre(code, i)): + while m := _synchre(code, i): s, i = m.span() if not is_char_in_string(s): pos = s diff --git a/Lib/idlelib/replace.py b/Lib/idlelib/replace.py index ac04ed94dd475..ca83173877ad1 100644 --- a/Lib/idlelib/replace.py +++ b/Lib/idlelib/replace.py @@ -158,8 +158,8 @@ def replace_all(self, event=None): first = last = None # XXX ought to replace circular instead of top-to-bottom when wrapping text.undo_block_start() - while (res := self.engine.search_forward( - text, prog, line, col, wrap=False, ok=ok)): + while res := self.engine.search_forward( + text, prog, line, col, wrap=False, ok=ok): line, m = res chars = text.get("%d.0" % line, "%d.0" % (line+1)) orig = m.group() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py index 01f8d65426abc..aaa9b5ce8d181 100644 --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -482,7 +482,7 @@ def read(self, size=-1): result = self._line_buffer self._line_buffer = '' if size < 0: - while (line := self.shell.readline()): + while line := self.shell.readline(): result += line else: while len(result) < size: From webhook-mailer at python.org Thu Feb 3 15:44:28 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 03 Feb 2022 20:44:28 -0000 Subject: [Python-checkins] bpo-45975: IDLE - Remove extraneous parens (GH-31107) Message-ID: https://github.com/python/cpython/commit/cf7cb1a2bf40516dc571d1d90c12b632dcd9b8c8 commit: cf7cb1a2bf40516dc571d1d90c12b632dcd9b8c8 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-03T12:44:23-08:00 summary: bpo-45975: IDLE - Remove extraneous parens (GH-31107) mistakenly included in 3 files in previous PR and backported both to 3.10 and 3.9. (cherry picked from commit 916d0d822c79933f4c420f7a36f16f3eb788646b) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/pyparse.py M Lib/idlelib/replace.py M Lib/idlelib/run.py diff --git a/Lib/idlelib/pyparse.py b/Lib/idlelib/pyparse.py index a94327533d865..8545c63e1435d 100644 --- a/Lib/idlelib/pyparse.py +++ b/Lib/idlelib/pyparse.py @@ -179,7 +179,7 @@ def find_good_parse_start(self, is_char_in_string): # Peeking back worked; look forward until _synchre no longer # matches. i = pos + 1 - while (m := _synchre(code, i)): + while m := _synchre(code, i): s, i = m.span() if not is_char_in_string(s): pos = s diff --git a/Lib/idlelib/replace.py b/Lib/idlelib/replace.py index 70d761db12630..7e55a4a43a44b 100644 --- a/Lib/idlelib/replace.py +++ b/Lib/idlelib/replace.py @@ -156,8 +156,8 @@ def replace_all(self, event=None): first = last = None # XXX ought to replace circular instead of top-to-bottom when wrapping text.undo_block_start() - while (res := self.engine.search_forward( - text, prog, line, col, wrap=False, ok=ok)): + while res := self.engine.search_forward( + text, prog, line, col, wrap=False, ok=ok): line, m = res chars = text.get("%d.0" % line, "%d.0" % (line+1)) orig = m.group() diff --git a/Lib/idlelib/run.py b/Lib/idlelib/run.py index 4246f497cb382..d6f1049612f71 100644 --- a/Lib/idlelib/run.py +++ b/Lib/idlelib/run.py @@ -468,7 +468,7 @@ def read(self, size=-1): result = self._line_buffer self._line_buffer = '' if size < 0: - while (line := self.shell.readline()): + while line := self.shell.readline(): result += line else: while len(result) < size: From webhook-mailer at python.org Thu Feb 3 17:06:26 2022 From: webhook-mailer at python.org (terryjreedy) Date: Thu, 03 Feb 2022 22:06:26 -0000 Subject: [Python-checkins] bpo-46630: Fix initial focus of IDLE query dialogs (GH-31112) Message-ID: https://github.com/python/cpython/commit/d1df81a730499cc6286d02afa6028a1e9c22bbbf commit: d1df81a730499cc6286d02afa6028a1e9c22bbbf branch: main author: Terry Jan Reedy committer: terryjreedy date: 2022-02-03T17:06:17-05:00 summary: bpo-46630: Fix initial focus of IDLE query dialogs (GH-31112) On Windows, one had to Tab or click on the entry box to get a cursor and be able to enter anything. files: A Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/query.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 34b2c08a92256..0bfadfd81e2dd 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,6 +4,9 @@ Released on 2022-10-03 ========================= +bpo-46630: Make query dialogs on Windows start with a cursor in the +entry box. + bpo-46591: Make the IDLE doc URL on the About IDLE dialog clickable. bpo-45296: Clarify close, quit, and exit in IDLE. In the File menu, diff --git a/Lib/idlelib/query.py b/Lib/idlelib/query.py index fefa5aac1b7f5..df02f2123ab02 100644 --- a/Lib/idlelib/query.py +++ b/Lib/idlelib/query.py @@ -83,6 +83,7 @@ def __init__(self, parent, title, message, *, text0='', used_names={}, if not _utest: self.deiconify() # Unhide now that geometry set. + self.entry.focus_set() self.wait_window() def create_widgets(self, ok_text='OK'): # Do not replace. @@ -100,7 +101,6 @@ def create_widgets(self, ok_text='OK'): # Do not replace. text=self.message) self.entryvar = StringVar(self, self.text0) self.entry = Entry(frame, width=30, textvariable=self.entryvar) - self.entry.focus_set() self.error_font = Font(name='TkCaptionFont', exists=True, root=self.parent) self.entry_error = Label(frame, text=' ', foreground='red', diff --git a/Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst b/Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst new file mode 100644 index 0000000000000..81e35486eaf21 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst @@ -0,0 +1 @@ +Make query dialogs on Windows start with a cursor in the entry box. From webhook-mailer at python.org Thu Feb 3 17:33:35 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 03 Feb 2022 22:33:35 -0000 Subject: [Python-checkins] bpo-46630: Fix initial focus of IDLE query dialogs (GH-31112) Message-ID: https://github.com/python/cpython/commit/dc315f30f86a1dc7c4607398b379d7c0b55c7549 commit: dc315f30f86a1dc7c4607398b379d7c0b55c7549 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-03T14:33:26-08:00 summary: bpo-46630: Fix initial focus of IDLE query dialogs (GH-31112) On Windows, one had to Tab or click on the entry box to get a cursor and be able to enter anything. (cherry picked from commit d1df81a730499cc6286d02afa6028a1e9c22bbbf) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/query.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 2ba8d581bbc73..73c8b7a32d767 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,6 +4,9 @@ Released on 2022-05-16 ========================= +bpo-46630: Make query dialogs on Windows start with a cursor in the +entry box. + bpo-46591: Make the IDLE doc URL on the About IDLE dialog clickable. bpo-45296: Clarify close, quit, and exit in IDLE. In the File menu, diff --git a/Lib/idlelib/query.py b/Lib/idlelib/query.py index fefa5aac1b7f5..df02f2123ab02 100644 --- a/Lib/idlelib/query.py +++ b/Lib/idlelib/query.py @@ -83,6 +83,7 @@ def __init__(self, parent, title, message, *, text0='', used_names={}, if not _utest: self.deiconify() # Unhide now that geometry set. + self.entry.focus_set() self.wait_window() def create_widgets(self, ok_text='OK'): # Do not replace. @@ -100,7 +101,6 @@ def create_widgets(self, ok_text='OK'): # Do not replace. text=self.message) self.entryvar = StringVar(self, self.text0) self.entry = Entry(frame, width=30, textvariable=self.entryvar) - self.entry.focus_set() self.error_font = Font(name='TkCaptionFont', exists=True, root=self.parent) self.entry_error = Label(frame, text=' ', foreground='red', diff --git a/Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst b/Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst new file mode 100644 index 0000000000000..81e35486eaf21 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst @@ -0,0 +1 @@ +Make query dialogs on Windows start with a cursor in the entry box. From webhook-mailer at python.org Thu Feb 3 17:34:10 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 03 Feb 2022 22:34:10 -0000 Subject: [Python-checkins] bpo-46630: Fix initial focus of IDLE query dialogs (GH-31112) Message-ID: https://github.com/python/cpython/commit/4f76b3667d856a13107c65d44d802d0e73c3f104 commit: 4f76b3667d856a13107c65d44d802d0e73c3f104 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-03T14:34:03-08:00 summary: bpo-46630: Fix initial focus of IDLE query dialogs (GH-31112) On Windows, one had to Tab or click on the entry box to get a cursor and be able to enter anything. (cherry picked from commit d1df81a730499cc6286d02afa6028a1e9c22bbbf) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/query.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 724709a29ff63..e8055a054640b 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,6 +4,9 @@ Released on 2022-05-16 ========================= +bpo-46630: Make query dialogs on Windows start with a cursor in the +entry box. + bpo-46591: Make the IDLE doc URL on the About IDLE dialog clickable. bpo-45296: Clarify close, quit, and exit in IDLE. In the File menu, diff --git a/Lib/idlelib/query.py b/Lib/idlelib/query.py index fefa5aac1b7f5..df02f2123ab02 100644 --- a/Lib/idlelib/query.py +++ b/Lib/idlelib/query.py @@ -83,6 +83,7 @@ def __init__(self, parent, title, message, *, text0='', used_names={}, if not _utest: self.deiconify() # Unhide now that geometry set. + self.entry.focus_set() self.wait_window() def create_widgets(self, ok_text='OK'): # Do not replace. @@ -100,7 +101,6 @@ def create_widgets(self, ok_text='OK'): # Do not replace. text=self.message) self.entryvar = StringVar(self, self.text0) self.entry = Entry(frame, width=30, textvariable=self.entryvar) - self.entry.focus_set() self.error_font = Font(name='TkCaptionFont', exists=True, root=self.parent) self.entry_error = Label(frame, text=' ', foreground='red', diff --git a/Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst b/Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst new file mode 100644 index 0000000000000..81e35486eaf21 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2022-02-03-15-47-53.bpo-46630.tREOjo.rst @@ -0,0 +1 @@ +Make query dialogs on Windows start with a cursor in the entry box. From webhook-mailer at python.org Thu Feb 3 17:57:42 2022 From: webhook-mailer at python.org (pablogsal) Date: Thu, 03 Feb 2022 22:57:42 -0000 Subject: [Python-checkins] Python 3.11.0a5 Message-ID: https://github.com/python/cpython/commit/c4e4b91557f18f881f393d80f5d8ce29de760e67 commit: c4e4b91557f18f881f393d80f5d8ce29de760e67 branch: main author: Pablo Galindo committer: pablogsal date: 2022-02-03T18:37:08Z summary: Python 3.11.0a5 files: A Misc/NEWS.d/3.11.0a5.rst D Misc/NEWS.d/next/Build/2021-02-10-17-54-04.bpo-43112.H5Lat6.rst D Misc/NEWS.d/next/Build/2022-01-08-12-43-31.bpo-45925.38F3NO.rst D Misc/NEWS.d/next/Build/2022-01-09-11-24-54.bpo-45569.zCIENy.rst D Misc/NEWS.d/next/Build/2022-01-19-04-36-15.bpo-46429.y0OtVL.rst D Misc/NEWS.d/next/Build/2022-01-20-05-27-07.bpo-46443.udCVII.rst D Misc/NEWS.d/next/Build/2022-01-22-11-06-23.bpo-46471.03snrE.rst D Misc/NEWS.d/next/Build/2022-01-25-12-32-37.bpo-46513.mPm9B4.rst D Misc/NEWS.d/next/Build/2022-01-26-22-59-12.bpo-38472.RxfLho.rst D Misc/NEWS.d/next/Build/2022-02-01-14-07-37.bpo-46600.NNLnfj.rst D Misc/NEWS.d/next/Build/2022-02-02-02-06-07.bpo-46602.8GaOZ2.rst D Misc/NEWS.d/next/C API/2020-09-11-02-50-41.bpo-14916.QN1Y03.rst D Misc/NEWS.d/next/C API/2021-10-18-16-54-24.bpo-45459.Y1pEZs.rst D Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst D Misc/NEWS.d/next/C API/2022-01-27-02-37-18.bpo-40170.XxQB0i.rst D Misc/NEWS.d/next/C API/2022-01-27-02-51-22.bpo-40170.uPolek.rst D Misc/NEWS.d/next/Core and Builtins/2021-05-04-21-55-49.bpo-44024.M9m8Qd.rst D Misc/NEWS.d/next/Core and Builtins/2021-12-11-11-36-48.bpo-46045.sfThay.rst D Misc/NEWS.d/next/Core and Builtins/2021-12-12-00-49-19.bpo-30512.nU9E9V.rst D Misc/NEWS.d/next/Core and Builtins/2021-12-16-00-24-00.bpo-46091.rJ_e_e.rst D Misc/NEWS.d/next/Core and Builtins/2021-12-16-15-04-58.bpo-46028.zfWacB.rst D Misc/NEWS.d/next/Core and Builtins/2021-12-23-12-32-45.bpo-46161.EljBmu.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-12-17-15-17.bpo-46361.mgI_j_.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-14-20-55-34.bpo-46383.v8MTl4.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-16-15-40-11.bpo-46406.g0mke-.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-17-12-57-27.bpo-46409.HouS6m.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-17-23-12-01.bpo-46407.2_5a7R.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-20-17-13-49.bpo-43683.BqQ26Z.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-21-12-24-14.bpo-46417.i3IqMf.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-22-14-39-23.bpo-46417.3U5SfN.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-23-06-56-33.bpo-46481.X_FfnB.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-24-15-39-34.bpo-46476.cvP1Mr.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-24-16-58-01.bpo-46431.N6mKAx.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-24-21-24-41.bpo-46503.4UrPsE.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-25-11-44-17.bpo-46329.SEhynE.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-25-17-40-07.bpo-46528.2Qmni9.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-25-19-34-55.bpo-46527.mQLNPk.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-27-10-49-34.bpo-46458.5Gm3Gv.rst D Misc/NEWS.d/next/Core and Builtins/2022-01-30-18-23-08.bpo-44977.BQV_zS.rst D Misc/NEWS.d/next/Core and Builtins/2022-02-01-01-17-28.bpo-45885.CjyNf_.rst D Misc/NEWS.d/next/Core and Builtins/2022-02-01-10-23-21.bpo-46564.6Xc2_H.rst D Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst D Misc/NEWS.d/next/Documentation/2022-01-21-21-33-48.bpo-46463.fBbdTG.rst D Misc/NEWS.d/next/IDLE/2022-01-26-19-33-55.bpo-45296.LzZKdU.rst D Misc/NEWS.d/next/Library/2021-07-31-23-18-50.bpo-44791.4jFdpO.rst D Misc/NEWS.d/next/Library/2021-12-15-06-29-00.bpo-46080.AuQpLt.rst D Misc/NEWS.d/next/Library/2021-12-16-23-42-54.bpo-46103.LMnZAN.rst D Misc/NEWS.d/next/Library/2021-12-18-18-41-30.bpo-46124.ESPrb7.rst D Misc/NEWS.d/next/Library/2021-12-28-11-55-10.bpo-21987.avBK-p.rst D Misc/NEWS.d/next/Library/2021-12-29-13-42-55.bpo-26552.1BqeAn.rst D Misc/NEWS.d/next/Library/2021-12-29-14-42-09.bpo-43118.BoVi_5.rst D Misc/NEWS.d/next/Library/2022-01-03-16-25-06.bpo-46242.f4l_CL.rst D Misc/NEWS.d/next/Library/2022-01-04-18-05-25.bpo-46258.DYgwRo.rst D Misc/NEWS.d/next/Library/2022-01-05-03-09-29.bpo-43012.RVhLIL.rst D Misc/NEWS.d/next/Library/2022-01-05-03-21-21.bpo-29688.W06bSH.rst D Misc/NEWS.d/next/Library/2022-01-09-15-04-56.bpo-46316.AMTyd0.rst D Misc/NEWS.d/next/Library/2022-01-13-11-41-24.bpo-40066.1QuVli.rst D Misc/NEWS.d/next/Library/2022-01-16-14-07-14.bpo-40280.LtFHfF.rst D Misc/NEWS.d/next/Library/2022-01-17-10-00-02.bpo-46414.Ld0b_y.rst D Misc/NEWS.d/next/Library/2022-01-18-17-24-21.bpo-43869.NayN12.rst D Misc/NEWS.d/next/Library/2022-01-20-10-35-10.bpo-46434.geS-aP.rst D Misc/NEWS.d/next/Library/2022-01-20-10-35-50.bpo-46422.1UAEHL.rst D Misc/NEWS.d/next/Library/2022-01-21-18-19-45.bpo-41906.YBaquj.rst D Misc/NEWS.d/next/Library/2022-01-22-05-05-08.bpo-46469.plUab5.rst D Misc/NEWS.d/next/Library/2022-01-22-13-17-35.bpo-46470.MnNhgU.rst D Misc/NEWS.d/next/Library/2022-01-22-14-45-46.bpo-46474.2DUC62.rst D Misc/NEWS.d/next/Library/2022-01-22-14-49-10.bpo-46474.eKQhvx.rst D Misc/NEWS.d/next/Library/2022-01-23-18-04-45.bpo-41403.SgoHqV.rst D Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst D Misc/NEWS.d/next/Library/2022-01-24-13-00-09.bpo-46483.9XnmKp.rst D Misc/NEWS.d/next/Library/2022-01-24-23-55-30.bpo-46491.jmIKHo.rst D Misc/NEWS.d/next/Library/2022-01-25-10-59-41.bpo-46510.PM5svI.rst D Misc/NEWS.d/next/Library/2022-01-26-20-36-30.bpo-46539.23iW1d.rst D Misc/NEWS.d/next/Library/2022-01-26-23-58-48.bpo-45162.4Jmg_j.rst D Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst D Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst D Misc/NEWS.d/next/Library/2022-01-27-13-30-02.bpo-46544.oFDVWj.rst D Misc/NEWS.d/next/Library/2022-01-28-08-47-53.bpo-46553.f7Uc96.rst D Misc/NEWS.d/next/Library/2022-01-28-19-48-31.bpo-46565.bpZXO4.rst D Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst D Misc/NEWS.d/next/Library/2022-02-03-12-07-41.bpo-46624.f_Qqh0.rst D Misc/NEWS.d/next/Tests/2021-12-18-22-23-50.bpo-46126.0LH3Yb.rst D Misc/NEWS.d/next/Tests/2022-01-14-23-22-41.bpo-40280.nHLWoD.rst D Misc/NEWS.d/next/Tests/2022-01-16-14-11-57.bpo-40280.fNnFfx.rst D Misc/NEWS.d/next/Tests/2022-01-17-13-10-04.bpo-13886.5mZH4b.rst D Misc/NEWS.d/next/Tests/2022-01-28-01-17-10.bpo-46542.xRLTdj.rst D Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst D Misc/NEWS.d/next/Tests/2022-01-31-17-34-13.bpo-46542.RTMm1T.rst D Misc/NEWS.d/next/Tests/2022-02-01-17-13-53.bpo-46600.FMCk8Z.rst D Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst D Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst D Misc/NEWS.d/next/Tests/2022-02-03-00-21-32.bpo-43478.0nfcam.rst D Misc/NEWS.d/next/Windows/2021-09-01-10-48-11.bpo-44934.W1xPATH.rst D Misc/NEWS.d/next/Windows/2022-01-13-22-31-09.bpo-46362.f2cuEb.rst D Misc/NEWS.d/next/Windows/2022-01-25-14-48-39.bpo-33125.5WyY_J.rst D Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst M Include/patchlevel.h M Lib/pydoc_data/topics.py M README.rst diff --git a/Include/patchlevel.h b/Include/patchlevel.h index 1715d53e83ab9..3ab1ce2051a30 100644 --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 11 #define PY_MICRO_VERSION 0 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_ALPHA -#define PY_RELEASE_SERIAL 4 +#define PY_RELEASE_SERIAL 5 /* Version as a string */ -#define PY_VERSION "3.11.0a4+" +#define PY_VERSION "3.11.0a5" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py index 5ce05420414de..cd47603e6cf24 100644 --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Thu Jan 13 19:37:48 2022 +# Autogenerated by Sphinx on Thu Feb 3 18:35:23 2022 topics = {'assert': 'The "assert" statement\n' '**********************\n' '\n' @@ -2518,22 +2518,21 @@ ' >>> print(sys.exc_info())\n' ' (None, None, None)\n' '\n' - 'The "except*" clause(s) are used for handling "ExceptionGroup`s. ' + 'The "except*" clause(s) are used for handling "ExceptionGroup"s. ' 'The\n' - 'exception type for matching is interpreted as in the case of\n' - ':keyword:`except", but in the case of exception groups we can ' - 'have\n' - 'partial matches when the type matches some of the exceptions in ' - 'the\n' - 'group. This means that multiple except* clauses can execute, ' - 'each\n' - 'handling part of the exception group. Each clause executes once ' - 'and\n' - 'handles an exception group of all matching exceptions. Each ' - 'exception\n' - 'in the group is handled by at most one except* clause, the first ' + 'exception type for matching is interpreted as in the case of ' + '"except",\n' + 'but in the case of exception groups we can have partial matches ' + 'when\n' + 'the type matches some of the exceptions in the group. This means ' 'that\n' - 'matches it.\n' + 'multiple except* clauses can execute, each handling part of the\n' + 'exception group. Each clause executes once and handles an ' + 'exception\n' + 'group of all matching exceptions. Each exception in the group ' + 'is\n' + 'handled by at most one except* clause, the first that matches ' + 'it.\n' '\n' ' >>> try:\n' ' ... raise ExceptionGroup("eg",\n' @@ -8082,7 +8081,11 @@ '\n' ' The built-in function "int()" falls back to ' '"__trunc__()" if\n' - ' neither "__int__()" nor "__index__()" is defined.\n', + ' neither "__int__()" nor "__index__()" is defined.\n' + '\n' + ' Changed in version 3.11: The delegation of "int()" to ' + '"__trunc__()"\n' + ' is deprecated.\n', 'objects': 'Objects, values and types\n' '*************************\n' '\n' @@ -8445,12 +8448,12 @@ '\n' ' raise_stmt ::= "raise" [expression ["from" expression]]\n' '\n' - 'If no expressions are present, "raise" re-raises the last ' - 'exception\n' - 'that was active in the current scope. If no exception is active ' - 'in\n' - 'the current scope, a "RuntimeError" exception is raised indicating\n' - 'that this is an error.\n' + 'If no expressions are present, "raise" re-raises the exception that ' + 'is\n' + 'currently being handled, which is also known as the *active\n' + 'exception*. If there isn?t currently an active exception, a\n' + '"RuntimeError" exception is raised indicating that this is an ' + 'error.\n' '\n' 'Otherwise, "raise" evaluates the first expression as the exception\n' 'object. It must be either a subclass or an instance of\n' @@ -8505,11 +8508,14 @@ ' File "", line 4, in \n' ' RuntimeError: Something bad happened\n' '\n' - 'A similar mechanism works implicitly if an exception is raised ' - 'inside\n' - 'an exception handler or a "finally" clause: the previous exception ' - 'is\n' - 'then attached as the new exception?s "__context__" attribute:\n' + 'A similar mechanism works implicitly if a new exception is raised ' + 'when\n' + 'an exception is already being handled. An exception may be ' + 'handled\n' + 'when an "except" or "finally" clause, or a "with" statement, is ' + 'used.\n' + 'The previous exception is then attached as the new exception?s\n' + '"__context__" attribute:\n' '\n' ' >>> try:\n' ' ... print(1 / 0)\n' @@ -10995,6 +11001,10 @@ 'if\n' ' neither "__int__()" nor "__index__()" is defined.\n' '\n' + ' Changed in version 3.11: The delegation of "int()" to ' + '"__trunc__()"\n' + ' is deprecated.\n' + '\n' '\n' 'With Statement Context Managers\n' '===============================\n' @@ -12562,17 +12572,16 @@ ' >>> print(sys.exc_info())\n' ' (None, None, None)\n' '\n' - 'The "except*" clause(s) are used for handling "ExceptionGroup`s. The\n' - 'exception type for matching is interpreted as in the case of\n' - ':keyword:`except", but in the case of exception groups we can have\n' - 'partial matches when the type matches some of the exceptions in the\n' - 'group. This means that multiple except* clauses can execute, each\n' - 'handling part of the exception group. Each clause executes once and\n' - 'handles an exception group of all matching exceptions. Each ' - 'exception\n' - 'in the group is handled by at most one except* clause, the first ' + 'The "except*" clause(s) are used for handling "ExceptionGroup"s. The\n' + 'exception type for matching is interpreted as in the case of ' + '"except",\n' + 'but in the case of exception groups we can have partial matches when\n' + 'the type matches some of the exceptions in the group. This means ' 'that\n' - 'matches it.\n' + 'multiple except* clauses can execute, each handling part of the\n' + 'exception group. Each clause executes once and handles an exception\n' + 'group of all matching exceptions. Each exception in the group is\n' + 'handled by at most one except* clause, the first that matches it.\n' '\n' ' >>> try:\n' ' ... raise ExceptionGroup("eg",\n' diff --git a/Misc/NEWS.d/3.11.0a5.rst b/Misc/NEWS.d/3.11.0a5.rst new file mode 100644 index 0000000000000..c28078da8d833 --- /dev/null +++ b/Misc/NEWS.d/3.11.0a5.rst @@ -0,0 +1,986 @@ +.. bpo: 45773 +.. date: 2022-02-01-14-30-56 +.. nonce: Up77LD +.. release date: 2022-02-03 +.. section: Core and Builtins + +Remove two invalid "peephole" optimizations from the bytecode compiler. + +.. + +.. bpo: 46564 +.. date: 2022-02-01-10-23-21 +.. nonce: 6Xc2_H +.. section: Core and Builtins + +Do not create frame objects when creating :class:`super` object. Patch by +Kumar Aditya. + +.. + +.. bpo: 45885 +.. date: 2022-02-01-01-17-28 +.. nonce: CjyNf_ +.. section: Core and Builtins + +Added more fined-grained specialization failure stats regarding the +``COMPARE_OP`` bytecode. + +.. + +.. bpo: 44977 +.. date: 2022-01-30-18-23-08 +.. nonce: BQV_zS +.. section: Core and Builtins + +The delegation of :func:`int` to :meth:`__trunc__` is now deprecated. +Calling ``int(a)`` when ``type(a)`` implements :meth:`__trunc__` but not +:meth:`__int__` or :meth:`__index__` now raises a :exc:`DeprecationWarning`. + +.. + +.. bpo: 46458 +.. date: 2022-01-27-10-49-34 +.. nonce: 5Gm3Gv +.. section: Core and Builtins + +Reorder code emitted by the compiler for a :keyword:`try`-:keyword:`except` +block so that the :keyword:`else` block's code immediately follows the +:keyword:`try` body (without a jump). This is more optimal for the happy +path. + +.. + +.. bpo: 46527 +.. date: 2022-01-25-19-34-55 +.. nonce: mQLNPk +.. section: Core and Builtins + +Allow passing ``iterable`` as a keyword argument to :func:`enumerate` again. +Patch by Jelle Zijlstra. + +.. + +.. bpo: 46528 +.. date: 2022-01-25-17-40-07 +.. nonce: 2Qmni9 +.. section: Core and Builtins + +Replace several stack manipulation instructions (``DUP_TOP``, +``DUP_TOP_TWO``, ``ROT_TWO``, ``ROT_THREE``, ``ROT_FOUR``, and ``ROT_N``) +with new :opcode:`COPY` and :opcode:`SWAP` instructions. + +.. + +.. bpo: 46329 +.. date: 2022-01-25-11-44-17 +.. nonce: SEhynE +.. section: Core and Builtins + +Use two or three bytecodes to implement most calls. + +Calls without named arguments are implemented as a sequence of two +instructions: ``PRECALL; CALL``. Calls with named arguments are implemented +as a sequence of three instructions: ``PRECALL; KW_NAMES; CALL``. There are +two different ``PRECALL`` instructions: ``PRECALL_FUNTION`` and +``PRECALL_METHOD``. The latter pairs with ``LOAD_METHOD``. + +This partition into pre-call and call allows better specialization, and thus +better performance ultimately. + +There is no change in semantics. + +.. + +.. bpo: 46503 +.. date: 2022-01-24-21-24-41 +.. nonce: 4UrPsE +.. section: Core and Builtins + +Fix an assert when parsing some invalid \N escape sequences in f-strings. + +.. + +.. bpo: 46431 +.. date: 2022-01-24-16-58-01 +.. nonce: N6mKAx +.. section: Core and Builtins + +Improve error message on invalid calls to +:meth:`BaseExceptionGroup.__new__`. + +.. + +.. bpo: 46476 +.. date: 2022-01-24-15-39-34 +.. nonce: cvP1Mr +.. section: Core and Builtins + +Fix memory leak in code objects generated by deepfreeze. Patch by Kumar +Aditya. + +.. + +.. bpo: 46481 +.. date: 2022-01-23-06-56-33 +.. nonce: X_FfnB +.. section: Core and Builtins + +Speed up calls to :meth:`weakref.ref.__call__` by using the :pep:`590` +``vectorcall`` calling convention. Patch by Dong-hee Na. + +.. + +.. bpo: 46417 +.. date: 2022-01-22-14-39-23 +.. nonce: 3U5SfN +.. section: Core and Builtins + +Fix a race condition on setting a type ``__bases__`` attribute: the internal +function ``add_subclass()`` now gets the ``PyTypeObject.tp_subclasses`` +member after calling :c:func:`PyWeakref_NewRef` which can trigger a garbage +collection which can indirectly modify ``PyTypeObject.tp_subclasses``. Patch +by Victor Stinner. + +.. + +.. bpo: 46417 +.. date: 2022-01-21-12-24-14 +.. nonce: i3IqMf +.. section: Core and Builtins + +``python -X showrefcount`` now shows the total reference count after +clearing and destroyed the main Python interpreter. Previously, it was shown +before. Patch by Victor Stinner. + +.. + +.. bpo: 43683 +.. date: 2022-01-20-17-13-49 +.. nonce: BqQ26Z +.. section: Core and Builtins + +Add ASYNC_GEN_WRAP opcode to wrap the value to be yielded in async +generators. Removes the need to special case async generators in the +``YIELD_VALUE`` instruction. + +.. + +.. bpo: 46407 +.. date: 2022-01-17-23-12-01 +.. nonce: 2_5a7R +.. section: Core and Builtins + +Optimize some modulo operations in ``Objects/longobject.c``. Patch by +Jeremiah Vivian. + +.. + +.. bpo: 46409 +.. date: 2022-01-17-12-57-27 +.. nonce: HouS6m +.. section: Core and Builtins + +Add new ``RETURN_GENERATOR`` bytecode to make generators. Simplifies calling +Python functions in the VM, as they no longer any need to special case +generator functions. + +Also add ``JUMP_NO_INTERRUPT`` bytecode that acts like ``JUMP_ABSOLUTE``, +but does not check for interrupts. + +.. + +.. bpo: 46406 +.. date: 2022-01-16-15-40-11 +.. nonce: g0mke- +.. section: Core and Builtins + +The integer division ``//`` implementation has been optimized to better let +the compiler understand its constraints. It can be 20% faster on the amd64 +platform when dividing an int by a value smaller than ``2**30``. + +.. + +.. bpo: 46383 +.. date: 2022-01-14-20-55-34 +.. nonce: v8MTl4 +.. section: Core and Builtins + +Fix invalid signature of ``_zoneinfo``'s ``module_free`` function to resolve +a crash on wasm32-emscripten platform. + +.. + +.. bpo: 46361 +.. date: 2022-01-12-17-15-17 +.. nonce: mgI_j_ +.. section: Core and Builtins + +Ensure that "small" integers created by :meth:`int.from_bytes` and +:class:`decimal.Decimal` are properly cached. + +.. + +.. bpo: 46161 +.. date: 2021-12-23-12-32-45 +.. nonce: EljBmu +.. section: Core and Builtins + +Fix the class building error when the arguments are constants and +CALL_FUNCTION_EX is used. + +.. + +.. bpo: 46028 +.. date: 2021-12-16-15-04-58 +.. nonce: zfWacB +.. section: Core and Builtins + +Fixes calculation of :data:`sys._base_executable` when inside a virtual +environment that uses symlinks with different binary names than the base +environment provides. + +.. + +.. bpo: 46091 +.. date: 2021-12-16-00-24-00 +.. nonce: rJ_e_e +.. section: Core and Builtins + +Correctly calculate indentation levels for lines with whitespace character +that are ended by line continuation characters. Patch by Pablo Galindo + +.. + +.. bpo: 30512 +.. date: 2021-12-12-00-49-19 +.. nonce: nU9E9V +.. section: Core and Builtins + +Add CAN Socket support for NetBSD. + +.. + +.. bpo: 46045 +.. date: 2021-12-11-11-36-48 +.. nonce: sfThay +.. section: Core and Builtins + +Do not use POSIX semaphores on NetBSD + +.. + +.. bpo: 44024 +.. date: 2021-05-04-21-55-49 +.. nonce: M9m8Qd +.. section: Core and Builtins + +Improve the exc:`TypeError` message for non-string second arguments passed +to the built-in functions :func:`getattr` and :func:`hasattr`. Patch by G?ry +Ogam. + +.. + +.. bpo: 46624 +.. date: 2022-02-03-12-07-41 +.. nonce: f_Qqh0 +.. section: Library + +Restore support for non-integer arguments of :func:`random.randrange` and +:func:`random.randint`. + +.. + +.. bpo: 46591 +.. date: 2022-01-31-15-40-38 +.. nonce: prBD1M +.. section: Library + +Make the IDLE doc URL on the About IDLE dialog clickable. + +.. + +.. bpo: 46565 +.. date: 2022-01-28-19-48-31 +.. nonce: bpZXO4 +.. section: Library + +Remove loop variables that are leaking into modules' namespaces. + +.. + +.. bpo: 46553 +.. date: 2022-01-28-08-47-53 +.. nonce: f7Uc96 +.. section: Library + +In :func:`typing.get_type_hints`, support evaluating bare stringified +``ClassVar`` annotations. Patch by Gregory Beauregard. + +.. + +.. bpo: 46544 +.. date: 2022-01-27-13-30-02 +.. nonce: oFDVWj +.. section: Library + +Don't leak ``x`` & ``uspace`` intermediate vars in +:class:`textwrap.TextWrapper`. + +.. + +.. bpo: 46487 +.. date: 2022-01-27-12-24-38 +.. nonce: UDkN2z +.. section: Library + +Add the ``get_write_buffer_limits`` method to +:class:`asyncio.transports.WriteTransport` and to the SSL transport. + +.. + +.. bpo: 45173 +.. date: 2022-01-27-11-16-59 +.. nonce: wreRF2 +.. section: Library + +Note the configparser deprecations will be removed in Python 3.12. + +.. + +.. bpo: 45162 +.. date: 2022-01-26-23-58-48 +.. nonce: 4Jmg_j +.. section: Library + +The deprecated :mod:`unittest` APIs removed in 3.11a1 have been temporarily +restored to be removed in 3.12 while cleanups in external projects go in. + +.. + +.. bpo: 46539 +.. date: 2022-01-26-20-36-30 +.. nonce: 23iW1d +.. section: Library + +In :func:`typing.get_type_hints`, support evaluating stringified +``ClassVar`` and ``Final`` annotations inside ``Annotated``. Patch by +Gregory Beauregard. + +.. + +.. bpo: 46510 +.. date: 2022-01-25-10-59-41 +.. nonce: PM5svI +.. section: Library + +Add missing test for :class:`types.TracebackType` and +:class:`types.FrameType`. Calculate them directly from the caught exception +without calling :func:`sys.exc_info`. + +.. + +.. bpo: 46491 +.. date: 2022-01-24-23-55-30 +.. nonce: jmIKHo +.. section: Library + +Allow :data:`typing.Annotated` to wrap :data:`typing.Final` and +:data:`typing.ClassVar`. Patch by Gregory Beauregard. + +.. + +.. bpo: 46483 +.. date: 2022-01-24-13-00-09 +.. nonce: 9XnmKp +.. section: Library + +Remove :meth:`~object.__class_getitem__` from :class:`pathlib.PurePath` as +this class was not supposed to be generic. + +.. + +.. bpo: 46436 +.. date: 2022-01-23-19-37-00 +.. nonce: Biz1p9 +.. section: Library + +Fix command-line option ``-d``/``--directory`` in module :mod:`http.server` +which is ignored when combined with command-line option ``--cgi``. Patch by +G?ry Ogam. + +.. + +.. bpo: 41403 +.. date: 2022-01-23-18-04-45 +.. nonce: SgoHqV +.. section: Library + +Make :meth:`mock.patch` raise a :exc:`TypeError` with a relevant error +message on invalid arg. Previously it allowed a cryptic +:exc:`AttributeError` to escape. + +.. + +.. bpo: 46474 +.. date: 2022-01-22-14-49-10 +.. nonce: eKQhvx +.. section: Library + +In ``importlib.metadata.EntryPoint.pattern``, avoid potential REDoS by +limiting ambiguity in consecutive whitespace. + +.. + +.. bpo: 46474 +.. date: 2022-01-22-14-45-46 +.. nonce: 2DUC62 +.. section: Library + +Removed private method from ``importlib.metadata.Path``. Sync with +importlib_metadata 4.10.0. + +.. + +.. bpo: 46470 +.. date: 2022-01-22-13-17-35 +.. nonce: MnNhgU +.. section: Library + +Remove unused branch from ``typing._remove_dups_flatten`` + +.. + +.. bpo: 46469 +.. date: 2022-01-22-05-05-08 +.. nonce: plUab5 +.. section: Library + +:mod:`asyncio` generic classes now return :class:`types.GenericAlias` in +``__class_getitem__`` instead of the same class. + +.. + +.. bpo: 41906 +.. date: 2022-01-21-18-19-45 +.. nonce: YBaquj +.. section: Library + +Support passing filter instances in the ``filters`` values of ``handlers`` +and ``loggers`` in the dictionary passed to +:func:`logging.config.dictConfig`. + +.. + +.. bpo: 46422 +.. date: 2022-01-20-10-35-50 +.. nonce: 1UAEHL +.. section: Library + +Use ``dis.Positions`` in ``dis.Instruction`` instead of a regular ``tuple``. + +.. + +.. bpo: 46434 +.. date: 2022-01-20-10-35-10 +.. nonce: geS-aP +.. section: Library + +:mod:`pdb` now gracefully handles ``help`` when :attr:`__doc__` is missing, +for example when run with pregenerated optimized ``.pyc`` files. + +.. + +.. bpo: 43869 +.. date: 2022-01-18-17-24-21 +.. nonce: NayN12 +.. section: Library + +Python uses the same time Epoch on all platforms. Add an explicit unit test +to ensure that it's the case. Patch by Victor Stinner. + +.. + +.. bpo: 46414 +.. date: 2022-01-17-10-00-02 +.. nonce: Ld0b_y +.. section: Library + +Add :func:`typing.reveal_type`. Patch by Jelle Zijlstra. + +.. + +.. bpo: 40280 +.. date: 2022-01-16-14-07-14 +.. nonce: LtFHfF +.. section: Library + +:mod:`subprocess` now imports Windows-specific imports when ``msvcrt`` +module is available, and POSIX-specific imports on all other platforms. This +gives a clean exception when ``_posixsubprocess`` is not available (e.g. +Emscripten browser target). + +.. + +.. bpo: 40066 +.. date: 2022-01-13-11-41-24 +.. nonce: 1QuVli +.. section: Library + +``IntEnum``, ``IntFlag``, and ``StrEnum`` use the mixed-in type for their +``str()`` and ``format()`` output. + +.. + +.. bpo: 46316 +.. date: 2022-01-09-15-04-56 +.. nonce: AMTyd0 +.. section: Library + +Optimize :meth:`pathlib.Path.iterdir` by removing an unnecessary check for +special entries. + +.. + +.. bpo: 29688 +.. date: 2022-01-05-03-21-21 +.. nonce: W06bSH +.. section: Library + +Document :meth:`pathlib.Path.absolute` (which has always existed). + +.. + +.. bpo: 43012 +.. date: 2022-01-05-03-09-29 +.. nonce: RVhLIL +.. section: Library + +The pathlib module's obsolete and internal ``_Accessor`` class has been +removed to prepare the terrain for upcoming enhancements to the module. + +.. + +.. bpo: 46258 +.. date: 2022-01-04-18-05-25 +.. nonce: DYgwRo +.. section: Library + +Speed up :func:`math.isqrt` for small positive integers by replacing two +division steps with a lookup table. + +.. + +.. bpo: 46242 +.. date: 2022-01-03-16-25-06 +.. nonce: f4l_CL +.. section: Library + +Improve error message when creating a new :class:`enum.Enum` type +subclassing an existing ``Enum`` with ``_member_names_`` using +:meth:`enum.Enum.__call__`. + +.. + +.. bpo: 43118 +.. date: 2021-12-29-14-42-09 +.. nonce: BoVi_5 +.. section: Library + +Fix a bug in :func:`inspect.signature` that was causing it to fail on some +subclasses of classes with a ``__text_signature__`` referencing module +globals. Patch by Weipeng Hong. + +.. + +.. bpo: 26552 +.. date: 2021-12-29-13-42-55 +.. nonce: 1BqeAn +.. section: Library + +Fixed case where failing :func:`asyncio.ensure_future` did not close the +coroutine. Patch by Kumar Aditya. + +.. + +.. bpo: 21987 +.. date: 2021-12-28-11-55-10 +.. nonce: avBK-p +.. section: Library + +Fix an issue with :meth:`tarfile.TarFile.getmember` getting a directory name +with a trailing slash. + +.. + +.. bpo: 46124 +.. date: 2021-12-18-18-41-30 +.. nonce: ESPrb7 +.. section: Library + +Update :mod:`zoneinfo` to rely on importlib.resources traversable API. + +.. + +.. bpo: 46103 +.. date: 2021-12-16-23-42-54 +.. nonce: LMnZAN +.. section: Library + +Now :func:`inspect.getmembers` only gets :attr:`__bases__` attribute from +class type. Patch by Weipeng Hong. + +.. + +.. bpo: 46080 +.. date: 2021-12-15-06-29-00 +.. nonce: AuQpLt +.. section: Library + +Fix exception in argparse help text generation if a +:class:`argparse.BooleanOptionalAction` argument's default is +``argparse.SUPPRESS`` and it has ``help`` specified. Patch by Felix +Fontein. + +.. + +.. bpo: 44791 +.. date: 2021-07-31-23-18-50 +.. nonce: 4jFdpO +.. section: Library + +Fix substitution of :class:`~typing.ParamSpec` in +:data:`~typing.Concatenate` with different parameter expressions. +Substitution with a list of types returns now a tuple of types. Substitution +with ``Concatenate`` returns now a ``Concatenate`` with concatenated lists +of arguments. + +.. + +.. bpo: 46463 +.. date: 2022-01-21-21-33-48 +.. nonce: fBbdTG +.. section: Documentation + +Fixes :file:`escape4chm.py` script used when building the CHM documentation +file + +.. + +.. bpo: 43478 +.. date: 2022-02-03-00-21-32 +.. nonce: 0nfcam +.. section: Tests + +Mocks can no longer be provided as the specs for other Mocks. As a result, +an already-mocked object cannot be passed to `mock.Mock()`. This can uncover +bugs in tests since these Mock-derived Mocks will always pass certain tests +(e.g. isinstance) and builtin assert functions (e.g. +assert_called_once_with) will unconditionally pass. + +.. + +.. bpo: 46616 +.. date: 2022-02-02-18-14-38 +.. nonce: URvBtE +.. section: Tests + +Ensures ``test_importlib.test_windows`` cleans up registry keys after +completion. + +.. + +.. bpo: 44359 +.. date: 2022-02-02-02-24-04 +.. nonce: kPPSmN +.. section: Tests + +test_ftplib now silently ignores socket errors to prevent logging unhandled +threading exceptions. Patch by Victor Stinner. + +.. + +.. bpo: 46600 +.. date: 2022-02-01-17-13-53 +.. nonce: FMCk8Z +.. section: Tests + +Fix test_gdb.test_pycfunction() for Python built with ``clang -Og``. +Tolerate inlined functions in the gdb traceback. Patch by Victor Stinner. + +.. + +.. bpo: 46542 +.. date: 2022-01-31-17-34-13 +.. nonce: RTMm1T +.. section: Tests + +Fix a Python crash in test_lib2to3 when using Python built in debug mode: +limit the recursion limit. Patch by Victor Stinner. + +.. + +.. bpo: 46576 +.. date: 2022-01-29-12-37-53 +.. nonce: -prRaV +.. section: Tests + +test_peg_generator now disables compiler optimization when testing +compilation of its own C extensions to significantly speed up the testing on +non-debug builds of CPython. + +.. + +.. bpo: 46542 +.. date: 2022-01-28-01-17-10 +.. nonce: xRLTdj +.. section: Tests + +Fix ``test_json`` tests checking for :exc:`RecursionError`: modify these +tests to use ``support.infinite_recursion()``. Patch by Victor Stinner. + +.. + +.. bpo: 13886 +.. date: 2022-01-17-13-10-04 +.. nonce: 5mZH4b +.. section: Tests + +Skip test_builtin PTY tests on non-ASCII characters if the readline module +is loaded. The readline module changes input() behavior, but test_builtin is +not intented to test the readline module. Patch by Victor Stinner. + +.. + +.. bpo: 40280 +.. date: 2022-01-16-14-11-57 +.. nonce: fNnFfx +.. section: Tests + +Add :func:`test.support.requires_fork` decorators to mark tests that require +a working :func:`os.fork`. + +.. + +.. bpo: 40280 +.. date: 2022-01-14-23-22-41 +.. nonce: nHLWoD +.. section: Tests + +Add :func:`test.support.requires_subprocess` decorator to mark tests which +require working :mod:`subprocess` module or ``os.spawn*``. The +wasm32-emscripten platform has no support for processes. + +.. + +.. bpo: 46126 +.. date: 2021-12-18-22-23-50 +.. nonce: 0LH3Yb +.. section: Tests + +Disable 'descriptions' when running tests internally. + +.. + +.. bpo: 46602 +.. date: 2022-02-02-02-06-07 +.. nonce: 8GaOZ2 +.. section: Build + +Tidied up configure.ac so that conftest.c is truncated rather than appended. +This assists in the case where the 'rm' of conftest.c fails to happen +between tests. Downstream issues such as a clobbered SOABI can result. + +.. + +.. bpo: 46600 +.. date: 2022-02-01-14-07-37 +.. nonce: NNLnfj +.. section: Build + +Fix the test checking if the C compiler supports ``-Og`` option in the +``./configure`` script to also use ``-Og`` on clang which supports it. Patch +by Victor Stinner. + +.. + +.. bpo: 38472 +.. date: 2022-01-26-22-59-12 +.. nonce: RxfLho +.. section: Build + +Fix GCC detection in setup.py when cross-compiling. The C compiler is now +run with LC_ALL=C. Previously, the detection failed with a German locale. + +.. + +.. bpo: 46513 +.. date: 2022-01-25-12-32-37 +.. nonce: mPm9B4 +.. section: Build + +:program:`configure` no longer uses ``AC_C_CHAR_UNSIGNED`` macro and +``pyconfig.h`` no longer defines reserved symbol ``__CHAR_UNSIGNED__``. + +.. + +.. bpo: 46471 +.. date: 2022-01-22-11-06-23 +.. nonce: 03snrE +.. section: Build + +Use global singletons for single byte bytes objects in deepfreeze. + +.. + +.. bpo: 46443 +.. date: 2022-01-20-05-27-07 +.. nonce: udCVII +.. section: Build + +Deepfreeze now uses cached small integers as it saves some space for common +small integers. + +.. + +.. bpo: 46429 +.. date: 2022-01-19-04-36-15 +.. nonce: y0OtVL +.. section: Build + +Merge all deep-frozen files into one for space savings. Patch by Kumar +Aditya. + +.. + +.. bpo: 45569 +.. date: 2022-01-09-11-24-54 +.. nonce: zCIENy +.. section: Build + +The build now defaults to using 30-bit digits for Python integers. +Previously either 15-bit or 30-bit digits would be selected, depending on +the platform. 15-bit digits may still be selected using the +``--enable-big-digits=15`` option to the ``configure`` script, or by +defining ``PYLONG_BITS_IN_DIGIT`` in ``pyconfig.h``. + +.. + +.. bpo: 45925 +.. date: 2022-01-08-12-43-31 +.. nonce: 38F3NO +.. section: Build + +Update Windows installer to use SQLite 3.37.2. + +.. + +.. bpo: 43112 +.. date: 2021-02-10-17-54-04 +.. nonce: H5Lat6 +.. section: Build + +Detect musl libc as a separate SOABI (tagged as ``linux-musl``). + +.. + +.. bpo: 33125 +.. date: 2022-01-25-14-48-39 +.. nonce: 5WyY_J +.. section: Windows + +The traditional EXE/MSI based installer for Windows is now available for +ARM64 + +.. + +.. bpo: 46362 +.. date: 2022-01-13-22-31-09 +.. nonce: f2cuEb +.. section: Windows + +os.path.abspath("C:\CON") is now fixed to return "\\.\CON", not the same +path. The regression was true of all legacy DOS devices such as COM1, LPT1, +or NUL. + +.. + +.. bpo: 44934 +.. date: 2021-09-01-10-48-11 +.. nonce: W1xPATH +.. section: Windows + +The installer now offers a command-line only option to add the installation +directory to the end of :envvar:`PATH` instead of at the start. + +.. + +.. bpo: 45925 +.. date: 2022-01-26-12-04-09 +.. nonce: yBSiYO +.. section: macOS + +Update macOS installer to SQLite 3.37.2. + +.. + +.. bpo: 45296 +.. date: 2022-01-26-19-33-55 +.. nonce: LzZKdU +.. section: IDLE + +Clarify close, quit, and exit in IDLE. In the File menu, 'Close' and 'Exit' +are now 'Close Window' (the current one) and 'Exit' is now 'Exit IDLE' (by +closing all windows). In Shell, 'quit()' and 'exit()' mean 'close Shell'. +If there are no other windows, this also exits IDLE. + +.. + +.. bpo: 40170 +.. date: 2022-01-27-02-51-22 +.. nonce: uPolek +.. section: C API + +Remove the ``PyHeapType_GET_MEMBERS()`` macro. It was exposed in the public +C API by mistake, it must only be used by Python internally. Use the +``PyTypeObject.tp_members`` member instead. Patch by Victor Stinner. + +.. + +.. bpo: 40170 +.. date: 2022-01-27-02-37-18 +.. nonce: XxQB0i +.. section: C API + +Move _Py_GetAllocatedBlocks() and _PyObject_DebugMallocStats() private +functions to the internal C API. Patch by Victor Stinner. + +.. + +.. bpo: 46433 +.. date: 2022-01-19-16-51-54 +.. nonce: Er9ApS +.. section: C API + +The internal function _PyType_GetModuleByDef now correctly handles +inheritance patterns involving static types. + +.. + +.. bpo: 45459 +.. date: 2021-10-18-16-54-24 +.. nonce: Y1pEZs +.. section: C API + +:c:type:`Py_buffer` and various ``Py_buffer`` related functions are now part +of the limited API and stable ABI. + +.. + +.. bpo: 14916 +.. date: 2020-09-11-02-50-41 +.. nonce: QN1Y03 +.. section: C API + +Fixed bug in the tokenizer that prevented ``PyRun_InteractiveOne`` from +parsing from the provided FD. diff --git a/Misc/NEWS.d/next/Build/2021-02-10-17-54-04.bpo-43112.H5Lat6.rst b/Misc/NEWS.d/next/Build/2021-02-10-17-54-04.bpo-43112.H5Lat6.rst deleted file mode 100644 index 0e250edfa3ec1..0000000000000 --- a/Misc/NEWS.d/next/Build/2021-02-10-17-54-04.bpo-43112.H5Lat6.rst +++ /dev/null @@ -1 +0,0 @@ -Detect musl libc as a separate SOABI (tagged as ``linux-musl``). \ No newline at end of file diff --git a/Misc/NEWS.d/next/Build/2022-01-08-12-43-31.bpo-45925.38F3NO.rst b/Misc/NEWS.d/next/Build/2022-01-08-12-43-31.bpo-45925.38F3NO.rst deleted file mode 100644 index e802912bfcff7..0000000000000 --- a/Misc/NEWS.d/next/Build/2022-01-08-12-43-31.bpo-45925.38F3NO.rst +++ /dev/null @@ -1 +0,0 @@ -Update Windows installer to use SQLite 3.37.2. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Build/2022-01-09-11-24-54.bpo-45569.zCIENy.rst b/Misc/NEWS.d/next/Build/2022-01-09-11-24-54.bpo-45569.zCIENy.rst deleted file mode 100644 index 69716cd9af5b2..0000000000000 --- a/Misc/NEWS.d/next/Build/2022-01-09-11-24-54.bpo-45569.zCIENy.rst +++ /dev/null @@ -1,5 +0,0 @@ -The build now defaults to using 30-bit digits for Python integers. Previously -either 15-bit or 30-bit digits would be selected, depending on the platform. -15-bit digits may still be selected using the ``--enable-big-digits=15`` option -to the ``configure`` script, or by defining ``PYLONG_BITS_IN_DIGIT`` in -``pyconfig.h``. diff --git a/Misc/NEWS.d/next/Build/2022-01-19-04-36-15.bpo-46429.y0OtVL.rst b/Misc/NEWS.d/next/Build/2022-01-19-04-36-15.bpo-46429.y0OtVL.rst deleted file mode 100644 index c983d9637fc89..0000000000000 --- a/Misc/NEWS.d/next/Build/2022-01-19-04-36-15.bpo-46429.y0OtVL.rst +++ /dev/null @@ -1 +0,0 @@ -Merge all deep-frozen files into one for space savings. Patch by Kumar Aditya. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Build/2022-01-20-05-27-07.bpo-46443.udCVII.rst b/Misc/NEWS.d/next/Build/2022-01-20-05-27-07.bpo-46443.udCVII.rst deleted file mode 100644 index 8e3fa197be9da..0000000000000 --- a/Misc/NEWS.d/next/Build/2022-01-20-05-27-07.bpo-46443.udCVII.rst +++ /dev/null @@ -1 +0,0 @@ -Deepfreeze now uses cached small integers as it saves some space for common small integers. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Build/2022-01-22-11-06-23.bpo-46471.03snrE.rst b/Misc/NEWS.d/next/Build/2022-01-22-11-06-23.bpo-46471.03snrE.rst deleted file mode 100644 index ca8f72868e69e..0000000000000 --- a/Misc/NEWS.d/next/Build/2022-01-22-11-06-23.bpo-46471.03snrE.rst +++ /dev/null @@ -1 +0,0 @@ -Use global singletons for single byte bytes objects in deepfreeze. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Build/2022-01-25-12-32-37.bpo-46513.mPm9B4.rst b/Misc/NEWS.d/next/Build/2022-01-25-12-32-37.bpo-46513.mPm9B4.rst deleted file mode 100644 index b8986ae31a340..0000000000000 --- a/Misc/NEWS.d/next/Build/2022-01-25-12-32-37.bpo-46513.mPm9B4.rst +++ /dev/null @@ -1,2 +0,0 @@ -:program:`configure` no longer uses ``AC_C_CHAR_UNSIGNED`` macro and -``pyconfig.h`` no longer defines reserved symbol ``__CHAR_UNSIGNED__``. diff --git a/Misc/NEWS.d/next/Build/2022-01-26-22-59-12.bpo-38472.RxfLho.rst b/Misc/NEWS.d/next/Build/2022-01-26-22-59-12.bpo-38472.RxfLho.rst deleted file mode 100644 index 4e0ee70bdc513..0000000000000 --- a/Misc/NEWS.d/next/Build/2022-01-26-22-59-12.bpo-38472.RxfLho.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix GCC detection in setup.py when cross-compiling. The C compiler is now -run with LC_ALL=C. Previously, the detection failed with a German locale. diff --git a/Misc/NEWS.d/next/Build/2022-02-01-14-07-37.bpo-46600.NNLnfj.rst b/Misc/NEWS.d/next/Build/2022-02-01-14-07-37.bpo-46600.NNLnfj.rst deleted file mode 100644 index 1fab655f9b271..0000000000000 --- a/Misc/NEWS.d/next/Build/2022-02-01-14-07-37.bpo-46600.NNLnfj.rst +++ /dev/null @@ -1,3 +0,0 @@ -Fix the test checking if the C compiler supports ``-Og`` option in the -``./configure`` script to also use ``-Og`` on clang which supports it. Patch -by Victor Stinner. diff --git a/Misc/NEWS.d/next/Build/2022-02-02-02-06-07.bpo-46602.8GaOZ2.rst b/Misc/NEWS.d/next/Build/2022-02-02-02-06-07.bpo-46602.8GaOZ2.rst deleted file mode 100644 index a1123b44ecd59..0000000000000 --- a/Misc/NEWS.d/next/Build/2022-02-02-02-06-07.bpo-46602.8GaOZ2.rst +++ /dev/null @@ -1 +0,0 @@ -Tidied up configure.ac so that conftest.c is truncated rather than appended. This assists in the case where the 'rm' of conftest.c fails to happen between tests. Downstream issues such as a clobbered SOABI can result. \ No newline at end of file diff --git a/Misc/NEWS.d/next/C API/2020-09-11-02-50-41.bpo-14916.QN1Y03.rst b/Misc/NEWS.d/next/C API/2020-09-11-02-50-41.bpo-14916.QN1Y03.rst deleted file mode 100644 index 885cfc53aba38..0000000000000 --- a/Misc/NEWS.d/next/C API/2020-09-11-02-50-41.bpo-14916.QN1Y03.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed bug in the tokenizer that prevented ``PyRun_InteractiveOne`` from parsing from the provided FD. diff --git a/Misc/NEWS.d/next/C API/2021-10-18-16-54-24.bpo-45459.Y1pEZs.rst b/Misc/NEWS.d/next/C API/2021-10-18-16-54-24.bpo-45459.Y1pEZs.rst deleted file mode 100644 index a8d93227817c4..0000000000000 --- a/Misc/NEWS.d/next/C API/2021-10-18-16-54-24.bpo-45459.Y1pEZs.rst +++ /dev/null @@ -1,2 +0,0 @@ -:c:type:`Py_buffer` and various ``Py_buffer`` related functions are now -part of the limited API and stable ABI. diff --git a/Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst b/Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst deleted file mode 100644 index e1987c4536b5c..0000000000000 --- a/Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst +++ /dev/null @@ -1,2 +0,0 @@ -The internal function _PyType_GetModuleByDef now correctly handles -inheritance patterns involving static types. diff --git a/Misc/NEWS.d/next/C API/2022-01-27-02-37-18.bpo-40170.XxQB0i.rst b/Misc/NEWS.d/next/C API/2022-01-27-02-37-18.bpo-40170.XxQB0i.rst deleted file mode 100644 index 7b743827bb168..0000000000000 --- a/Misc/NEWS.d/next/C API/2022-01-27-02-37-18.bpo-40170.XxQB0i.rst +++ /dev/null @@ -1,2 +0,0 @@ -Move _Py_GetAllocatedBlocks() and _PyObject_DebugMallocStats() private -functions to the internal C API. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/C API/2022-01-27-02-51-22.bpo-40170.uPolek.rst b/Misc/NEWS.d/next/C API/2022-01-27-02-51-22.bpo-40170.uPolek.rst deleted file mode 100644 index 6b185f0284e65..0000000000000 --- a/Misc/NEWS.d/next/C API/2022-01-27-02-51-22.bpo-40170.uPolek.rst +++ /dev/null @@ -1,4 +0,0 @@ -Remove the ``PyHeapType_GET_MEMBERS()`` macro. It was exposed in the public C -API by mistake, it must only be used by Python internally. Use the -``PyTypeObject.tp_members`` member instead. Patch by Victor Stinner. - diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-05-04-21-55-49.bpo-44024.M9m8Qd.rst b/Misc/NEWS.d/next/Core and Builtins/2021-05-04-21-55-49.bpo-44024.M9m8Qd.rst deleted file mode 100644 index 5037413353d28..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2021-05-04-21-55-49.bpo-44024.M9m8Qd.rst +++ /dev/null @@ -1,2 +0,0 @@ -Improve the exc:`TypeError` message for non-string second arguments passed to -the built-in functions :func:`getattr` and :func:`hasattr`. Patch by G?ry Ogam. diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-12-11-11-36-48.bpo-46045.sfThay.rst b/Misc/NEWS.d/next/Core and Builtins/2021-12-11-11-36-48.bpo-46045.sfThay.rst deleted file mode 100644 index 97fd1883eb2ab..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2021-12-11-11-36-48.bpo-46045.sfThay.rst +++ /dev/null @@ -1 +0,0 @@ -Do not use POSIX semaphores on NetBSD diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-12-12-00-49-19.bpo-30512.nU9E9V.rst b/Misc/NEWS.d/next/Core and Builtins/2021-12-12-00-49-19.bpo-30512.nU9E9V.rst deleted file mode 100644 index da2ce12fec15d..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2021-12-12-00-49-19.bpo-30512.nU9E9V.rst +++ /dev/null @@ -1 +0,0 @@ -Add CAN Socket support for NetBSD. diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-12-16-00-24-00.bpo-46091.rJ_e_e.rst b/Misc/NEWS.d/next/Core and Builtins/2021-12-16-00-24-00.bpo-46091.rJ_e_e.rst deleted file mode 100644 index a2eee0f3ebd51..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2021-12-16-00-24-00.bpo-46091.rJ_e_e.rst +++ /dev/null @@ -1,2 +0,0 @@ -Correctly calculate indentation levels for lines with whitespace character -that are ended by line continuation characters. Patch by Pablo Galindo diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-12-16-15-04-58.bpo-46028.zfWacB.rst b/Misc/NEWS.d/next/Core and Builtins/2021-12-16-15-04-58.bpo-46028.zfWacB.rst deleted file mode 100644 index cc34c0fa2405b..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2021-12-16-15-04-58.bpo-46028.zfWacB.rst +++ /dev/null @@ -1,3 +0,0 @@ -Fixes calculation of :data:`sys._base_executable` when inside a virtual -environment that uses symlinks with different binary names than the base -environment provides. diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-12-23-12-32-45.bpo-46161.EljBmu.rst b/Misc/NEWS.d/next/Core and Builtins/2021-12-23-12-32-45.bpo-46161.EljBmu.rst deleted file mode 100644 index 3eeb358c52080..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2021-12-23-12-32-45.bpo-46161.EljBmu.rst +++ /dev/null @@ -1 +0,0 @@ -Fix the class building error when the arguments are constants and CALL_FUNCTION_EX is used. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-12-17-15-17.bpo-46361.mgI_j_.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-12-17-15-17.bpo-46361.mgI_j_.rst deleted file mode 100644 index eef877d5cbd8f..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-12-17-15-17.bpo-46361.mgI_j_.rst +++ /dev/null @@ -1,2 +0,0 @@ -Ensure that "small" integers created by :meth:`int.from_bytes` and -:class:`decimal.Decimal` are properly cached. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-14-20-55-34.bpo-46383.v8MTl4.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-14-20-55-34.bpo-46383.v8MTl4.rst deleted file mode 100644 index 8f8b12732a690..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-14-20-55-34.bpo-46383.v8MTl4.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix invalid signature of ``_zoneinfo``'s ``module_free`` function to resolve -a crash on wasm32-emscripten platform. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-16-15-40-11.bpo-46406.g0mke-.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-16-15-40-11.bpo-46406.g0mke-.rst deleted file mode 100644 index 20d1e08bfd48b..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-16-15-40-11.bpo-46406.g0mke-.rst +++ /dev/null @@ -1,3 +0,0 @@ -The integer division ``//`` implementation has been optimized to better let the -compiler understand its constraints. It can be 20% faster on the amd64 platform -when dividing an int by a value smaller than ``2**30``. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-17-12-57-27.bpo-46409.HouS6m.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-17-12-57-27.bpo-46409.HouS6m.rst deleted file mode 100644 index aa61bc5201118..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-17-12-57-27.bpo-46409.HouS6m.rst +++ /dev/null @@ -1,6 +0,0 @@ -Add new ``RETURN_GENERATOR`` bytecode to make generators. -Simplifies calling Python functions in the VM, as they no -longer any need to special case generator functions. - -Also add ``JUMP_NO_INTERRUPT`` bytecode that acts like -``JUMP_ABSOLUTE``, but does not check for interrupts. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-17-23-12-01.bpo-46407.2_5a7R.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-17-23-12-01.bpo-46407.2_5a7R.rst deleted file mode 100644 index e7f555f1ffc2f..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-17-23-12-01.bpo-46407.2_5a7R.rst +++ /dev/null @@ -1 +0,0 @@ -Optimize some modulo operations in ``Objects/longobject.c``. Patch by Jeremiah Vivian. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-20-17-13-49.bpo-43683.BqQ26Z.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-20-17-13-49.bpo-43683.BqQ26Z.rst deleted file mode 100644 index 737f44f296cb1..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-20-17-13-49.bpo-43683.BqQ26Z.rst +++ /dev/null @@ -1,3 +0,0 @@ -Add ASYNC_GEN_WRAP opcode to wrap the value to be yielded in async -generators. Removes the need to special case async generators in the -``YIELD_VALUE`` instruction. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-21-12-24-14.bpo-46417.i3IqMf.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-21-12-24-14.bpo-46417.i3IqMf.rst deleted file mode 100644 index c7e2ee33500d9..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-21-12-24-14.bpo-46417.i3IqMf.rst +++ /dev/null @@ -1,3 +0,0 @@ -``python -X showrefcount`` now shows the total reference count after clearing -and destroyed the main Python interpreter. Previously, it was shown before. -Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-22-14-39-23.bpo-46417.3U5SfN.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-22-14-39-23.bpo-46417.3U5SfN.rst deleted file mode 100644 index 54fe09b7ba454..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-22-14-39-23.bpo-46417.3U5SfN.rst +++ /dev/null @@ -1,5 +0,0 @@ -Fix a race condition on setting a type ``__bases__`` attribute: the internal -function ``add_subclass()`` now gets the ``PyTypeObject.tp_subclasses`` -member after calling :c:func:`PyWeakref_NewRef` which can trigger a garbage -collection which can indirectly modify ``PyTypeObject.tp_subclasses``. Patch -by Victor Stinner. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-23-06-56-33.bpo-46481.X_FfnB.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-23-06-56-33.bpo-46481.X_FfnB.rst deleted file mode 100644 index edab2eb014430..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-23-06-56-33.bpo-46481.X_FfnB.rst +++ /dev/null @@ -1,2 +0,0 @@ -Speed up calls to :meth:`weakref.ref.__call__` by using the :pep:`590` -``vectorcall`` calling convention. Patch by Dong-hee Na. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-24-15-39-34.bpo-46476.cvP1Mr.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-24-15-39-34.bpo-46476.cvP1Mr.rst deleted file mode 100644 index 26079839a5f25..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-24-15-39-34.bpo-46476.cvP1Mr.rst +++ /dev/null @@ -1 +0,0 @@ -Fix memory leak in code objects generated by deepfreeze. Patch by Kumar Aditya. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-24-16-58-01.bpo-46431.N6mKAx.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-24-16-58-01.bpo-46431.N6mKAx.rst deleted file mode 100644 index 3a2af9df03c38..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-24-16-58-01.bpo-46431.N6mKAx.rst +++ /dev/null @@ -1 +0,0 @@ -Improve error message on invalid calls to :meth:`BaseExceptionGroup.__new__`. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-24-21-24-41.bpo-46503.4UrPsE.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-24-21-24-41.bpo-46503.4UrPsE.rst deleted file mode 100644 index e48028d72ca8e..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-24-21-24-41.bpo-46503.4UrPsE.rst +++ /dev/null @@ -1 +0,0 @@ -Fix an assert when parsing some invalid \N escape sequences in f-strings. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-25-11-44-17.bpo-46329.SEhynE.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-25-11-44-17.bpo-46329.SEhynE.rst deleted file mode 100644 index 43332975dc042..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-25-11-44-17.bpo-46329.SEhynE.rst +++ /dev/null @@ -1,12 +0,0 @@ -Use two or three bytecodes to implement most calls. - -Calls without named arguments are implemented as a sequence of two -instructions: ``PRECALL; CALL``. Calls with named arguments are implemented -as a sequence of three instructions: ``PRECALL; KW_NAMES; CALL``. There are -two different ``PRECALL`` instructions: ``PRECALL_FUNTION`` and -``PRECALL_METHOD``. The latter pairs with ``LOAD_METHOD``. - -This partition into pre-call and call allows better specialization, and thus -better performance ultimately. - -There is no change in semantics. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-25-17-40-07.bpo-46528.2Qmni9.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-25-17-40-07.bpo-46528.2Qmni9.rst deleted file mode 100644 index f1639f8b3f06e..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-25-17-40-07.bpo-46528.2Qmni9.rst +++ /dev/null @@ -1,3 +0,0 @@ -Replace several stack manipulation instructions (``DUP_TOP``, -``DUP_TOP_TWO``, ``ROT_TWO``, ``ROT_THREE``, ``ROT_FOUR``, and ``ROT_N``) -with new :opcode:`COPY` and :opcode:`SWAP` instructions. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-25-19-34-55.bpo-46527.mQLNPk.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-25-19-34-55.bpo-46527.mQLNPk.rst deleted file mode 100644 index c9fd0ed05e2ae..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-25-19-34-55.bpo-46527.mQLNPk.rst +++ /dev/null @@ -1,2 +0,0 @@ -Allow passing ``iterable`` as a keyword argument to :func:`enumerate` again. -Patch by Jelle Zijlstra. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-27-10-49-34.bpo-46458.5Gm3Gv.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-27-10-49-34.bpo-46458.5Gm3Gv.rst deleted file mode 100644 index 25f9ca3b45422..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-27-10-49-34.bpo-46458.5Gm3Gv.rst +++ /dev/null @@ -1,4 +0,0 @@ -Reorder code emitted by the compiler for a :keyword:`try`-:keyword:`except` -block so that the :keyword:`else` block's code immediately follows the -:keyword:`try` body (without a jump). This is more optimal for the happy -path. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-30-18-23-08.bpo-44977.BQV_zS.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-30-18-23-08.bpo-44977.BQV_zS.rst deleted file mode 100644 index 84c1191bdbd31..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-01-30-18-23-08.bpo-44977.BQV_zS.rst +++ /dev/null @@ -1,3 +0,0 @@ -The delegation of :func:`int` to :meth:`__trunc__` is now deprecated. -Calling ``int(a)`` when ``type(a)`` implements :meth:`__trunc__` but not -:meth:`__int__` or :meth:`__index__` now raises a :exc:`DeprecationWarning`. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-01-17-28.bpo-45885.CjyNf_.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-01-17-28.bpo-45885.CjyNf_.rst deleted file mode 100644 index 6395bd1f18d5e..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-01-17-28.bpo-45885.CjyNf_.rst +++ /dev/null @@ -1 +0,0 @@ -Added more fined-grained specialization failure stats regarding the ``COMPARE_OP`` bytecode. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-10-23-21.bpo-46564.6Xc2_H.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-10-23-21.bpo-46564.6Xc2_H.rst deleted file mode 100644 index 4ffa6800989d7..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-10-23-21.bpo-46564.6Xc2_H.rst +++ /dev/null @@ -1 +0,0 @@ -Do not create frame objects when creating :class:`super` object. Patch by Kumar Aditya. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst deleted file mode 100644 index 45da5116fc940..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2022-02-01-14-30-56.bpo-45773.Up77LD.rst +++ /dev/null @@ -1 +0,0 @@ -Remove two invalid "peephole" optimizations from the bytecode compiler. diff --git a/Misc/NEWS.d/next/Documentation/2022-01-21-21-33-48.bpo-46463.fBbdTG.rst b/Misc/NEWS.d/next/Documentation/2022-01-21-21-33-48.bpo-46463.fBbdTG.rst deleted file mode 100644 index d418190bb8fc8..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2022-01-21-21-33-48.bpo-46463.fBbdTG.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fixes :file:`escape4chm.py` script used when building the CHM documentation -file diff --git a/Misc/NEWS.d/next/IDLE/2022-01-26-19-33-55.bpo-45296.LzZKdU.rst b/Misc/NEWS.d/next/IDLE/2022-01-26-19-33-55.bpo-45296.LzZKdU.rst deleted file mode 100644 index a5b0f8b4ffa80..0000000000000 --- a/Misc/NEWS.d/next/IDLE/2022-01-26-19-33-55.bpo-45296.LzZKdU.rst +++ /dev/null @@ -1,4 +0,0 @@ -Clarify close, quit, and exit in IDLE. In the File menu, 'Close' and 'Exit' -are now 'Close Window' (the current one) and 'Exit' is now 'Exit IDLE' -(by closing all windows). In Shell, 'quit()' and 'exit()' mean 'close Shell'. -If there are no other windows, this also exits IDLE. diff --git a/Misc/NEWS.d/next/Library/2021-07-31-23-18-50.bpo-44791.4jFdpO.rst b/Misc/NEWS.d/next/Library/2021-07-31-23-18-50.bpo-44791.4jFdpO.rst deleted file mode 100644 index 8182aa4e5358a..0000000000000 --- a/Misc/NEWS.d/next/Library/2021-07-31-23-18-50.bpo-44791.4jFdpO.rst +++ /dev/null @@ -1,5 +0,0 @@ -Fix substitution of :class:`~typing.ParamSpec` in -:data:`~typing.Concatenate` with different parameter expressions. -Substitution with a list of types returns now a tuple of types. Substitution -with ``Concatenate`` returns now a ``Concatenate`` with concatenated lists -of arguments. diff --git a/Misc/NEWS.d/next/Library/2021-12-15-06-29-00.bpo-46080.AuQpLt.rst b/Misc/NEWS.d/next/Library/2021-12-15-06-29-00.bpo-46080.AuQpLt.rst deleted file mode 100644 index e42d84e31e759..0000000000000 --- a/Misc/NEWS.d/next/Library/2021-12-15-06-29-00.bpo-46080.AuQpLt.rst +++ /dev/null @@ -1,3 +0,0 @@ -Fix exception in argparse help text generation if a -:class:`argparse.BooleanOptionalAction` argument's default is -``argparse.SUPPRESS`` and it has ``help`` specified. Patch by Felix Fontein. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2021-12-16-23-42-54.bpo-46103.LMnZAN.rst b/Misc/NEWS.d/next/Library/2021-12-16-23-42-54.bpo-46103.LMnZAN.rst deleted file mode 100644 index 3becbc3de8fc2..0000000000000 --- a/Misc/NEWS.d/next/Library/2021-12-16-23-42-54.bpo-46103.LMnZAN.rst +++ /dev/null @@ -1,2 +0,0 @@ -Now :func:`inspect.getmembers` only gets :attr:`__bases__` attribute from -class type. Patch by Weipeng Hong. diff --git a/Misc/NEWS.d/next/Library/2021-12-18-18-41-30.bpo-46124.ESPrb7.rst b/Misc/NEWS.d/next/Library/2021-12-18-18-41-30.bpo-46124.ESPrb7.rst deleted file mode 100644 index 26f9f81303a96..0000000000000 --- a/Misc/NEWS.d/next/Library/2021-12-18-18-41-30.bpo-46124.ESPrb7.rst +++ /dev/null @@ -1 +0,0 @@ -Update :mod:`zoneinfo` to rely on importlib.resources traversable API. diff --git a/Misc/NEWS.d/next/Library/2021-12-28-11-55-10.bpo-21987.avBK-p.rst b/Misc/NEWS.d/next/Library/2021-12-28-11-55-10.bpo-21987.avBK-p.rst deleted file mode 100644 index 305dd16d53b49..0000000000000 --- a/Misc/NEWS.d/next/Library/2021-12-28-11-55-10.bpo-21987.avBK-p.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix an issue with :meth:`tarfile.TarFile.getmember` getting a directory name -with a trailing slash. diff --git a/Misc/NEWS.d/next/Library/2021-12-29-13-42-55.bpo-26552.1BqeAn.rst b/Misc/NEWS.d/next/Library/2021-12-29-13-42-55.bpo-26552.1BqeAn.rst deleted file mode 100644 index 85b6a64ef53d1..0000000000000 --- a/Misc/NEWS.d/next/Library/2021-12-29-13-42-55.bpo-26552.1BqeAn.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed case where failing :func:`asyncio.ensure_future` did not close the coroutine. Patch by Kumar Aditya. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2021-12-29-14-42-09.bpo-43118.BoVi_5.rst b/Misc/NEWS.d/next/Library/2021-12-29-14-42-09.bpo-43118.BoVi_5.rst deleted file mode 100644 index a37c22cd78c09..0000000000000 --- a/Misc/NEWS.d/next/Library/2021-12-29-14-42-09.bpo-43118.BoVi_5.rst +++ /dev/null @@ -1,3 +0,0 @@ -Fix a bug in :func:`inspect.signature` that was causing it to fail on some -subclasses of classes with a ``__text_signature__`` referencing module -globals. Patch by Weipeng Hong. diff --git a/Misc/NEWS.d/next/Library/2022-01-03-16-25-06.bpo-46242.f4l_CL.rst b/Misc/NEWS.d/next/Library/2022-01-03-16-25-06.bpo-46242.f4l_CL.rst deleted file mode 100644 index 6a5b5fdffda40..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-03-16-25-06.bpo-46242.f4l_CL.rst +++ /dev/null @@ -1 +0,0 @@ -Improve error message when creating a new :class:`enum.Enum` type subclassing an existing ``Enum`` with ``_member_names_`` using :meth:`enum.Enum.__call__`. diff --git a/Misc/NEWS.d/next/Library/2022-01-04-18-05-25.bpo-46258.DYgwRo.rst b/Misc/NEWS.d/next/Library/2022-01-04-18-05-25.bpo-46258.DYgwRo.rst deleted file mode 100644 index b918ed1a5d9e0..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-04-18-05-25.bpo-46258.DYgwRo.rst +++ /dev/null @@ -1,2 +0,0 @@ -Speed up :func:`math.isqrt` for small positive integers by replacing two -division steps with a lookup table. diff --git a/Misc/NEWS.d/next/Library/2022-01-05-03-09-29.bpo-43012.RVhLIL.rst b/Misc/NEWS.d/next/Library/2022-01-05-03-09-29.bpo-43012.RVhLIL.rst deleted file mode 100644 index c6a6723a96b34..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-05-03-09-29.bpo-43012.RVhLIL.rst +++ /dev/null @@ -1,2 +0,0 @@ -The pathlib module's obsolete and internal ``_Accessor`` class has been -removed to prepare the terrain for upcoming enhancements to the module. diff --git a/Misc/NEWS.d/next/Library/2022-01-05-03-21-21.bpo-29688.W06bSH.rst b/Misc/NEWS.d/next/Library/2022-01-05-03-21-21.bpo-29688.W06bSH.rst deleted file mode 100644 index 1a202e59b075e..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-05-03-21-21.bpo-29688.W06bSH.rst +++ /dev/null @@ -1 +0,0 @@ -Document :meth:`pathlib.Path.absolute` (which has always existed). diff --git a/Misc/NEWS.d/next/Library/2022-01-09-15-04-56.bpo-46316.AMTyd0.rst b/Misc/NEWS.d/next/Library/2022-01-09-15-04-56.bpo-46316.AMTyd0.rst deleted file mode 100644 index 09acb77855f15..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-09-15-04-56.bpo-46316.AMTyd0.rst +++ /dev/null @@ -1 +0,0 @@ -Optimize :meth:`pathlib.Path.iterdir` by removing an unnecessary check for special entries. diff --git a/Misc/NEWS.d/next/Library/2022-01-13-11-41-24.bpo-40066.1QuVli.rst b/Misc/NEWS.d/next/Library/2022-01-13-11-41-24.bpo-40066.1QuVli.rst deleted file mode 100644 index 2df487855785e..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-13-11-41-24.bpo-40066.1QuVli.rst +++ /dev/null @@ -1,2 +0,0 @@ -``IntEnum``, ``IntFlag``, and ``StrEnum`` use the mixed-in type for their -``str()`` and ``format()`` output. diff --git a/Misc/NEWS.d/next/Library/2022-01-16-14-07-14.bpo-40280.LtFHfF.rst b/Misc/NEWS.d/next/Library/2022-01-16-14-07-14.bpo-40280.LtFHfF.rst deleted file mode 100644 index f5d76760678f6..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-16-14-07-14.bpo-40280.LtFHfF.rst +++ /dev/null @@ -1,4 +0,0 @@ -:mod:`subprocess` now imports Windows-specific imports when -``msvcrt`` module is available, and POSIX-specific imports on all other -platforms. This gives a clean exception when ``_posixsubprocess`` is not -available (e.g. Emscripten browser target). diff --git a/Misc/NEWS.d/next/Library/2022-01-17-10-00-02.bpo-46414.Ld0b_y.rst b/Misc/NEWS.d/next/Library/2022-01-17-10-00-02.bpo-46414.Ld0b_y.rst deleted file mode 100644 index 0fdbfa77d9bc8..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-17-10-00-02.bpo-46414.Ld0b_y.rst +++ /dev/null @@ -1 +0,0 @@ -Add :func:`typing.reveal_type`. Patch by Jelle Zijlstra. diff --git a/Misc/NEWS.d/next/Library/2022-01-18-17-24-21.bpo-43869.NayN12.rst b/Misc/NEWS.d/next/Library/2022-01-18-17-24-21.bpo-43869.NayN12.rst deleted file mode 100644 index 5486c95b0689b..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-18-17-24-21.bpo-43869.NayN12.rst +++ /dev/null @@ -1,2 +0,0 @@ -Python uses the same time Epoch on all platforms. Add an explicit unit test -to ensure that it's the case. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Library/2022-01-20-10-35-10.bpo-46434.geS-aP.rst b/Misc/NEWS.d/next/Library/2022-01-20-10-35-10.bpo-46434.geS-aP.rst deleted file mode 100644 index 6000781fa5aea..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-20-10-35-10.bpo-46434.geS-aP.rst +++ /dev/null @@ -1,2 +0,0 @@ -:mod:`pdb` now gracefully handles ``help`` when :attr:`__doc__` is missing, -for example when run with pregenerated optimized ``.pyc`` files. diff --git a/Misc/NEWS.d/next/Library/2022-01-20-10-35-50.bpo-46422.1UAEHL.rst b/Misc/NEWS.d/next/Library/2022-01-20-10-35-50.bpo-46422.1UAEHL.rst deleted file mode 100644 index 831f526359062..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-20-10-35-50.bpo-46422.1UAEHL.rst +++ /dev/null @@ -1 +0,0 @@ -Use ``dis.Positions`` in ``dis.Instruction`` instead of a regular ``tuple``. diff --git a/Misc/NEWS.d/next/Library/2022-01-21-18-19-45.bpo-41906.YBaquj.rst b/Misc/NEWS.d/next/Library/2022-01-21-18-19-45.bpo-41906.YBaquj.rst deleted file mode 100644 index be707130875f2..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-21-18-19-45.bpo-41906.YBaquj.rst +++ /dev/null @@ -1,2 +0,0 @@ -Support passing filter instances in the ``filters`` values of ``handlers`` and -``loggers`` in the dictionary passed to :func:`logging.config.dictConfig`. diff --git a/Misc/NEWS.d/next/Library/2022-01-22-05-05-08.bpo-46469.plUab5.rst b/Misc/NEWS.d/next/Library/2022-01-22-05-05-08.bpo-46469.plUab5.rst deleted file mode 100644 index 0d0e4b5d3d735..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-22-05-05-08.bpo-46469.plUab5.rst +++ /dev/null @@ -1 +0,0 @@ -:mod:`asyncio` generic classes now return :class:`types.GenericAlias` in ``__class_getitem__`` instead of the same class. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2022-01-22-13-17-35.bpo-46470.MnNhgU.rst b/Misc/NEWS.d/next/Library/2022-01-22-13-17-35.bpo-46470.MnNhgU.rst deleted file mode 100644 index 45b9cea3cd56a..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-22-13-17-35.bpo-46470.MnNhgU.rst +++ /dev/null @@ -1 +0,0 @@ -Remove unused branch from ``typing._remove_dups_flatten`` diff --git a/Misc/NEWS.d/next/Library/2022-01-22-14-45-46.bpo-46474.2DUC62.rst b/Misc/NEWS.d/next/Library/2022-01-22-14-45-46.bpo-46474.2DUC62.rst deleted file mode 100644 index a5eafdf30f148..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-22-14-45-46.bpo-46474.2DUC62.rst +++ /dev/null @@ -1,2 +0,0 @@ -Removed private method from ``importlib.metadata.Path``. Sync with -importlib_metadata 4.10.0. diff --git a/Misc/NEWS.d/next/Library/2022-01-22-14-49-10.bpo-46474.eKQhvx.rst b/Misc/NEWS.d/next/Library/2022-01-22-14-49-10.bpo-46474.eKQhvx.rst deleted file mode 100644 index 156b7de4f6787..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-22-14-49-10.bpo-46474.eKQhvx.rst +++ /dev/null @@ -1,2 +0,0 @@ -In ``importlib.metadata.EntryPoint.pattern``, avoid potential REDoS by -limiting ambiguity in consecutive whitespace. diff --git a/Misc/NEWS.d/next/Library/2022-01-23-18-04-45.bpo-41403.SgoHqV.rst b/Misc/NEWS.d/next/Library/2022-01-23-18-04-45.bpo-41403.SgoHqV.rst deleted file mode 100644 index ede159b25641f..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-23-18-04-45.bpo-41403.SgoHqV.rst +++ /dev/null @@ -1,3 +0,0 @@ -Make :meth:`mock.patch` raise a :exc:`TypeError` with a relevant error -message on invalid arg. Previously it allowed a cryptic -:exc:`AttributeError` to escape. diff --git a/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst b/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst deleted file mode 100644 index ccfd949506443..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst +++ /dev/null @@ -1,3 +0,0 @@ -Fix command-line option ``-d``/``--directory`` in module :mod:`http.server` -which is ignored when combined with command-line option ``--cgi``. Patch by -G?ry Ogam. diff --git a/Misc/NEWS.d/next/Library/2022-01-24-13-00-09.bpo-46483.9XnmKp.rst b/Misc/NEWS.d/next/Library/2022-01-24-13-00-09.bpo-46483.9XnmKp.rst deleted file mode 100644 index 89cc818a9c59d..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-24-13-00-09.bpo-46483.9XnmKp.rst +++ /dev/null @@ -1 +0,0 @@ -Remove :meth:`~object.__class_getitem__` from :class:`pathlib.PurePath` as this class was not supposed to be generic. diff --git a/Misc/NEWS.d/next/Library/2022-01-24-23-55-30.bpo-46491.jmIKHo.rst b/Misc/NEWS.d/next/Library/2022-01-24-23-55-30.bpo-46491.jmIKHo.rst deleted file mode 100644 index f66e8868f753f..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-24-23-55-30.bpo-46491.jmIKHo.rst +++ /dev/null @@ -1 +0,0 @@ -Allow :data:`typing.Annotated` to wrap :data:`typing.Final` and :data:`typing.ClassVar`. Patch by Gregory Beauregard. diff --git a/Misc/NEWS.d/next/Library/2022-01-25-10-59-41.bpo-46510.PM5svI.rst b/Misc/NEWS.d/next/Library/2022-01-25-10-59-41.bpo-46510.PM5svI.rst deleted file mode 100644 index b416a1692270b..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-25-10-59-41.bpo-46510.PM5svI.rst +++ /dev/null @@ -1,3 +0,0 @@ -Add missing test for :class:`types.TracebackType` and -:class:`types.FrameType`. Calculate them directly from the caught exception -without calling :func:`sys.exc_info`. diff --git a/Misc/NEWS.d/next/Library/2022-01-26-20-36-30.bpo-46539.23iW1d.rst b/Misc/NEWS.d/next/Library/2022-01-26-20-36-30.bpo-46539.23iW1d.rst deleted file mode 100644 index 2bdde21b6e58e..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-26-20-36-30.bpo-46539.23iW1d.rst +++ /dev/null @@ -1 +0,0 @@ -In :func:`typing.get_type_hints`, support evaluating stringified ``ClassVar`` and ``Final`` annotations inside ``Annotated``. Patch by Gregory Beauregard. diff --git a/Misc/NEWS.d/next/Library/2022-01-26-23-58-48.bpo-45162.4Jmg_j.rst b/Misc/NEWS.d/next/Library/2022-01-26-23-58-48.bpo-45162.4Jmg_j.rst deleted file mode 100644 index 29fee1ef9ce59..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-26-23-58-48.bpo-45162.4Jmg_j.rst +++ /dev/null @@ -1,3 +0,0 @@ -The deprecated :mod:`unittest` APIs removed in 3.11a1 have been -temporarily restored to be removed in 3.12 while cleanups in external -projects go in. diff --git a/Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst b/Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst deleted file mode 100644 index ee5a88f621498..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-27-11-16-59.bpo-45173.wreRF2.rst +++ /dev/null @@ -1 +0,0 @@ -Note the configparser deprecations will be removed in Python 3.12. diff --git a/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst b/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst deleted file mode 100644 index adbc50a4bf9b7..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-27-12-24-38.bpo-46487.UDkN2z.rst +++ /dev/null @@ -1 +0,0 @@ -Add the ``get_write_buffer_limits`` method to :class:`asyncio.transports.WriteTransport` and to the SSL transport. diff --git a/Misc/NEWS.d/next/Library/2022-01-27-13-30-02.bpo-46544.oFDVWj.rst b/Misc/NEWS.d/next/Library/2022-01-27-13-30-02.bpo-46544.oFDVWj.rst deleted file mode 100644 index 63b47e55f1b18..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-27-13-30-02.bpo-46544.oFDVWj.rst +++ /dev/null @@ -1,2 +0,0 @@ -Don't leak ``x`` & ``uspace`` intermediate vars in -:class:`textwrap.TextWrapper`. diff --git a/Misc/NEWS.d/next/Library/2022-01-28-08-47-53.bpo-46553.f7Uc96.rst b/Misc/NEWS.d/next/Library/2022-01-28-08-47-53.bpo-46553.f7Uc96.rst deleted file mode 100644 index 2c03912afcb45..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-28-08-47-53.bpo-46553.f7Uc96.rst +++ /dev/null @@ -1 +0,0 @@ -In :func:`typing.get_type_hints`, support evaluating bare stringified ``ClassVar`` annotations. Patch by Gregory Beauregard. diff --git a/Misc/NEWS.d/next/Library/2022-01-28-19-48-31.bpo-46565.bpZXO4.rst b/Misc/NEWS.d/next/Library/2022-01-28-19-48-31.bpo-46565.bpZXO4.rst deleted file mode 100644 index 9b0969ea5897b..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-28-19-48-31.bpo-46565.bpZXO4.rst +++ /dev/null @@ -1 +0,0 @@ -Remove loop variables that are leaking into modules' namespaces. diff --git a/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst b/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst deleted file mode 100644 index 7785faa1c4cbf..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-01-31-15-40-38.bpo-46591.prBD1M.rst +++ /dev/null @@ -1 +0,0 @@ -Make the IDLE doc URL on the About IDLE dialog clickable. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2022-02-03-12-07-41.bpo-46624.f_Qqh0.rst b/Misc/NEWS.d/next/Library/2022-02-03-12-07-41.bpo-46624.f_Qqh0.rst deleted file mode 100644 index b0203b9a8b50c..0000000000000 --- a/Misc/NEWS.d/next/Library/2022-02-03-12-07-41.bpo-46624.f_Qqh0.rst +++ /dev/null @@ -1 +0,0 @@ -Restore support for non-integer arguments of :func:`random.randrange` and :func:`random.randint`. diff --git a/Misc/NEWS.d/next/Tests/2021-12-18-22-23-50.bpo-46126.0LH3Yb.rst b/Misc/NEWS.d/next/Tests/2021-12-18-22-23-50.bpo-46126.0LH3Yb.rst deleted file mode 100644 index b7360b36454ea..0000000000000 --- a/Misc/NEWS.d/next/Tests/2021-12-18-22-23-50.bpo-46126.0LH3Yb.rst +++ /dev/null @@ -1 +0,0 @@ -Disable 'descriptions' when running tests internally. diff --git a/Misc/NEWS.d/next/Tests/2022-01-14-23-22-41.bpo-40280.nHLWoD.rst b/Misc/NEWS.d/next/Tests/2022-01-14-23-22-41.bpo-40280.nHLWoD.rst deleted file mode 100644 index 67134f1191cd2..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-01-14-23-22-41.bpo-40280.nHLWoD.rst +++ /dev/null @@ -1,3 +0,0 @@ -Add :func:`test.support.requires_subprocess` decorator to mark tests which -require working :mod:`subprocess` module or ``os.spawn*``. The -wasm32-emscripten platform has no support for processes. diff --git a/Misc/NEWS.d/next/Tests/2022-01-16-14-11-57.bpo-40280.fNnFfx.rst b/Misc/NEWS.d/next/Tests/2022-01-16-14-11-57.bpo-40280.fNnFfx.rst deleted file mode 100644 index 2d66db1210854..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-01-16-14-11-57.bpo-40280.fNnFfx.rst +++ /dev/null @@ -1,2 +0,0 @@ -Add :func:`test.support.requires_fork` decorators to mark tests that require -a working :func:`os.fork`. diff --git a/Misc/NEWS.d/next/Tests/2022-01-17-13-10-04.bpo-13886.5mZH4b.rst b/Misc/NEWS.d/next/Tests/2022-01-17-13-10-04.bpo-13886.5mZH4b.rst deleted file mode 100644 index cd19dce37d5c8..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-01-17-13-10-04.bpo-13886.5mZH4b.rst +++ /dev/null @@ -1,3 +0,0 @@ -Skip test_builtin PTY tests on non-ASCII characters if the readline module -is loaded. The readline module changes input() behavior, but test_builtin is -not intented to test the readline module. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Tests/2022-01-28-01-17-10.bpo-46542.xRLTdj.rst b/Misc/NEWS.d/next/Tests/2022-01-28-01-17-10.bpo-46542.xRLTdj.rst deleted file mode 100644 index c6b64ce017b6c..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-01-28-01-17-10.bpo-46542.xRLTdj.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix ``test_json`` tests checking for :exc:`RecursionError`: modify these tests -to use ``support.infinite_recursion()``. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst b/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst deleted file mode 100644 index be50fc8cbe0a5..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-01-29-12-37-53.bpo-46576.-prRaV.rst +++ /dev/null @@ -1,3 +0,0 @@ -test_peg_generator now disables compiler optimization when testing -compilation of its own C extensions to significantly speed up the -testing on non-debug builds of CPython. diff --git a/Misc/NEWS.d/next/Tests/2022-01-31-17-34-13.bpo-46542.RTMm1T.rst b/Misc/NEWS.d/next/Tests/2022-01-31-17-34-13.bpo-46542.RTMm1T.rst deleted file mode 100644 index 5596498724930..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-01-31-17-34-13.bpo-46542.RTMm1T.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix a Python crash in test_lib2to3 when using Python built in debug mode: -limit the recursion limit. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Tests/2022-02-01-17-13-53.bpo-46600.FMCk8Z.rst b/Misc/NEWS.d/next/Tests/2022-02-01-17-13-53.bpo-46600.FMCk8Z.rst deleted file mode 100644 index 0ae1d4d181d42..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-02-01-17-13-53.bpo-46600.FMCk8Z.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix test_gdb.test_pycfunction() for Python built with ``clang -Og``. -Tolerate inlined functions in the gdb traceback. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst b/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst deleted file mode 100644 index 00c29b1ca4089..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-02-02-02-24-04.bpo-44359.kPPSmN.rst +++ /dev/null @@ -1,2 +0,0 @@ -test_ftplib now silently ignores socket errors to prevent logging unhandled -threading exceptions. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst b/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst deleted file mode 100644 index 31c63c3d8f181..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-02-02-18-14-38.bpo-46616.URvBtE.rst +++ /dev/null @@ -1,2 +0,0 @@ -Ensures ``test_importlib.test_windows`` cleans up registry keys after -completion. diff --git a/Misc/NEWS.d/next/Tests/2022-02-03-00-21-32.bpo-43478.0nfcam.rst b/Misc/NEWS.d/next/Tests/2022-02-03-00-21-32.bpo-43478.0nfcam.rst deleted file mode 100644 index 7c8fc47cfc751..0000000000000 --- a/Misc/NEWS.d/next/Tests/2022-02-03-00-21-32.bpo-43478.0nfcam.rst +++ /dev/null @@ -1 +0,0 @@ -Mocks can no longer be provided as the specs for other Mocks. As a result, an already-mocked object cannot be passed to `mock.Mock()`. This can uncover bugs in tests since these Mock-derived Mocks will always pass certain tests (e.g. isinstance) and builtin assert functions (e.g. assert_called_once_with) will unconditionally pass. diff --git a/Misc/NEWS.d/next/Windows/2021-09-01-10-48-11.bpo-44934.W1xPATH.rst b/Misc/NEWS.d/next/Windows/2021-09-01-10-48-11.bpo-44934.W1xPATH.rst deleted file mode 100644 index 0f1c25a0705df..0000000000000 --- a/Misc/NEWS.d/next/Windows/2021-09-01-10-48-11.bpo-44934.W1xPATH.rst +++ /dev/null @@ -1 +0,0 @@ -The installer now offers a command-line only option to add the installation directory to the end of :envvar:`PATH` instead of at the start. diff --git a/Misc/NEWS.d/next/Windows/2022-01-13-22-31-09.bpo-46362.f2cuEb.rst b/Misc/NEWS.d/next/Windows/2022-01-13-22-31-09.bpo-46362.f2cuEb.rst deleted file mode 100644 index 0b59cd28ba4fd..0000000000000 --- a/Misc/NEWS.d/next/Windows/2022-01-13-22-31-09.bpo-46362.f2cuEb.rst +++ /dev/null @@ -1,2 +0,0 @@ -os.path.abspath("C:\CON") is now fixed to return "\\.\CON", not the same path. -The regression was true of all legacy DOS devices such as COM1, LPT1, or NUL. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Windows/2022-01-25-14-48-39.bpo-33125.5WyY_J.rst b/Misc/NEWS.d/next/Windows/2022-01-25-14-48-39.bpo-33125.5WyY_J.rst deleted file mode 100644 index 54811db67add8..0000000000000 --- a/Misc/NEWS.d/next/Windows/2022-01-25-14-48-39.bpo-33125.5WyY_J.rst +++ /dev/null @@ -1,2 +0,0 @@ -The traditional EXE/MSI based installer for Windows is now available for -ARM64 diff --git a/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst b/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst deleted file mode 100644 index 3705266c154b7..0000000000000 --- a/Misc/NEWS.d/next/macOS/2022-01-26-12-04-09.bpo-45925.yBSiYO.rst +++ /dev/null @@ -1 +0,0 @@ -Update macOS installer to SQLite 3.37.2. \ No newline at end of file diff --git a/README.rst b/README.rst index dcc1c39d78a08..36611add5ccac 100644 --- a/README.rst +++ b/README.rst @@ -1,4 +1,4 @@ -This is Python version 3.11.0 alpha 4 +This is Python version 3.11.0 alpha 5 ===================================== .. image:: https://github.com/python/cpython/workflows/Tests/badge.svg From webhook-mailer at python.org Thu Feb 3 23:40:03 2022 From: webhook-mailer at python.org (terryjreedy) Date: Fri, 04 Feb 2022 04:40:03 -0000 Subject: [Python-checkins] bpo-46588: fix typo in test_calltip.py (GH-31119) Message-ID: https://github.com/python/cpython/commit/222865daabfa7a8b12ca9a5e9c23b9ce217448f1 commit: 222865daabfa7a8b12ca9a5e9c23b9ce217448f1 branch: main author: Caio Agiani committer: terryjreedy date: 2022-02-03T23:39:59-05:00 summary: bpo-46588: fix typo in test_calltip.py (GH-31119) files: M Lib/idlelib/idle_test/test_calltip.py diff --git a/Lib/idlelib/idle_test/test_calltip.py b/Lib/idlelib/idle_test/test_calltip.py index b23915c5ab784..e8d2bd17cb681 100644 --- a/Lib/idlelib/idle_test/test_calltip.py +++ b/Lib/idlelib/idle_test/test_calltip.py @@ -106,7 +106,7 @@ def test_signature_wrap(self): If you want to completely replace the main wrapping algorithm, you\'ll probably have to override _wrap_chunks().''') - def test_properly_formated(self): + def test_properly_formatted(self): def foo(s='a'*100): pass From webhook-mailer at python.org Fri Feb 4 00:05:48 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 04 Feb 2022 05:05:48 -0000 Subject: [Python-checkins] bpo-46588: fix typo in test_calltip.py (GH-31119) Message-ID: https://github.com/python/cpython/commit/663370aea5dcf9074455b45283e980c7bc353674 commit: 663370aea5dcf9074455b45283e980c7bc353674 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-03T21:05:42-08:00 summary: bpo-46588: fix typo in test_calltip.py (GH-31119) (cherry picked from commit 222865daabfa7a8b12ca9a5e9c23b9ce217448f1) Co-authored-by: Caio Agiani files: M Lib/idlelib/idle_test/test_calltip.py diff --git a/Lib/idlelib/idle_test/test_calltip.py b/Lib/idlelib/idle_test/test_calltip.py index b23915c5ab784..e8d2bd17cb681 100644 --- a/Lib/idlelib/idle_test/test_calltip.py +++ b/Lib/idlelib/idle_test/test_calltip.py @@ -106,7 +106,7 @@ def test_signature_wrap(self): If you want to completely replace the main wrapping algorithm, you\'ll probably have to override _wrap_chunks().''') - def test_properly_formated(self): + def test_properly_formatted(self): def foo(s='a'*100): pass From webhook-mailer at python.org Fri Feb 4 00:11:23 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 04 Feb 2022 05:11:23 -0000 Subject: [Python-checkins] bpo-46588: fix typo in test_calltip.py (GH-31119) Message-ID: https://github.com/python/cpython/commit/9ce0b00fb1172e743de985eda28e3335aa95014a commit: 9ce0b00fb1172e743de985eda28e3335aa95014a branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-03T21:11:15-08:00 summary: bpo-46588: fix typo in test_calltip.py (GH-31119) (cherry picked from commit 222865daabfa7a8b12ca9a5e9c23b9ce217448f1) Co-authored-by: Caio Agiani files: M Lib/idlelib/idle_test/test_calltip.py diff --git a/Lib/idlelib/idle_test/test_calltip.py b/Lib/idlelib/idle_test/test_calltip.py index b23915c5ab784..e8d2bd17cb681 100644 --- a/Lib/idlelib/idle_test/test_calltip.py +++ b/Lib/idlelib/idle_test/test_calltip.py @@ -106,7 +106,7 @@ def test_signature_wrap(self): If you want to completely replace the main wrapping algorithm, you\'ll probably have to override _wrap_chunks().''') - def test_properly_formated(self): + def test_properly_formatted(self): def foo(s='a'*100): pass From webhook-mailer at python.org Fri Feb 4 01:49:53 2022 From: webhook-mailer at python.org (methane) Date: Fri, 04 Feb 2022 06:49:53 -0000 Subject: [Python-checkins] Optimize images by IMGbot (GH-21348) Message-ID: https://github.com/python/cpython/commit/ba650af7d660084e08859dd1ee1917cccee24e88 commit: ba650af7d660084e08859dd1ee1917cccee24e88 branch: main author: Manish Kumar ? committer: methane date: 2022-02-04T15:49:43+09:00 summary: Optimize images by IMGbot (GH-21348) Co-authored-by: ImgBotApp files: M Doc/howto/logging_flow.png M Doc/library/hashlib-blake2-tree.png M Doc/library/pathlib-inheritance.svg M Doc/library/tk_msg.png M Doc/using/win_installer.png M Lib/idlelib/Icons/idle_16.gif M Lib/idlelib/Icons/idle_16.png M Lib/idlelib/Icons/idle_256.png M Lib/idlelib/Icons/idle_32.gif M Lib/idlelib/Icons/idle_32.png M Lib/idlelib/Icons/idle_48.png M Lib/idlelib/Icons/minusnode.gif M Lib/idlelib/Icons/plusnode.gif M Lib/idlelib/Icons/python.gif M Lib/idlelib/Icons/tk.gif M Lib/test/imghdrdata/python.gif M Lib/test/test_email/data/PyBanner048.gif M PC/icons/logox128.png M PC/icons/py.png M PC/icons/pythonwx150.png M PC/icons/pythonwx44.png M PC/icons/pythonx150.png M PC/icons/pythonx44.png M PC/icons/pythonx50.png M Tools/msi/bundle/SideBar.png diff --git a/Doc/howto/logging_flow.png b/Doc/howto/logging_flow.png index fac4acd775530..d65e597f811db 100644 Binary files a/Doc/howto/logging_flow.png and b/Doc/howto/logging_flow.png differ diff --git a/Doc/library/hashlib-blake2-tree.png b/Doc/library/hashlib-blake2-tree.png index 73e849444ed7d..faef21b55f96b 100644 Binary files a/Doc/library/hashlib-blake2-tree.png and b/Doc/library/hashlib-blake2-tree.png differ diff --git a/Doc/library/pathlib-inheritance.svg b/Doc/library/pathlib-inheritance.svg index 49057f678fd7d..01f8684a2b201 100644 --- a/Doc/library/pathlib-inheritance.svg +++ b/Doc/library/pathlib-inheritance.svg @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file diff --git a/Doc/library/tk_msg.png b/Doc/library/tk_msg.png index c122d8f8ae5ba..6495e63e84650 100644 Binary files a/Doc/library/tk_msg.png and b/Doc/library/tk_msg.png differ diff --git a/Doc/using/win_installer.png b/Doc/using/win_installer.png index 9c18ff19cf5e8..03bf2d7b16c59 100644 Binary files a/Doc/using/win_installer.png and b/Doc/using/win_installer.png differ diff --git a/Lib/idlelib/Icons/idle_16.gif b/Lib/idlelib/Icons/idle_16.gif index 9f001b1d79cf6..bcedcd6400390 100644 Binary files a/Lib/idlelib/Icons/idle_16.gif and b/Lib/idlelib/Icons/idle_16.gif differ diff --git a/Lib/idlelib/Icons/idle_16.png b/Lib/idlelib/Icons/idle_16.png index 6abde0af90cb6..77d8f711804f9 100644 Binary files a/Lib/idlelib/Icons/idle_16.png and b/Lib/idlelib/Icons/idle_16.png differ diff --git a/Lib/idlelib/Icons/idle_256.png b/Lib/idlelib/Icons/idle_256.png index 99ffa6fad4a92..64f276b40d23b 100644 Binary files a/Lib/idlelib/Icons/idle_256.png and b/Lib/idlelib/Icons/idle_256.png differ diff --git a/Lib/idlelib/Icons/idle_32.gif b/Lib/idlelib/Icons/idle_32.gif index af5b2d52cce89..a1b12a0531d64 100644 Binary files a/Lib/idlelib/Icons/idle_32.gif and b/Lib/idlelib/Icons/idle_32.gif differ diff --git a/Lib/idlelib/Icons/idle_32.png b/Lib/idlelib/Icons/idle_32.png index 41b70dbc37766..2aaa55805ed53 100644 Binary files a/Lib/idlelib/Icons/idle_32.png and b/Lib/idlelib/Icons/idle_32.png differ diff --git a/Lib/idlelib/Icons/idle_48.png b/Lib/idlelib/Icons/idle_48.png index e5fa9280e21f8..705eec42e8a64 100644 Binary files a/Lib/idlelib/Icons/idle_48.png and b/Lib/idlelib/Icons/idle_48.png differ diff --git a/Lib/idlelib/Icons/minusnode.gif b/Lib/idlelib/Icons/minusnode.gif index c72e46ff863e4..173e97095913a 100644 Binary files a/Lib/idlelib/Icons/minusnode.gif and b/Lib/idlelib/Icons/minusnode.gif differ diff --git a/Lib/idlelib/Icons/plusnode.gif b/Lib/idlelib/Icons/plusnode.gif index 13ace90eb3269..abedde047f2e2 100644 Binary files a/Lib/idlelib/Icons/plusnode.gif and b/Lib/idlelib/Icons/plusnode.gif differ diff --git a/Lib/idlelib/Icons/python.gif b/Lib/idlelib/Icons/python.gif index b189c2c2d22c1..01b13e3583403 100644 Binary files a/Lib/idlelib/Icons/python.gif and b/Lib/idlelib/Icons/python.gif differ diff --git a/Lib/idlelib/Icons/tk.gif b/Lib/idlelib/Icons/tk.gif index a603f5ecb08a9..03bf3832abb7f 100644 Binary files a/Lib/idlelib/Icons/tk.gif and b/Lib/idlelib/Icons/tk.gif differ diff --git a/Lib/test/imghdrdata/python.gif b/Lib/test/imghdrdata/python.gif index 96fd9fef76b10..efa0be3861d79 100644 Binary files a/Lib/test/imghdrdata/python.gif and b/Lib/test/imghdrdata/python.gif differ diff --git a/Lib/test/test_email/data/PyBanner048.gif b/Lib/test/test_email/data/PyBanner048.gif index 1a5c87f647fbf..7e308f542b864 100644 Binary files a/Lib/test/test_email/data/PyBanner048.gif and b/Lib/test/test_email/data/PyBanner048.gif differ diff --git a/PC/icons/logox128.png b/PC/icons/logox128.png index d2655c72e7df4..a403de5818cfe 100644 Binary files a/PC/icons/logox128.png and b/PC/icons/logox128.png differ diff --git a/PC/icons/py.png b/PC/icons/py.png index 5c184e6a745f3..629022a70bdc4 100644 Binary files a/PC/icons/py.png and b/PC/icons/py.png differ diff --git a/PC/icons/pythonwx150.png b/PC/icons/pythonwx150.png index 4c3eb316739c7..589873aeb100b 100644 Binary files a/PC/icons/pythonwx150.png and b/PC/icons/pythonwx150.png differ diff --git a/PC/icons/pythonwx44.png b/PC/icons/pythonwx44.png index e3b32a871f90a..8cd8dd0286f68 100644 Binary files a/PC/icons/pythonwx44.png and b/PC/icons/pythonwx44.png differ diff --git a/PC/icons/pythonx150.png b/PC/icons/pythonx150.png index 5f8d30418386f..e00aa758edcd4 100644 Binary files a/PC/icons/pythonx150.png and b/PC/icons/pythonx150.png differ diff --git a/PC/icons/pythonx44.png b/PC/icons/pythonx44.png index 3881daaef2335..db0e9b91d40fe 100644 Binary files a/PC/icons/pythonx44.png and b/PC/icons/pythonx44.png differ diff --git a/PC/icons/pythonx50.png b/PC/icons/pythonx50.png index 7cc3aecd0242b..8eb5ffce221fa 100644 Binary files a/PC/icons/pythonx50.png and b/PC/icons/pythonx50.png differ diff --git a/Tools/msi/bundle/SideBar.png b/Tools/msi/bundle/SideBar.png index a23ce5e145848..52f15dd46c18f 100644 Binary files a/Tools/msi/bundle/SideBar.png and b/Tools/msi/bundle/SideBar.png differ From webhook-mailer at python.org Fri Feb 4 02:15:26 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 04 Feb 2022 07:15:26 -0000 Subject: [Python-checkins] Optimize images by IMGbot (GH-21348) Message-ID: https://github.com/python/cpython/commit/ee3d050e85381f5117f01b709f138cd094b52253 commit: ee3d050e85381f5117f01b709f138cd094b52253 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-03T23:15:17-08:00 summary: Optimize images by IMGbot (GH-21348) Co-authored-by: ImgBotApp (cherry picked from commit ba650af7d660084e08859dd1ee1917cccee24e88) Co-authored-by: Manish Kumar ? files: M Doc/howto/logging_flow.png M Doc/library/hashlib-blake2-tree.png M Doc/library/pathlib-inheritance.svg M Doc/library/tk_msg.png M Doc/using/win_installer.png M Lib/idlelib/Icons/idle_16.gif M Lib/idlelib/Icons/idle_16.png M Lib/idlelib/Icons/idle_256.png M Lib/idlelib/Icons/idle_32.gif M Lib/idlelib/Icons/idle_32.png M Lib/idlelib/Icons/idle_48.png M Lib/idlelib/Icons/minusnode.gif M Lib/idlelib/Icons/plusnode.gif M Lib/idlelib/Icons/python.gif M Lib/idlelib/Icons/tk.gif M Lib/test/imghdrdata/python.gif M Lib/test/test_email/data/PyBanner048.gif M PC/icons/logox128.png M PC/icons/py.png M PC/icons/pythonwx150.png M PC/icons/pythonwx44.png M PC/icons/pythonx150.png M PC/icons/pythonx44.png M PC/icons/pythonx50.png M Tools/msi/bundle/SideBar.png diff --git a/Doc/howto/logging_flow.png b/Doc/howto/logging_flow.png index fac4acd775530..d65e597f811db 100644 Binary files a/Doc/howto/logging_flow.png and b/Doc/howto/logging_flow.png differ diff --git a/Doc/library/hashlib-blake2-tree.png b/Doc/library/hashlib-blake2-tree.png index 73e849444ed7d..faef21b55f96b 100644 Binary files a/Doc/library/hashlib-blake2-tree.png and b/Doc/library/hashlib-blake2-tree.png differ diff --git a/Doc/library/pathlib-inheritance.svg b/Doc/library/pathlib-inheritance.svg index 49057f678fd7d..01f8684a2b201 100644 --- a/Doc/library/pathlib-inheritance.svg +++ b/Doc/library/pathlib-inheritance.svg @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file diff --git a/Doc/library/tk_msg.png b/Doc/library/tk_msg.png index c122d8f8ae5ba..6495e63e84650 100644 Binary files a/Doc/library/tk_msg.png and b/Doc/library/tk_msg.png differ diff --git a/Doc/using/win_installer.png b/Doc/using/win_installer.png index 9c18ff19cf5e8..03bf2d7b16c59 100644 Binary files a/Doc/using/win_installer.png and b/Doc/using/win_installer.png differ diff --git a/Lib/idlelib/Icons/idle_16.gif b/Lib/idlelib/Icons/idle_16.gif index 9f001b1d79cf6..bcedcd6400390 100644 Binary files a/Lib/idlelib/Icons/idle_16.gif and b/Lib/idlelib/Icons/idle_16.gif differ diff --git a/Lib/idlelib/Icons/idle_16.png b/Lib/idlelib/Icons/idle_16.png index 6abde0af90cb6..77d8f711804f9 100644 Binary files a/Lib/idlelib/Icons/idle_16.png and b/Lib/idlelib/Icons/idle_16.png differ diff --git a/Lib/idlelib/Icons/idle_256.png b/Lib/idlelib/Icons/idle_256.png index 99ffa6fad4a92..64f276b40d23b 100644 Binary files a/Lib/idlelib/Icons/idle_256.png and b/Lib/idlelib/Icons/idle_256.png differ diff --git a/Lib/idlelib/Icons/idle_32.gif b/Lib/idlelib/Icons/idle_32.gif index af5b2d52cce89..a1b12a0531d64 100644 Binary files a/Lib/idlelib/Icons/idle_32.gif and b/Lib/idlelib/Icons/idle_32.gif differ diff --git a/Lib/idlelib/Icons/idle_32.png b/Lib/idlelib/Icons/idle_32.png index 41b70dbc37766..2aaa55805ed53 100644 Binary files a/Lib/idlelib/Icons/idle_32.png and b/Lib/idlelib/Icons/idle_32.png differ diff --git a/Lib/idlelib/Icons/idle_48.png b/Lib/idlelib/Icons/idle_48.png index e5fa9280e21f8..705eec42e8a64 100644 Binary files a/Lib/idlelib/Icons/idle_48.png and b/Lib/idlelib/Icons/idle_48.png differ diff --git a/Lib/idlelib/Icons/minusnode.gif b/Lib/idlelib/Icons/minusnode.gif index c72e46ff863e4..173e97095913a 100644 Binary files a/Lib/idlelib/Icons/minusnode.gif and b/Lib/idlelib/Icons/minusnode.gif differ diff --git a/Lib/idlelib/Icons/plusnode.gif b/Lib/idlelib/Icons/plusnode.gif index 13ace90eb3269..abedde047f2e2 100644 Binary files a/Lib/idlelib/Icons/plusnode.gif and b/Lib/idlelib/Icons/plusnode.gif differ diff --git a/Lib/idlelib/Icons/python.gif b/Lib/idlelib/Icons/python.gif index b189c2c2d22c1..01b13e3583403 100644 Binary files a/Lib/idlelib/Icons/python.gif and b/Lib/idlelib/Icons/python.gif differ diff --git a/Lib/idlelib/Icons/tk.gif b/Lib/idlelib/Icons/tk.gif index a603f5ecb08a9..03bf3832abb7f 100644 Binary files a/Lib/idlelib/Icons/tk.gif and b/Lib/idlelib/Icons/tk.gif differ diff --git a/Lib/test/imghdrdata/python.gif b/Lib/test/imghdrdata/python.gif index 96fd9fef76b10..efa0be3861d79 100644 Binary files a/Lib/test/imghdrdata/python.gif and b/Lib/test/imghdrdata/python.gif differ diff --git a/Lib/test/test_email/data/PyBanner048.gif b/Lib/test/test_email/data/PyBanner048.gif index 1a5c87f647fbf..7e308f542b864 100644 Binary files a/Lib/test/test_email/data/PyBanner048.gif and b/Lib/test/test_email/data/PyBanner048.gif differ diff --git a/PC/icons/logox128.png b/PC/icons/logox128.png index d2655c72e7df4..a403de5818cfe 100644 Binary files a/PC/icons/logox128.png and b/PC/icons/logox128.png differ diff --git a/PC/icons/py.png b/PC/icons/py.png index 5c184e6a745f3..629022a70bdc4 100644 Binary files a/PC/icons/py.png and b/PC/icons/py.png differ diff --git a/PC/icons/pythonwx150.png b/PC/icons/pythonwx150.png index 4c3eb316739c7..589873aeb100b 100644 Binary files a/PC/icons/pythonwx150.png and b/PC/icons/pythonwx150.png differ diff --git a/PC/icons/pythonwx44.png b/PC/icons/pythonwx44.png index e3b32a871f90a..8cd8dd0286f68 100644 Binary files a/PC/icons/pythonwx44.png and b/PC/icons/pythonwx44.png differ diff --git a/PC/icons/pythonx150.png b/PC/icons/pythonx150.png index 5f8d30418386f..e00aa758edcd4 100644 Binary files a/PC/icons/pythonx150.png and b/PC/icons/pythonx150.png differ diff --git a/PC/icons/pythonx44.png b/PC/icons/pythonx44.png index 3881daaef2335..db0e9b91d40fe 100644 Binary files a/PC/icons/pythonx44.png and b/PC/icons/pythonx44.png differ diff --git a/PC/icons/pythonx50.png b/PC/icons/pythonx50.png index 7cc3aecd0242b..8eb5ffce221fa 100644 Binary files a/PC/icons/pythonx50.png and b/PC/icons/pythonx50.png differ diff --git a/Tools/msi/bundle/SideBar.png b/Tools/msi/bundle/SideBar.png index a23ce5e145848..52f15dd46c18f 100644 Binary files a/Tools/msi/bundle/SideBar.png and b/Tools/msi/bundle/SideBar.png differ From webhook-mailer at python.org Fri Feb 4 02:31:38 2022 From: webhook-mailer at python.org (methane) Date: Fri, 04 Feb 2022 07:31:38 -0000 Subject: [Python-checkins] Optimize images by IMGbot (GH-21348) Message-ID: https://github.com/python/cpython/commit/34895f694ccddbd4c9320a625c1b4b139c9c3bb3 commit: 34895f694ccddbd4c9320a625c1b4b139c9c3bb3 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: methane date: 2022-02-04T16:31:21+09:00 summary: Optimize images by IMGbot (GH-21348) Co-authored-by: ImgBotApp (cherry picked from commit ba650af7d660084e08859dd1ee1917cccee24e88) files: M Doc/howto/logging_flow.png M Doc/library/hashlib-blake2-tree.png M Doc/library/pathlib-inheritance.svg M Doc/library/tk_msg.png M Doc/using/win_installer.png M Lib/idlelib/Icons/idle_16.gif M Lib/idlelib/Icons/idle_16.png M Lib/idlelib/Icons/idle_256.png M Lib/idlelib/Icons/idle_32.gif M Lib/idlelib/Icons/idle_32.png M Lib/idlelib/Icons/idle_48.png M Lib/idlelib/Icons/minusnode.gif M Lib/idlelib/Icons/plusnode.gif M Lib/idlelib/Icons/python.gif M Lib/idlelib/Icons/tk.gif M Lib/test/imghdrdata/python.gif M Lib/test/test_email/data/PyBanner048.gif M PC/icons/logox128.png M PC/icons/py.png M PC/icons/pythonwx150.png M PC/icons/pythonwx44.png M PC/icons/pythonx150.png M PC/icons/pythonx44.png M PC/icons/pythonx50.png M Tools/msi/bundle/SideBar.png diff --git a/Doc/howto/logging_flow.png b/Doc/howto/logging_flow.png index fac4acd7755302..d65e597f811db5 100644 Binary files a/Doc/howto/logging_flow.png and b/Doc/howto/logging_flow.png differ diff --git a/Doc/library/hashlib-blake2-tree.png b/Doc/library/hashlib-blake2-tree.png index 73e849444ed7d3..faef21b55f96b1 100644 Binary files a/Doc/library/hashlib-blake2-tree.png and b/Doc/library/hashlib-blake2-tree.png differ diff --git a/Doc/library/pathlib-inheritance.svg b/Doc/library/pathlib-inheritance.svg index 49057f678fd7d0..01f8684a2b201c 100644 --- a/Doc/library/pathlib-inheritance.svg +++ b/Doc/library/pathlib-inheritance.svg @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file diff --git a/Doc/library/tk_msg.png b/Doc/library/tk_msg.png index c122d8f8ae5ba9..6495e63e84650d 100644 Binary files a/Doc/library/tk_msg.png and b/Doc/library/tk_msg.png differ diff --git a/Doc/using/win_installer.png b/Doc/using/win_installer.png index 9c18ff19cf5e81..03bf2d7b16c599 100644 Binary files a/Doc/using/win_installer.png and b/Doc/using/win_installer.png differ diff --git a/Lib/idlelib/Icons/idle_16.gif b/Lib/idlelib/Icons/idle_16.gif index 9f001b1d79cf69..bcedcd64003906 100644 Binary files a/Lib/idlelib/Icons/idle_16.gif and b/Lib/idlelib/Icons/idle_16.gif differ diff --git a/Lib/idlelib/Icons/idle_16.png b/Lib/idlelib/Icons/idle_16.png index 6abde0af90cb6b..77d8f711804f95 100644 Binary files a/Lib/idlelib/Icons/idle_16.png and b/Lib/idlelib/Icons/idle_16.png differ diff --git a/Lib/idlelib/Icons/idle_256.png b/Lib/idlelib/Icons/idle_256.png index 99ffa6fad4a92d..64f276b40d23b7 100644 Binary files a/Lib/idlelib/Icons/idle_256.png and b/Lib/idlelib/Icons/idle_256.png differ diff --git a/Lib/idlelib/Icons/idle_32.gif b/Lib/idlelib/Icons/idle_32.gif index af5b2d52cce895..a1b12a0531d64a 100644 Binary files a/Lib/idlelib/Icons/idle_32.gif and b/Lib/idlelib/Icons/idle_32.gif differ diff --git a/Lib/idlelib/Icons/idle_32.png b/Lib/idlelib/Icons/idle_32.png index 41b70dbc37766a..2aaa55805ed530 100644 Binary files a/Lib/idlelib/Icons/idle_32.png and b/Lib/idlelib/Icons/idle_32.png differ diff --git a/Lib/idlelib/Icons/idle_48.png b/Lib/idlelib/Icons/idle_48.png index e5fa9280e21f86..705eec42e8a64e 100644 Binary files a/Lib/idlelib/Icons/idle_48.png and b/Lib/idlelib/Icons/idle_48.png differ diff --git a/Lib/idlelib/Icons/minusnode.gif b/Lib/idlelib/Icons/minusnode.gif index c72e46ff863e47..173e97095913ae 100644 Binary files a/Lib/idlelib/Icons/minusnode.gif and b/Lib/idlelib/Icons/minusnode.gif differ diff --git a/Lib/idlelib/Icons/plusnode.gif b/Lib/idlelib/Icons/plusnode.gif index 13ace90eb3269a..abedde047f2e2b 100644 Binary files a/Lib/idlelib/Icons/plusnode.gif and b/Lib/idlelib/Icons/plusnode.gif differ diff --git a/Lib/idlelib/Icons/python.gif b/Lib/idlelib/Icons/python.gif index b189c2c2d22c12..01b13e35834038 100644 Binary files a/Lib/idlelib/Icons/python.gif and b/Lib/idlelib/Icons/python.gif differ diff --git a/Lib/idlelib/Icons/tk.gif b/Lib/idlelib/Icons/tk.gif index a603f5ecb08a95..03bf3832abb7f6 100644 Binary files a/Lib/idlelib/Icons/tk.gif and b/Lib/idlelib/Icons/tk.gif differ diff --git a/Lib/test/imghdrdata/python.gif b/Lib/test/imghdrdata/python.gif index 96fd9fef76b108..efa0be3861d79f 100644 Binary files a/Lib/test/imghdrdata/python.gif and b/Lib/test/imghdrdata/python.gif differ diff --git a/Lib/test/test_email/data/PyBanner048.gif b/Lib/test/test_email/data/PyBanner048.gif index 1a5c87f647fbf3..7e308f542b864c 100644 Binary files a/Lib/test/test_email/data/PyBanner048.gif and b/Lib/test/test_email/data/PyBanner048.gif differ diff --git a/PC/icons/logox128.png b/PC/icons/logox128.png index d2655c72e7df49..a403de5818cfec 100644 Binary files a/PC/icons/logox128.png and b/PC/icons/logox128.png differ diff --git a/PC/icons/py.png b/PC/icons/py.png index 5c184e6a745f36..629022a70bdc44 100644 Binary files a/PC/icons/py.png and b/PC/icons/py.png differ diff --git a/PC/icons/pythonwx150.png b/PC/icons/pythonwx150.png index 4c3eb316739c79..589873aeb100b2 100644 Binary files a/PC/icons/pythonwx150.png and b/PC/icons/pythonwx150.png differ diff --git a/PC/icons/pythonwx44.png b/PC/icons/pythonwx44.png index e3b32a871f90a7..8cd8dd0286f681 100644 Binary files a/PC/icons/pythonwx44.png and b/PC/icons/pythonwx44.png differ diff --git a/PC/icons/pythonx150.png b/PC/icons/pythonx150.png index 5f8d30418386fd..e00aa758edcd47 100644 Binary files a/PC/icons/pythonx150.png and b/PC/icons/pythonx150.png differ diff --git a/PC/icons/pythonx44.png b/PC/icons/pythonx44.png index 3881daaef2335a..db0e9b91d40fe5 100644 Binary files a/PC/icons/pythonx44.png and b/PC/icons/pythonx44.png differ diff --git a/PC/icons/pythonx50.png b/PC/icons/pythonx50.png index 7cc3aecd0242b9..8eb5ffce221fae 100644 Binary files a/PC/icons/pythonx50.png and b/PC/icons/pythonx50.png differ diff --git a/Tools/msi/bundle/SideBar.png b/Tools/msi/bundle/SideBar.png index a23ce5e1458488..52f15dd46c18fe 100644 Binary files a/Tools/msi/bundle/SideBar.png and b/Tools/msi/bundle/SideBar.png differ From webhook-mailer at python.org Fri Feb 4 04:56:50 2022 From: webhook-mailer at python.org (markshannon) Date: Fri, 04 Feb 2022 09:56:50 -0000 Subject: [Python-checkins] Add miss stats for specialized instructions. (GH-31108) Message-ID: https://github.com/python/cpython/commit/832876b99212999c063cbf41bdacc574da699190 commit: 832876b99212999c063cbf41bdacc574da699190 branch: main author: Mark Shannon committer: markshannon date: 2022-02-04T09:56:46Z summary: Add miss stats for specialized instructions. (GH-31108) files: M Python/ceval.c M Python/specialize.c M Tools/scripts/summarize_stats.py diff --git a/Python/ceval.c b/Python/ceval.c index b4029d1081dcb..e6b5d3ae24237 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -5403,6 +5403,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr #define MISS_WITH_CACHE(opname) \ opname ## _miss: \ { \ + STAT_INC(opcode, miss); \ STAT_INC(opname, miss); \ _PyAdaptiveEntry *cache = &GET_CACHE()->adaptive; \ cache->counter--; \ diff --git a/Python/specialize.c b/Python/specialize.c index 4070d6a6a0be4..b7ef478ee5590 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -38,6 +38,33 @@ */ +/* Map from opcode to adaptive opcode. + Values of zero are ignored. */ +static uint8_t adaptive_opcodes[256] = { + [LOAD_ATTR] = LOAD_ATTR_ADAPTIVE, + [LOAD_GLOBAL] = LOAD_GLOBAL_ADAPTIVE, + [LOAD_METHOD] = LOAD_METHOD_ADAPTIVE, + [BINARY_SUBSCR] = BINARY_SUBSCR_ADAPTIVE, + [STORE_SUBSCR] = STORE_SUBSCR_ADAPTIVE, + [CALL] = CALL_ADAPTIVE, + [STORE_ATTR] = STORE_ATTR_ADAPTIVE, + [BINARY_OP] = BINARY_OP_ADAPTIVE, + [COMPARE_OP] = COMPARE_OP_ADAPTIVE, +}; + +/* The number of cache entries required for a "family" of instructions. */ +static uint8_t cache_requirements[256] = { + [LOAD_ATTR] = 2, /* _PyAdaptiveEntry and _PyAttrCache */ + [LOAD_GLOBAL] = 2, /* _PyAdaptiveEntry and _PyLoadGlobalCache */ + [LOAD_METHOD] = 3, /* _PyAdaptiveEntry, _PyAttrCache and _PyObjectCache */ + [BINARY_SUBSCR] = 2, /* _PyAdaptiveEntry, _PyObjectCache */ + [STORE_SUBSCR] = 0, + [CALL] = 2, /* _PyAdaptiveEntry and _PyObjectCache/_PyCallCache */ + [STORE_ATTR] = 2, /* _PyAdaptiveEntry and _PyAttrCache */ + [BINARY_OP] = 1, // _PyAdaptiveEntry + [COMPARE_OP] = 1, /* _PyAdaptiveEntry */ +}; + Py_ssize_t _Py_QuickenedCount = 0; #ifdef Py_STATS PyStats _py_stats = { 0 }; @@ -144,7 +171,14 @@ _Py_GetSpecializationStats(void) { static void print_spec_stats(FILE *out, OpcodeStats *stats) { + /* Mark some opcodes as specializable for stats, + * even though we don't specialize them yet. */ + fprintf(out, " opcode[%d].specializable : 1\n", FOR_ITER); + fprintf(out, " opcode[%d].specializable : 1\n", UNPACK_SEQUENCE); for (int i = 0; i < 256; i++) { + if (adaptive_opcodes[i]) { + fprintf(out, " opcode[%d].specializable : 1\n", i); + } PRINT_STAT(i, specialization.success); PRINT_STAT(i, specialization.failure); PRINT_STAT(i, specialization.hit); @@ -266,33 +300,6 @@ get_cache_count(SpecializedCacheOrInstruction *quickened) { return quickened[0].entry.zero.cache_count; } -/* Map from opcode to adaptive opcode. - Values of zero are ignored. */ -static uint8_t adaptive_opcodes[256] = { - [LOAD_ATTR] = LOAD_ATTR_ADAPTIVE, - [LOAD_GLOBAL] = LOAD_GLOBAL_ADAPTIVE, - [LOAD_METHOD] = LOAD_METHOD_ADAPTIVE, - [BINARY_SUBSCR] = BINARY_SUBSCR_ADAPTIVE, - [STORE_SUBSCR] = STORE_SUBSCR_ADAPTIVE, - [CALL] = CALL_ADAPTIVE, - [STORE_ATTR] = STORE_ATTR_ADAPTIVE, - [BINARY_OP] = BINARY_OP_ADAPTIVE, - [COMPARE_OP] = COMPARE_OP_ADAPTIVE, -}; - -/* The number of cache entries required for a "family" of instructions. */ -static uint8_t cache_requirements[256] = { - [LOAD_ATTR] = 2, /* _PyAdaptiveEntry and _PyAttrCache */ - [LOAD_GLOBAL] = 2, /* _PyAdaptiveEntry and _PyLoadGlobalCache */ - [LOAD_METHOD] = 3, /* _PyAdaptiveEntry, _PyAttrCache and _PyObjectCache */ - [BINARY_SUBSCR] = 2, /* _PyAdaptiveEntry, _PyObjectCache */ - [STORE_SUBSCR] = 0, - [CALL] = 2, /* _PyAdaptiveEntry and _PyObjectCache/_PyCallCache */ - [STORE_ATTR] = 2, /* _PyAdaptiveEntry and _PyAttrCache */ - [BINARY_OP] = 1, // _PyAdaptiveEntry - [COMPARE_OP] = 1, /* _PyAdaptiveEntry */ -}; - /* Return the oparg for the cache_offset and instruction index. * * If no cache is needed then return the original oparg. diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index f67a35a04a9dd..6d0020739a31f 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -22,11 +22,10 @@ pass opname.append(name) - TOTAL = "specialization.deferred", "specialization.hit", "specialization.miss", "execution_count" def print_specialization_stats(name, family_stats): - if "specialization.failure" not in family_stats: + if "specializable" not in family_stats: return total = sum(family_stats.get(kind, 0) for kind in TOTAL) if total == 0: @@ -87,13 +86,18 @@ def main(): for i, opcode_stat in enumerate(opcode_stats): if "execution_count" in opcode_stat: count = opcode_stat['execution_count'] - counts.append((count, opname[i])) + miss = 0 + if "specializable" not in opcode_stat: + miss = opcode_stat.get("specialization.miss") + counts.append((count, opname[i], miss)) total += count counts.sort(reverse=True) cummulative = 0 - for (count, name) in counts: + for (count, name, miss) in counts: cummulative += count print(f"{name}: {count} {100*count/total:0.1f}% {100*cummulative/total:0.1f}%") + if miss: + print(f" Misses: {miss} {100*miss/count:0.1f}%") print("Specialization stats:") for i, opcode_stat in enumerate(opcode_stats): name = opname[i] From webhook-mailer at python.org Fri Feb 4 11:11:50 2022 From: webhook-mailer at python.org (zooba) Date: Fri, 04 Feb 2022 16:11:50 -0000 Subject: [Python-checkins] bpo-46629: Update classicAppCompat.sccd for new signing certificate (GH-31111) Message-ID: https://github.com/python/cpython/commit/9b4e3d94a5746af093392ed8e977b26fcc1bfd11 commit: 9b4e3d94a5746af093392ed8e977b26fcc1bfd11 branch: main author: Steve Dower committer: zooba date: 2022-02-04T16:11:19Z summary: bpo-46629: Update classicAppCompat.sccd for new signing certificate (GH-31111) files: M PC/classicAppCompat.can.xml M PC/classicAppCompat.sccd diff --git a/PC/classicAppCompat.can.xml b/PC/classicAppCompat.can.xml index f00475c8da312..df361f8e3e2cd 100644 --- a/PC/classicAppCompat.can.xml +++ b/PC/classicAppCompat.can.xml @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file diff --git a/PC/classicAppCompat.sccd b/PC/classicAppCompat.sccd index 97648985a2ccb..d7a70cdd0533c 100644 --- a/PC/classicAppCompat.sccd +++ b/PC/classicAppCompat.sccd @@ -1,28 +1,41 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - MIIq5AYJKoZIhvcNAQcCoIIq1TCCKtECAQExDzANBglghkgBZQMEAgEFADCCARAGCSsGAQQBgjcKAaCCAQEwgf4wDAYKKwYBBAGCNwwBAQQQaM+L42jwBUGvBczrtolMmhcNMTgxMTMwMDA1OTAzWjAOBgorBgEEAYI3DAEDBQAwgbwwKgQUWKcU3R38DGPlKK33XGIwKtVL1r4xEjAQBgorBgEEAYI3DAIDMQKCADCBjQQg3K+KBOQX7HfxjRNZC9cx8gIPkEhPRO1nJFRdWQrVEJ4xaTAQBgorBgEEAYI3DAIDMQKCADBVBgorBgEEAYI3AgEEMUcwRTAQBgorBgEEAYI3AgEZogKAADAxMA0GCWCGSAFlAwQCAQUABCDcr4oE5Bfsd/GNE1kL1zHyAg+QSE9E7WckVF1ZCtUQnqCCFFAwggZSMIIEOqADAgECAhMzAAMu49KhfNamygpWAAIAAy7jMA0GCSqGSIb3DQEBCwUAMIGMMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMQ0wCwYDVQQLEwRNT1BSMScwJQYDVQQDEx5NaWNyb3NvZnQgTWFya2V0cGxhY2UgQ0EgRyAwMTMwHhcNMTgxMTMwMDA1NTA1WhcNMTgxMjAzMDA1NTA1WjB0MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMR4wHAYDVQQDExVNaWNyb3NvZnQgQ29ycG9yYXRpb24wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCwpcimfAx3HEpba1GLL/gDaRVddHE5PXTRmwlgaz8kt6/rq5rlrPFnCnbIc5818v0xJIznastbmrq26xyCEHyMLBKnyneTKE36I7+TGjcY0D7ow+o2vY7LDKMCTGlh31fx1Tvrl+5xTbWX5jdLU/3MB5faeOGh+0Knzwx1KDoXWgPtfXnD8I5jxJieoWoCwCjKTJgBOklLy9nbOalxf0h+xQRy2p5fj+PxAwQPgHWft36AF7/IMbt9FcXMtg4xdpnTYz4OV3dFOPz4m3M8HwVgNMv89W/1Ozc7uOyZt0Ij1baT6r2L3IjYg5ftzpGqaDOFcWlyDFSdhMR6BIKW8xEpAgMBAAGjggHCMIIBvjAYBgNVHSUBAf8EDjAMBgorBgEEAYI3TBwBMB0GA1UdDgQWBBRdpGYiCytx83FYzPSl+o97YzpxGzAPBgNVHREECDAGggRNT1BSMB8GA1UdIwQYMBaAFEnYB1RFhpclHtZZcRLDcpt0OE3oMGIGA1UdHwRbMFkwV6BVoFOGUWh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY3JsL01pY3Jvc29mdCUyME1hcmtldHBsYWNlJTIwQ0ElMjBHJTIwMDEzKDIpLmNybDBvBggrBgEFBQcBAQRjMGEwXwYIKwYBBQUHMAKGU2h0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY2VydHMvTWljcm9zb2Z0JTIwTWFya2V0cGxhY2UlMjBDQSUyMEclMjAwMTMoMikuY3J0MAwGA1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgXgMDwGCSsGAQQBgjcVBwQvMC0GJSsGAQQBgjcVCIOS9kTqrxCDkY0wgqzgLIKinDE0g+6NOIaE7wACAWQCARYwIAYJKwYBBAGCNxUKAQH/BBAwDjAMBgorBgEEAYI3TBwBMA0GCSqGSIb3DQEBCwUAA4ICAQB3Dk3rXH52CDq/z1fwqn9xI5WGjGmu6oAE4HSc3sNdFrSVMMGm4gTlYGWSZ0wJUUf16mVr/rdXhxuR3MZn+m4Bhdl8KQqYjYbIvCUVj0o9nZ+yT6foeY8bKnB+K5h6rol+mjDj5IfcutC4x2Kx5RrtDtRTSoKA63iZ74DYngPpBGBBgaS2c/QzgqPRAMMRqy2KBDP0miCnpR3F4YlzHGyOZwyHhESjYd9kwF47+msuHS04JZpnGHIvBppKN9XQzH3WezNnnX3lz4AyAUMsMFuARqEnacUhrAHL9n5zMv9CzxDYN1r1/aDh/788RuGuZM+E3NtmbxJJ7j6T5/VtXNBRgKtIq8d2+11j6qvKLigOTxSC25/A70BZBEvllLFnvc1vA2LrC9drwt1KpSmWie1nvpilw7o+gHMOG9utUxGha2VuVizuVNGCywTRRjvmGS1QqTfaun1URVrLfnDINXuTgN1Vwp0J5IGpJ3D8yj01NDQ/RworE+3W/R531NBYova9QRhU/igEw/Aa/q8wjZ4Pzxr9oBIo0Ta3Tv6qIggaWXw0U9+F0J7SCqIhn0d0ATO+E1Qs/SxZIAICLwmqzoLYUAh8q153esBs4uesueqgt5ueyHK8V3WjMS4wxEyVN5ZMET3hFtEshsZC31tLDdjq750U4SgQVmoYSm3F3ZOKQDCCBtcwggS/oAMCAQICCmESRKIAAAAAAAIwDQYJKoZIhvcNAQELBQAwgYgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xMjAwBgNVBAMTKU1pY3Jvc29mdCBSb290IENlcnRpZmljYXRlIEF1dGhvcml0eSAyMDExMB4XDTExMDMyODIxMDkzOVoXDTMxMDMyODIxMTkzOVowfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAubUaSwGYVsE3MAnPfvmozUhAB3qxBABgJRW1vDp4+tVinXxD32f7k1K89JQ6zDOgS/iDgULC+yFK1K/1Qjac/0M7P6c8v5LSjnWGlERLa/qY32j46S7SLQcit3g2jgoTTO03eUG+9yHZUTGV/FJdRYB8uXhrznJBa+Y+yGwiQKF+m6XFeBH/KORoKFx+dmMoy9EWJ/m/o9IiUj2kzm9C691+vZ/I2w0Bj93W9SPPkV2PCNHlzgfIAoeajWpHmi38Wi3xZHonkzAVBHxPsCBppOoNsWvmAfUM7eBthkSPvFruekyDCPNEYhfGqgqtqLkoBebXLZCOVybF7wTQaLvse60//3P003icRcCoQYgY4NAqrF7j80o5U7DkeXxcB0xvengsaKgiAaV1DKkRbpe98wCqr1AASvm5rAJUYMU+mXmOieV2EelY2jGrenWe9FQpNXYV1NoWBh0WKoFxttoWYAnF705bIWtSZsz08ZfK6WLX4GXNLcPBlgCzfTm1sdKYASWdBbH2haaNhPapFhQQBJHKwnVW2iXErImhuPi45W3MVTZ5D9ASshZx69cLYY6xAdIa+89Kf/uRrsGOVZfahDuDw+NI183iAyzC8z/QRt2P32LYxP0xrCdqVh+DJo2i4NoE8Uk1usCdbVRuBMBQl/AwpOTq7IMvHGElf65CqzUCAwEAAaOCAUswggFHMBAGCSsGAQQBgjcVAQQDAgEAMB0GA1UdDgQWBBQPU8s/FmEl/mCJHdO5fOiQrbOU0TAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBRyLToCMZBDuRQFTuHqp8cx0SOJNDBaBgNVHR8EUzBRME+gTaBLhklodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3JsMF4GCCsGAQUFBwEBBFIwUDBOBggrBgEFBQcwAoZCaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3J0MA0GCSqGSIb3DQEBCwUAA4ICAQCjuZmM8ZVNDgp9wHsL4RY8KJ8nLinvxFTphNGCrxaLknkYG5pmMhVlX+UB/tSiW8W13W60nggz9u5xwMx7v/1t/Tgm6g2brVyOKI5A7u6/2SIJwkJKFw953K0YIKVT28w9zl8dSJnmRnyR0G86ncWbF6CLQ6A6lBQ9o2mTGVqDr4m35WKAnc6YxUUM1y74mbzFFZr63VHsCcOp3pXWnUqAY1rb6Q6NX1b3clncKqLFm0EjKHcQ56grTbwuuB7pMdh/IFCJR01MQzQbDtpEisbOeZUi43YVAAHKqI1EO9bRwg3frCjwAbml9MmI4utMW94gWFgvrMxIX+n42RBDIjf3Ot3jkT6gt3XeTTmO9bptgblZimhERdkFRUFpVtkocJeLoGuuzP93uH/Yp032wzRH+XmMgujfZv+vnfllJqxdowoQLx55FxLLeTeYfwi/xMSjZO2gNven3U/3KeSCd1kUOFS3AOrwZ0UNOXJeW5JQC6Vfd1BavFZ6FAta1fMLu3WFvNB+FqeHUaU3ya7rmtxJnzk29DeSqXgGNmVSywBS4NajI5jJIKAA6UhNJlsg8CHYwUOKf5ej8OoQCkbadUxXygAfxCfW2YBbujtI+PoyejRFxWUjYFWO5LeTI62UMyqfOEiqugoYjNxmQZla2s4YHVuqIC34R85FQlg9pKQBsDCCBxswggUDoAMCAQICEzMAAABCs21EHGjyqKYAAAAAAEIwDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMB4XDTE4MDQyMDE2NDI0NFoXDTIxMDQyMDE2NDI0NFowgYwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xDTALBgNVBAsTBE1PUFIxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAxMzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAOZ2KM9Pq1YCOiqWOivmHjUtkMgznTMP/Mr2YfzZeIIJySg1F4WxFZc4jagGHHNof9NRT+GGnktWsXkZuH1DzQEG4Ps1ln8+4vhbDglqu5ymDnd6RmsyoD+8xfc8bBIvE5o6R+ES4/GVD5TqNsOrWbwETaIZVbmTulJLoTS1WSsSjowmbc+sHqZiY8BNJNThUEmXSjuHqkQKKshuiFWYEqOTitp71mBLyH1wN7/jThRzGpolOeFusRNJdb8sEqvNzEN9Qh+Kp6ndzrnjE+t8ixXW3lShyyOOZqQMwsQn9q9T0v7Q69GuojBTFBOHKwigcCHr4xahuN+ZYMk0xGg+sm3Uj7I9mrWTSTiIRMZNIWq3sFg4+rFg48NYfRlXUpONmL7vXq6v1pIU99d2MXQ6uUrnUr1/n5ZiHGCeFcvWwqO8BYHdcTlrSOkayfFp7W9oCk9QO4Xy0h9cQRedRo2kvdTHxIuJS70Hdv6oePPF2ZFaLucUzzwsR4/XMAVKY8Vsm950omsSSOImsMtzavUdQM+wZFxvHTRqVDkF3quPdME0bCZOWB4hQJmd+o2clw+1mpwPu0/M92nA9FJg7MGPxkFaYW7g26jSqUJZ9AcX+Xa5TSIeqMZt3cRVjMTx0T/v73Sv8TpalqIQ5Fde1+hFK07sOAm3TwgzvlVJnbYgp0/rAgMBAAGjggGCMIIBfjASBgkrBgEEAYI3FQEEBQIDAgACMCMGCSsGAQQBgjcVAgQWBBSbJnDhuc3nQXuKuACsPflEbwjbozAdBgNVHQ4EFgQUSdgHVEWGlyUe1llxEsNym3Q4TegwEQYDVR0gBAowCDAGBgRVHSAAMBkGCSsGAQQBgjcUAgQMHgoAUwB1AGIAQwBBMAsGA1UdDwQEAwIBhjASBgNVHRMBAf8ECDAGAQH/AgEAMB8GA1UdIwQYMBaAFA9Tyz8WYSX+YIkd07l86JCts5TRMFcGA1UdHwRQME4wTKBKoEiGRmh0dHA6Ly9jcmwubWljcm9zb2Z0LmNvbS9wa2kvY3JsL3Byb2R1Y3RzL01pY01hclBDQTIwMTFfMjAxMS0wMy0yOC5jcmwwWwYIKwYBBQUHAQEETzBNMEsGCCsGAQUFBzAChj9odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY01hclBDQTIwMTFfMjAxMS0wMy0yOC5jcnQwDQYJKoZIhvcNAQELBQADggIBAIa2oa6kvuIHCNfz7anlL0W9tOCt8gQNkxOGRK3yliQIelNQahDJojyEFlHQ2BcHL5oZit3WeSDoYddhojx6YzJIWwfGwtVqgc0JFDKJJ2ZXRYMRsuy01Hn25xob+zRMS6VmV1axQn6uwOSMcgYmzoroh6edjPKu7qXcpt6LmhF2qFvLySA7wBCwfI/rR5/PX6I7a07Av7PpbY6/+2ujd8m1H3hwMrb4Hq3z6gcq62zJ3nDXUbC0Bp6Jt2kV9f0rEFpDK9oxE2qrGBUf8c3O2XirHOgAjRyWjWWtVms+MP8qBIA1NSLrBmToEWVP3sEkQZWMkoZWo4rYEJZpX7UIgdDc9zYNakgTCJqPhqn8AE1sgSSnpqAdMkkP41rTlFCv2ig2QVzDerjGfEv+uPDnlAT0kucbBJxHHvUC4aqUxaTSa0sy2bZ6NWFx8/u0gW8JahzxYvvvZL8SfwaA9P4ETb8pH1jw+6N/LfM2zJrNKhf5hjKa0VDOXUpkYq60OqVVnWJ6oJaSIWNkZKfzPnl/UHA8Bh4qfVrhc9H5PExPhhB9WVTsjf4r+OOVuolJldThcWQqljiPjk5rultr63G5xLyFpxNi4BCrcNQBJFB5wKgOWOyjQTVWTmh2ESaeqZ2aWBjftFHlxJ/qYc7WOGJV0+cHGkB/dvFxmKnv6tuWexiMMYIVUTCCFU0CAQEwgaQwgYwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xDTALBgNVBAsTBE1PUFIxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAxMwITMwADLuPSoXzWpsoKVgACAAMu4zANBglghkgBZQMEAgEFAKCBlTAYBgkqhkiG9w0BCQMxCwYJKwYBBAGCNwoBMC8GCSqGSIb3DQEJBDEiBCAS0d3bw2YOODvKFr0S4e3BDnaDcZXUKeBO77yvkWzVojBIBgorBgEEAYI3AgEMMTowOKAegBwATQBpAGMAcgBvAHMAbwBmAHQAIABDAG8AcgBwoRaAFGh0dHA6Ly9NaWNyb3NvZnQuY29tMA0GCSqGSIb3DQEBAQUABIIBABoap3Y+2k+zFz2cCmkc8xxHnpIygLsUSRMXeXdjPVcYx3o5cPLIixnL6p8+LIrlIagPg23mzTEmnjZaO4aaexk+3XojlHj22w/bEigEDnKyWt5bHeS0UNHJbxEFYRfd84IP1+mSH4c4+GuU9p3LsAMh6wN03MYrGmczUOnlP6YlxHNQbQxnV0sl14yOE5ni9oT4y+l+SllvbV3/Jhwpov68aoP/2MazqxR4QyGfSxhCPJ4UuDHU7IrpnTxGBTL1/oUU8ED0FxyDoH/Sc5OhTLInFqbZaVzm5Mpr12wYUBL4nE5h0Kf6BCKdgM8a+Ti3wMUsBoC79ff3jE9U/xwSneOhghLlMIIS4QYKKwYBBAGCNwMDATGCEtEwghLNBgkqhkiG9w0BBwKgghK+MIISugIBAzEPMA0GCWCGSAFlAwQCAQUAMIIBUQYLKoZIhvcNAQkQAQSgggFABIIBPDCCATgCAQEGCisGAQQBhFkKAwEwMTANBglghkgBZQMEAgEFAAQghPy22lwuCYESw8jYhb4F9ZDPJ1LPgSSZgJDkyXYzVt4CBlv98KtAoBgTMjAxODExMzAwMTA1MTkuMTM4WjAEgAIB9KCB0KSBzTCByjELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAldBMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xLTArBgNVBAsTJE1pY3Jvc29mdCBJcmVsYW5kIE9wZXJhdGlvbnMgTGltaXRlZDEmMCQGA1UECxMdVGhhbGVzIFRTUyBFU046RDA4Mi00QkZELUVFQkExJTAjBgNVBAMTHE1pY3Jvc29mdCBUaW1lLVN0YW1wIHNlcnZpY2Wggg48MIIE8TCCA9mgAwIBAgITMwAAAOIYOHtm6erB2AAAAAAA4jANBgkqhkiG9w0BAQsFADB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMDAeFw0xODA4MjMyMDI3MDNaFw0xOTExMjMyMDI3MDNaMIHKMQswCQYDVQQGEwJVUzELMAkGA1UECBMCV0ExEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEtMCsGA1UECxMkTWljcm9zb2Z0IElyZWxhbmQgT3BlcmF0aW9ucyBMaW1pdGVkMSYwJAYDVQQLEx1UaGFsZXMgVFNTIEVTTjpEMDgyLTRCRkQtRUVCQTElMCMGA1UEAxMcTWljcm9zb2Z0IFRpbWUtU3RhbXAgc2VydmljZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKirA72FF3NCLW5mfLO/D0EZ5Ycs00oiMSissXLB6WF9GNdP78QzFwAypxW/+qZSczqaHbDH8hlbxkzf3DiYgAdpQjnGkLujwKtWSaP29/lVf7jFqHy9v6eH+LdOi0LvtrPRW34MyCvpxZyOW4H1h3PkxCBL5Ra21sDqgcVL1me0osw8QTURXmI4LyeLdTH3CcI2AgNDXTjsFBf3QsO+JYyAOYWrTcLnywVN6DrigmgrDJk5w+wR4VrHfl2T9PRZbZ+UDt13wwyB9d6IURuzV8lHsAVfF8t9S0aGVPmkQ3c2waOhHpsp6VEM+T5D2Ph8xJX1r82z67WRlmGcOP2NWC0CAwEAAaOCARswggEXMB0GA1UdDgQWBBSJPpD6BsP2p+crDJL232voEtLxezAfBgNVHSMEGDAWgBTVYzpcijGQ80N7fEYbxTNoWoVtVTBWBgNVHR8ETzBNMEugSaBHhkVodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNUaW1TdGFQQ0FfMjAxMC0wNy0wMS5jcmwwWgYIKwYBBQUHAQEETjBMMEoGCCsGAQUFBzAChj5odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY1RpbVN0YVBDQV8yMDEwLTA3LTAxLmNydDAMBgNVHRMBAf8EAjAAMBMGA1UdJQQMMAoGCCsGAQUFBwMIMA0GCSqGSIb3DQEBCwUAA4IBAQARQHu7ISeBuJSHKuDRI04704cH0B7BYzeEIrD15awviMRcYIfIOHpvGzZOWQgP2Hm0Rr7kvTUu1VrSSaQ7i1gPWdhqMmw5WBnSS5bxeMhhx9UsASeE84vUu82NeZapGSjH38YAb4WT+TtiTkcoI59rA+CTCq108ttIxVfZcr3id76OETIH0HvhlnxOOWjwGy4ul6Za5RoTLG/oo2rrGmVi3FwrNWGezYLBODuEsjzG36lCRtBKC2ZAHfbOz5wtkUHbqh79mUKocjP4r3qxf5TN87yf6g1uTx+J8pdnAi5iHt+ZtangWqnVTE8PoIREWhBVlGFfQdkELUx2Or90aAqWMIIGcTCCBFmgAwIBAgIKYQmBKgAAAAAAAjANBgkqhkiG9w0BAQsFADCBiDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEyMDAGA1UEAxMpTWljcm9zb2Z0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IDIwMTAwHhcNMTAwNzAxMjEzNjU1WhcNMjUwNzAxMjE0NjU1WjB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKkdDbx3EYo6IOz8E5f1+n9plGt0VBDVpQoAgoX77XxoSyxfxcPlYcJ2tz5mK1vwFVMnBDEfQRsalR3OCROOfGEwWbEwRA/xYIiEVEMM1024OAizQt2TrNZzMFcmgqNFDdDq9UeBzb8kYDJYYEbyWEeGMoQedGFnkV+BVLHPk0ySwcSmXdFhE24oxhr5hoC732H8RsEnHSRnEnIaIYqvS2SJUGKxXf13Hz3wV3WsvYpCTUBR0Q+cBj5nf/VmwAOWRH7v0Ev9buWayrGo8noqCjHw2k4GkbaICDXoeByw6ZnNPOcvRLqn9NxkvaQBwSAJk3jN/LzAyURdXhacAQVPIk0CAwEAAaOCAeYwggHiMBAGCSsGAQQBgjcVAQQDAgEAMB0GA1UdDgQWBBTVYzpcijGQ80N7fEYbxTNoWoVtVTAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBTV9lbLj+iiXGJo0T2UkFvXzpoYxDBWBgNVHR8ETzBNMEugSaBHhkVodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNSb29DZXJBdXRfMjAxMC0wNi0yMy5jcmwwWgYIKwYBBQUHAQEETjBMMEoGCCsGAQUFBzAChj5odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY1Jvb0NlckF1dF8yMDEwLTA2LTIzLmNydDCBoAYDVR0gAQH/BIGVMIGSMIGPBgkrBgEEAYI3LgMwgYEwPQYIKwYBBQUHAgEWMWh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9QS0kvZG9jcy9DUFMvZGVmYXVsdC5odG0wQAYIKwYBBQUHAgIwNB4yIB0ATABlAGcAYQBsAF8AUABvAGwAaQBjAHkAXwBTAHQAYQB0AGUAbQBlAG4AdAAuIB0wDQYJKoZIhvcNAQELBQADggIBAAfmiFEN4sbgmD+BcQM9naOhIW+z66bM9TG+zwXiqf76V20ZMLPCxWbJat/15/B4vceoniXj+bzta1RXCCtRgkQS+7lTjMz0YBKKdsxAQEGb3FwX/1z5Xhc1mCRWS3TvQhDIr79/xn/yN31aPxzymXlKkVIArzgPF/UveYFl2am1a+THzvbKegBvSzBEJCI8z+0DpZaPWSm8tv0E4XCfMkon/VWvL/625Y4zu2JfmttXQOnxzplmkIz/amJ/3cVKC5Em4jnsGUpxY517IW3DnKOiPPp/fZZqkHimbdLhnPkd/DjYlPTGpQqWhqS9nhquBEKDuLWAmyI4ILUl5WTs9/S/fmNZJQ96LjlXdqJxqgaKD4kWumGnEcua2A5HmoDF0M2n0O99g/DhO3EJ3110mCIIYdqwUB5vvfHhAN/nMQekkzr3ZUd46PioSKv33nJ+YWtvd6mBy6cJrDm77MbL2IK0cs0d9LiFAR6A+xuJKlQ5slvayA1VmXqHczsI5pgt6o3gMy4SKfXAL1QnIffIrE7aKLixqduWsqdCosnPGUFN4Ib5KpqjEWYw07t0MkvfY3v1mYovG8chr1m1rtxEPJdQcdeh0sVV42neV8HR3jDA/czmTfsNv11P6Z0eGTgvvM9YBS7vDaBQNdrvCScc1bN+NR4Iuto229Nfj950iEkSoYICzjCCAjcCAQEwgfihgdCkgc0wgcoxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJXQTEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMS0wKwYDVQQLEyRNaWNyb3NvZnQgSXJlbGFuZCBPcGVyYXRpb25zIExpbWl0ZWQxJjAkBgNVBAsTHVRoYWxlcyBUU1MgRVNOOkQwODItNEJGRC1FRUJBMSUwIwYDVQQDExxNaWNyb3NvZnQgVGltZS1TdGFtcCBzZXJ2aWNloiMKAQEwBwYFKw4DAhoDFQByQCUheEOevaI9Zc/3QGrkX42iC6CBgzCBgKR+MHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMA0GCSqGSIb3DQEBBQUAAgUA36ppYDAiGA8yMDE4MTEyOTIxMzQyNFoYDzIwMTgxMTMwMjEzNDI0WjB3MD0GCisGAQQBhFkKBAExLzAtMAoCBQDfqmlgAgEAMAoCAQACAitfAgH/MAcCAQACAhGtMAoCBQDfq7rgAgEAMDYGCisGAQQBhFkKBAIxKDAmMAwGCisGAQQBhFkKAwKgCjAIAgEAAgMHoSChCjAIAgEAAgMBhqAwDQYJKoZIhvcNAQEFBQADgYEAbAXXPR9wy4NA0892GGqetaZF+pNClpGcfEpSuHABaZ4Gzr1nY1nmrhexTtr/U6omHALRWzkQwthk0cy+mnEHXyOZGmoEEpgrLgK3AAP5NbK/XbtHQRyZJQyhZScFbOyQycoE8QQalSVOhWxk/bbBMQaQiYVMIexNd/T0KgaDDUMxggMNMIIDCQIBATCBkzB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMAITMwAAAOIYOHtm6erB2AAAAAAA4jANBglghkgBZQMEAgEFAKCCAUowGgYJKoZIhvcNAQkDMQ0GCyqGSIb3DQEJEAEEMC8GCSqGSIb3DQEJBDEiBCCr9IiSbx6s8MLdxldRG49+4h6CbicW8hWXAicI3jNmhDCB+gYLKoZIhvcNAQkQAi8xgeowgecwgeQwgb0EIN8BpJSmQCGubWwVa4tW+aMveoHMX/nDnVN8fiDOMsrLMIGYMIGApH4wfDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEmMCQGA1UEAxMdTWljcm9zb2Z0IFRpbWUtU3RhbXAgUENBIDIwMTACEzMAAADiGDh7ZunqwdgAAAAAAOIwIgQgTkOfRvGEZNbr5/hgWclsL4/Q7SOZihE/U0lz2wEMIGcwDQYJKoZIhvcNAQELBQAEggEATlxnCfTzFfTMDvK085zlYPVCroKYW6gKFYnbAhNmrNzcxqALKmIYXpFU7B6HH/vYzkUfCyXpf5tsyEWu0oTySOjyAZ9+2vdaG8nEgjOp0L737lcitgusIjpWtta3Ik0b+mzffnvyjrgTSuKDDni3mxGfvJU77k1Ctempma4H2FJso6Bur0PRH99vIYDu4lHigOSLbeyjR5CiDciBwEVUSA0FxhoFNX1yfpxz3sukOvkaoTduREIjH5LxUjNI1ZTMK/ZkeETI8IPRpWVzAc8q7CujErHKo4sdKej/O2cfUTUHplFLVCGGExpJUCg5FH5jVUUFt75ad8503sdGplggVQ== + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + MIIucgYJKoZIhvcNAQcCoIIuYzCCLl8CAQExDzANBglghkgBZQMEAgEFADCCARAGCSsGAQQBgjcKAaCCAQEwgf4wDAYKKwYBBAGCNwwBAQQQ293slyDTMUOmVyeQqmcNSBcNMjIwMjA0MDkwNjU2WjAOBgorBgEEAYI3DAEDBQAwgbwwKgQUdgFpbn9QXi/ly4CZFKA2Eimoq6YxEjAQBgorBgEEAYI3DAIDMQKCADCBjQQg3YHMbvV3unL0mx/RJ8ihTJd1C/SYSnMHbN0yMrWijuQxaTAQBgorBgEEAYI3DAIDMQKCADBVBgorBgEEAYI3AgEEMUcwRTAQBgorBgEEAYI3AgEZogKAADAxMA0GCWCGSAFlAwQCAQUABCDdgcxu9Xe6cvSbH9EnyKFMl3UL9JhKcwds3TIytaKO5KCCE8owggYEMIID7KADAgECAhMzAA/d9uEraWrDsxZGAAAAD932MA0GCSqGSIb3DQEBCwUAMIGLMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMQwwCgYDVQQLEwNBT0MxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAyMjAeFw0yMjAyMDQwODU5MzZaFw0yMjAyMDcwODU5MzZaMC8xLTArBgNVBAMTJDQ2ZTIwYzY2LTUxNTEtNDNhOS05MWQ4LTMwNDY0ZGZhMDg4ZjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALaMR5Ajlp2POidTxT0N47z5sukiE9fqMuI1QimTXIfB2/gPdMsNUbVCD57ha8J1DUJQpO5nuunbk3SONwfDTdWKx9u7zWubpEmgG7hZ8LTKFlF+xhS6lS/FBSSHyXpdScWPg62BMnDysqHMH/AjLw26HNPRWK8A0vx9jsalFwLg15u4MPuKN3Bpawr/OL0B+7eh/dGO3PutAqJ4aZs2lUCIyODg0q3Tzhgi7SIvFacFWJ8Qj6+D3AfasOv8oanfpNLLPhPlxXGEK9sMKHOOb8mTU9V/ibERqEKTHkJ24Vu+BwrXq5eVedtwmHT2WWR7teaIvrUT9AiPmMhx4hIF/0MCAwEAAaOCAbowggG2MBgGA1UdJQEB/wQOMAwGCisGAQQBgjdMHAEwHQYDVR0OBBYEFB4NpkpeSLk/j691tPw97rB4XKDJMA0GA1UdEQQGMASCAlVTMB8GA1UdIwQYMBaAFPeC7EkBPjJR1eFAcP4hAMeQ72IPMF8GA1UdHwRYMFYwVKBSoFCGTmh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY3JsL01pY3Jvc29mdCUyME1hcmtldHBsYWNlJTIwQ0ElMjBHJTIwMDIyLmNybDBsBggrBgEFBQcBAQRgMF4wXAYIKwYBBQUHMAKGUGh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY2VydHMvTWljcm9zb2Z0JTIwTWFya2V0cGxhY2UlMjBDQSUyMEclMjAwMjIuY3J0MAwGA1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgXgMDwGCSsGAQQBgjcVBwQvMC0GJSsGAQQBgjcVCIOS9kTqrxCDkY0wgqzgLIKinDE0g+6NOIaE7wACAWQCARcwIAYJKwYBBAGCNxUKAQH/BBAwDjAMBgorBgEEAYI3TBwBMA0GCSqGSIb3DQEBCwUAA4ICAQBAiJuIJeMS+PMrxG+kwPWT7yy7A6P8trchwtXyYep/WXu7xT2tH2ys3wuP4DYG5CY4nKekjTKraQhZGuYj4r4KrUviShe8ZWMIcK4MRduXAU81UOd/fsCq5djQ6NDN4Bzq1TxjwZKiEMEWmXaunj6XAlDSX3SgyyMb/ywmur8VVKO+xFVkOxAqCjNo1VaCv3zEvr3Y6dUwO9gYQui1bAQVQzdxRYUPEJlsFBmbNL1AnZ39r4/9K/6orxavU4qwkj4cXnhtMYKDjTIIbTBrfD4glP6mnmkcZpN0ItfEAVgU9rGpXg84hQLeP83nBs2Y8DrF3bBF9867dJCl90c5rK1DMcelmH3oUAwqZ3U+jIIj3HGyfefAQ6HPL0yiY9OYLXQZUxcZF13m92l5s7dy2C21cIh0W0iTcWPIhNn9cia4Hr7FabTC0RoQkO1bg0PKvPMnoe0AqEVmPKImNDncg538AjPy2qeUxn7+0kcuZWEdKVAPwWfzh5qvSAmYKk4XNBfN2E40GQ0ruRdvMVMFeYBLXKm4SJR6zoiBa5v7qzt1uQqZf6GD4BNFZivBp0P2B9lTubEAU6SmUsazRnEVmekIO+x7gDuXbMS8tv6mb1pbGCtIV/wRcIYNcDkHKTHMbW5xR/OmGTM3mPRxoMQtaWtxIeagZkU2nl+pElA/0h7pIjCCBtcwggS/oAMCAQICCmESRKIAAAAAAAIwDQYJKoZIhvcNAQELBQAwgYgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xMjAwBgNVBAMTKU1pY3Jvc29mdCBSb290IENlcnRpZmljYXRlIEF1dGhvcml0eSAyMDExMB4XDTExMDMyODIxMDkzOVoXDTMxMDMyODIxMTkzOVowfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAubUaSwGYVsE3MAnPfvmozUhAB3qxBABgJRW1vDp4+tVinXxD32f7k1K89JQ6zDOgS/iDgULC+yFK1K/1Qjac/0M7P6c8v5LSjnWGlERLa/qY32j46S7SLQcit3g2jgoTTO03eUG+9yHZUTGV/FJdRYB8uXhrznJBa+Y+yGwiQKF+m6XFeBH/KORoKFx+dmMoy9EWJ/m/o9IiUj2kzm9C691+vZ/I2w0Bj93W9SPPkV2PCNHlzgfIAoeajWpHmi38Wi3xZHonkzAVBHxPsCBppOoNsWvmAfUM7eBthkSPvFruekyDCPNEYhfGqgqtqLkoBebXLZCOVybF7wTQaLvse60//3P003icRcCoQYgY4NAqrF7j80o5U7DkeXxcB0xvengsaKgiAaV1DKkRbpe98wCqr1AASvm5rAJUYMU+mXmOieV2EelY2jGrenWe9FQpNXYV1NoWBh0WKoFxttoWYAnF705bIWtSZsz08ZfK6WLX4GXNLcPBlgCzfTm1sdKYASWdBbH2haaNhPapFhQQBJHKwnVW2iXErImhuPi45W3MVTZ5D9ASshZx69cLYY6xAdIa+89Kf/uRrsGOVZfahDuDw+NI183iAyzC8z/QRt2P32LYxP0xrCdqVh+DJo2i4NoE8Uk1usCdbVRuBMBQl/AwpOTq7IMvHGElf65CqzUCAwEAAaOCAUswggFHMBAGCSsGAQQBgjcVAQQDAgEAMB0GA1UdDgQWBBQPU8s/FmEl/mCJHdO5fOiQrbOU0TAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBRyLToCMZBDuRQFTuHqp8cx0SOJNDBaBgNVHR8EUzBRME+gTaBLhklodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3JsMF4GCCsGAQUFBwEBBFIwUDBOBggrBgEFBQcwAoZCaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3J0MA0GCSqGSIb3DQEBCwUAA4ICAQCjuZmM8ZVNDgp9wHsL4RY8KJ8nLinvxFTphNGCrxaLknkYG5pmMhVlX+UB/tSiW8W13W60nggz9u5xwMx7v/1t/Tgm6g2brVyOKI5A7u6/2SIJwkJKFw953K0YIKVT28w9zl8dSJnmRnyR0G86ncWbF6CLQ6A6lBQ9o2mTGVqDr4m35WKAnc6YxUUM1y74mbzFFZr63VHsCcOp3pXWnUqAY1rb6Q6NX1b3clncKqLFm0EjKHcQ56grTbwuuB7pMdh/IFCJR01MQzQbDtpEisbOeZUi43YVAAHKqI1EO9bRwg3frCjwAbml9MmI4utMW94gWFgvrMxIX+n42RBDIjf3Ot3jkT6gt3XeTTmO9bptgblZimhERdkFRUFpVtkocJeLoGuuzP93uH/Yp032wzRH+XmMgujfZv+vnfllJqxdowoQLx55FxLLeTeYfwi/xMSjZO2gNven3U/3KeSCd1kUOFS3AOrwZ0UNOXJeW5JQC6Vfd1BavFZ6FAta1fMLu3WFvNB+FqeHUaU3ya7rmtxJnzk29DeSqXgGNmVSywBS4NajI5jJIKAA6UhNJlsg8CHYwUOKf5ej8OoQCkbadUxXygAfxCfW2YBbujtI+PoyejRFxWUjYFWO5LeTI62UMyqfOEiqugoYjNxmQZla2s4YHVuqIC34R85FQlg9pKQBsDCCBuMwggTLoAMCAQICEzMAAABLZz0Eed1JFCkAAAAAAEswDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMB4XDTE5MDQxODIwNDUwNFoXDTI0MDQxODIwNDUwNFowgYsxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xDDAKBgNVBAsTA0FPQzEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldHBsYWNlIENBIEcgMDIyMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAzGfV5tDxZ3oPG8aq2OwrBLSRFC61lYvWtkqOdKswlBHVncuoOIzekLIFNsMwluoK0plNZ5w/JPmwkGTGIrt9ftbWQz0k7XRfacXvQQfhCaBgjkybvvDx7CTL0UDEUiXfEEkylUcQKOdm9LApPDP4oO/V8RS4ugkqkjsuVaOyfYv0TytjVVQon72W+wR/AyrBAXgfwzzEb85403GTJRzQMlTw1YqdgS/o9SCsvH9dRZTGlWCatIl+a00eTG5zgeu0xCtqwgERhw9UT6mlPqmp6RqJ8XMgKylh9Ss+jI53EmraAFlelph16kuqF6n3vNnRBfhQ+gmNdtKi9s3jI8Di7ip8hYGPHGGhtcw0AYUg5r/VQhIfXxO45zO7SFvjujX0ji0b3WKB/xD9Sg1KO1fiTpj82ifyPqHvL+iwn5dV98C4ru0dQ0hfK7tA4K5qkW+2gym0fBjGYWq81/smrj6LbPdNqCotgTMU4dCMBhmSJOD9fSiPUybrbHJrHGR5YNs5VJbHH5u1ia8UrgsSrGTb1bh38Tb10matJpBI7NwDWTkV7kMvgNRQsGO/1+dg47PLW3X7TbiZniTPC+oAqOh1CiF+ODnem4Nw1iRw854LN3yqibLHgTi093mCBtPGWZxg2sTcHUX4fo/UHNbSh8bERJznnennm4gEJXAYtpnEFSUCAwEAAaOCAUswggFHMA4GA1UdDwEB/wQEAwIBhjAQBgkrBgEEAYI3FQEEAwIBADAdBgNVHQ4EFgQU94LsSQE+MlHV4UBw/iEAx5DvYg8wGQYJKwYBBAGCNxQCBAweCgBTAHUAYgBDAEEwEgYDVR0TAQH/BAgwBgEB/wIBADAfBgNVHSMEGDAWgBQPU8s/FmEl/mCJHdO5fOiQrbOU0TBXBgNVHR8EUDBOMEygSqBIhkZodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNNYXJQQ0EyMDExXzIwMTEtMDMtMjguY3JsMFsGCCsGAQUFBwEBBE8wTTBLBggrBgEFBQcwAoY/aHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNNYXJQQ0EyMDExXzIwMTEtMDMtMjguY3J0MA0GCSqGSIb3DQEBCwUAA4ICAQAbTWps31OnBv77EVpCaU68drubToSKtpw9fYt4MvAhorXGRrRRfS6qou5o45QZAKjwzbOrr33dWxF4P3gCGqZxCotW32p6hlXGP7BnYlVw6242nlX0PahgpyWVWBmi0ElKH8N6HYhJqAW74tUNCzY7ALNMwk/JVEMKpWq+5XqutAEmtP0SdoSZPRwPtdiRjMLS8yyjP4HVAiKJ+m0wV3bLBtCG20BttEGkly7fYa2/E0ps41PLeX3T2yKRgmnGjhD16NUcMDdRpBh7MCXktwpISDEanB8QySJNdErU+NaKukgOW68+Oyk7lGiiZcG3boCGNec9JkM7dQdpA4tneQg0FI/o0mkDZR5aVItbAjRfMgAXpm6Hfu9DqOP+/R0K9v+CwyhwEjF3+SeL8YBAijXHD5YaiWzwZzBDtNQpuOXc91NmF7ifdHVy9IE1EhuIIQsMio3l0Uo87vGdwQ50po8o1F72zi1vfvKfDmszVezk6fF7cy7fZYh8I0ceVbZ4XY4xJ76TdYMItUAoPkvVdDq5B/d4oI7/xHfomRsEw08fuLxCRE8/BcvzSlDpnxu4wL6IULqVwHE+fbQ82Lr5ZDLxa1dYUw8zBGgCnx/ObVPj1RO1CaVi13wCxm8yaHeflKJWw7ZHOLeOGQyMRT+f9cFFk+5fzCA9GcwiSlEyYj4ilzGCGWUwghlhAgEBMIGjMIGLMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMQwwCgYDVQQLEwNBT0MxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAyMgITMwAP3fbhK2lqw7MWRgAAAA/d9jANBglghkgBZQMEAgEFAKCBjzAYBgkqhkiG9w0BCQMxCwYJKwYBBAGCNwoBMC8GCSqGSIb3DQEJBDEiBCBDgOMhvMousTIzk/P1eC2qGiSJLdSm3brGanFMLa41+DBCBgorBgEEAYI3AgEMMTQwMqAUgBIATQBpAGMAcgBvAHMAbwBmAHShGoAYaHR0cDovL3d3dy5taWNyb3NvZnQuY29tMA0GCSqGSIb3DQEBAQUABIIBAJ8LEwzLh2rDfLOOOodKT10yXoCzHPaQLv9sMwei+650mqV0iHaw9bUolBZAdBnF7JBRjY+yb9AIF7M/Av9pHRKvuLHWDntlo49RrfvNucHaJqPjCMm81h/GAuimPSBMpRtTCW+HHDWrw+HoGbRWQ+N/Vpt//pzj9gAzOzs8W9gU0RMz3ub4Hfgunm/CGACjFu5Hi2sjJbgigNlyRnxjI4+h3J7dGVOfR8hlhRB9YjXDB700F0bzaKqxC9LH7kZbAjiHPgLLCWz7OQ/rTo7wV5/e6v9GCKpamydPy/0MObq4TDi0fn75aVHTJJrzlwt1BJKyVPdAsOhiFUH3Mq4WheKhghcAMIIW/AYKKwYBBAGCNwMDATGCFuwwghboBgkqhkiG9w0BBwKgghbZMIIW1QIBAzEPMA0GCWCGSAFlAwQCAQUAMIIBUQYLKoZIhvcNAQkQAQSgggFABIIBPDCCATgCAQEGCisGAQQBhFkKAwEwMTANBglghkgBZQMEAgEFAAQgr34HNu5gF3j5DAITqq38vc6sNxs6ZqLz9i9zU2oQDtoCBmH66bcTCRgTMjAyMjAyMDQwOTA5NDkuNTkxWjAEgAIB9KCB0KSBzTCByjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjElMCMGA1UECxMcTWljcm9zb2Z0IEFtZXJpY2EgT3BlcmF0aW9uczEmMCQGA1UECxMdVGhhbGVzIFRTUyBFU046REQ4Qy1FMzM3LTJGQUUxJTAjBgNVBAMTHE1pY3Jvc29mdCBUaW1lLVN0YW1wIFNlcnZpY2WgghFXMIIHDDCCBPSgAwIBAgITMwAAAZwPpk1h0p5LKAABAAABnDANBgkqhkiG9w0BAQsFADB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMDAeFw0yMTEyMDIxOTA1MTlaFw0yMzAyMjgxOTA1MTlaMIHKMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSUwIwYDVQQLExxNaWNyb3NvZnQgQW1lcmljYSBPcGVyYXRpb25zMSYwJAYDVQQLEx1UaGFsZXMgVFNTIEVTTjpERDhDLUUzMzctMkZBRTElMCMGA1UEAxMcTWljcm9zb2Z0IFRpbWUtU3RhbXAgU2VydmljZTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBANtSKgwZXUkWP6zrXazTaYq7bco9Q2zvU6MN4ka3GRMX2tJZOK4DxeBiQACL/n7YV/sKTslwpD0f9cPU4rCDX9sfcTWo7XPxdHLQ+WkaGbKKWATsqw69bw8hkJ/bjcp2V2A6vGsvwcqJCh07BK3JPmUtZikyy5PZ8fyTyiKGN7hOWlaIU9oIoucUNoAHQJzLq8h20eNgHUh7eI5k+Kyq4v6810LHuA6EHyKJOZN2xTw5JSkLy0FN5Mhg/OaFrFBl3iag2Tqp4InKLt+Jbh/Jd0etnei2aDHFrmlfPmlRSv5wSNX5zAhgEyRpjmQcz1zp0QaSAefRkMm923/ngU51IbrVbAeHj569SHC9doHgsIxkh0K3lpw582+0ONXcIfIU6nkBT+qADAZ+0dT1uu/gRTBy614QAofjo258TbSX9aOU1SHuAC+3bMoyM7jNdHEJROH+msFDBcmJRl4VKsReI5+S69KUGeLIBhhmnmQ6drF8Ip0ZiO+vhAsD3e9AnqnY7Hcge850I9oKvwuwpVwWnKnwwSGElMz7UvCocmoUMXk7Vn2aNti+bdH28+GQb5EMsqhOmvuZOCRpOWN33G+b3g5unwEP0eTiY+LnWa2AuK43z/pplURJVle29K42QPkOcglB6sjLmNpEpb9basJ72eA0Mlp1LtH3oYZGXsggTfuXAgMBAAGjggE2MIIBMjAdBgNVHQ4EFgQUu2kJZ1Ndjl2112SynL6jGMID+rIwHwYDVR0jBBgwFoAUn6cVXQBeYl2D9OXSZacbUzUZ6XIwXwYDVR0fBFgwVjBUoFKgUIZOaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9jcmwvTWljcm9zb2Z0JTIwVGltZS1TdGFtcCUyMFBDQSUyMDIwMTAoMSkuY3JsMGwGCCsGAQUFBwEBBGAwXjBcBggrBgEFBQcwAoZQaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9jZXJ0cy9NaWNyb3NvZnQlMjBUaW1lLVN0YW1wJTIwUENBJTIwMjAxMCgxKS5jcnQwDAYDVR0TAQH/BAIwADATBgNVHSUEDDAKBggrBgEFBQcDCDANBgkqhkiG9w0BAQsFAAOCAgEApwAqpiMYRzNNYyz3PSbtijbeyCpUXcvIrqA4zPtMIcAk34W9u9mRDndWS+tlR3WwTpr1OgaV1wmc6YFzqK6EGWm903UEsFE7xBJMPXjfdVOPhcJB3vfvA0PX56oobcF2OvNsOSwTB8bi/ns+Cs39Puzs+QSNQZd8iAVBCSvxNCL78dln2RGU1xyB4AKqV9vi4Y/Gfmx2FA+jF0y+YLeob0M40nlSxL0q075t7L6iFRMNr0u8ROhzhDPLl+4ePYfUmyYJoobvydel9anAEsHFlhKl+aXb2ic3yNwbsoPycZJL/vo8OVvYYxCy+/5FrQmAvoW0ZEaBiYcKkzrNWt/hX9r5KgdwL61x0ZiTZopTko6W/58UTefTbhX7Pni0MApH3Pvyt6N0IFap+/LlwFRD1zn7e6ccPTwESnuo/auCmgPznq80OATA7vufsRZPvqeX8jKtsraSNscvNQymEWlcqdXV9hYkjb4T/Qse9cUYaoXg68wFHFuslWfTdPYPLl1vqzlPMnNJpC8KtdioDgcq+y1BaSqSm8EdNfwzT37+/JFtVc3Gs915fDqgPZDgOSzKQIV+fw3aPYt2LET3AbmKKW/r13Oy8cg3+D0D362GQBAJVv0NRI5NowgaCw6oNgWOFPrN72WSEcca/8QQiTGP2XpLiGpRDJZ6sWRpRYNdydkwggdxMIIFWaADAgECAhMzAAAAFcXna54Cm0mZAAAAAAAVMA0GCSqGSIb3DQEBCwUAMIGIMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMTIwMAYDVQQDEylNaWNyb3NvZnQgUm9vdCBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkgMjAxMDAeFw0yMTA5MzAxODIyMjVaFw0zMDA5MzAxODMyMjVaMHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA5OGmTOe0ciELeaLL1yR5vQ7VgtP97pwHB9KpbE51yMo1V/YBf2xK4OK9uT4XYDP/XE/HZveVU3Fa4n5KWv64NmeFRiMMtY0Tz3cywBAY6GB9alKDRLemjkZrBxTzxXb1hlDcwUTIcVxRMTegCjhuje3XD9gmU3w5YQJ6xKr9cmmvHaus9ja+NSZk2pg7uhp7M62AW36MEBydUv626GIl3GoPz130/o5Tz9bshVZN7928jaTjkY+yOSxRnOlwaQ3KNi1wjjHINSi947SHJMPgyY9+tVSP3PoFVZhtaDuaRr3tpK56KTesy+uDRedGbsoy1cCGMFxPLOJiss254o2I5JasAUq7vnGpF1tnYN74kpEeHT39IM9zfUGaRnXNxF803RKJ1v2lIH1+/NmeRd+2ci/bfV+AutuqfjbsNkz2K26oElHovwUDo9Fzpk03dJQcNIIP8BDyt0cY7afomXw/TNuvXsLz1dhzPUNOwTM5TI4CvEJoLhDqhFFG4tG9ahhaYQFzymeiXtcodgLiMxhy16cg8ML6EgrXY28MyTZki1ugpoMhXV8wdJGUlNi5UPkLiWHzNgY1GIRH29wb0f2y1BzFa/ZcUlFdEtsluq9QBXpsxREdcu+N+VLEhReTwDwV2xo3xwgVGD94q0W29R6HXtqPnhZyacaue7e3PmriLq0CAwEAAaOCAd0wggHZMBIGCSsGAQQBgjcVAQQFAgMBAAEwIwYJKwYBBAGCNxUCBBYEFCqnUv5kxJq+gpE8RjUpzxD/LwTuMB0GA1UdDgQWBBSfpxVdAF5iXYP05dJlpxtTNRnpcjBcBgNVHSAEVTBTMFEGDCsGAQQBgjdMg30BATBBMD8GCCsGAQUFBwIBFjNodHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpb3BzL0RvY3MvUmVwb3NpdG9yeS5odG0wEwYDVR0lBAwwCgYIKwYBBQUHAwgwGQYJKwYBBAGCNxQCBAweCgBTAHUAYgBDAEEwCwYDVR0PBAQDAgGGMA8GA1UdEwEB/wQFMAMBAf8wHwYDVR0jBBgwFoAU1fZWy4/oolxiaNE9lJBb186aGMQwVgYDVR0fBE8wTTBLoEmgR4ZFaHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraS9jcmwvcHJvZHVjdHMvTWljUm9vQ2VyQXV0XzIwMTAtMDYtMjMuY3JsMFoGCCsGAQUFBwEBBE4wTDBKBggrBgEFBQcwAoY+aHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNSb29DZXJBdXRfMjAxMC0wNi0yMy5jcnQwDQYJKoZIhvcNAQELBQADggIBAJ1VffwqreEsH2cBMSRb4Z5yS/ypb+pcFLY+TkdkeLEGk5c9MTO1OdfCcTY/2mRsfNB1OW27DzHkwo/7bNGhlBgi7ulmZzpTTd2YurYeeNg2LpypglYAA7AFvonoaeC6Ce5732pvvinLbtg/SHUB2RjebYIM9W0jVOR4U3UkV7ndn/OOPcbzaN9l9qRWqveVtihVJ9AkvUCgvxm2EhIRXT0n4ECWOKz3+SmJw7wXsFSFQrP8DJ6LGYnn8AtqgcKBGUIZUnWKNsIdw2FzLixre24/LAl4FOmRsqlb30mjdAy87JGA0j3mSj5mO0+7hvoyGtmW9I/2kQH2zsZ0/fZMcm8Qq3UwxTSwethQ/gpY3UA8x1RtnWN0SCyxTkctwRQEcb9k+SS+c23Kjgm9swFXSVRk2XPXfx5bRAGOWhmRaw2fpCjcZxkoJLo4S5pu+yFUa2pFEUep8beuyOiJXk+d0tBMdrVXVAmxaQFEfnyhYWxz/gq77EFmPWn9y8FBSX5+k77L+DvktxW/tM4+pTFRhLy/AsGConsXHRWJjXD+57XQKBqJC4822rpM+Zv/Cuk0+CQ1ZyvgDbjmjJnW4SLq8CdCPSWU5nR0W2rRnj7tfqAxM328y+l7vzhwRNGQ8cirOoo6CGJ/2XBjU02N7oJtpQUQwXEGahC0HVUzWLOhcGbyoYICzjCCAjcCAQEwgfihgdCkgc0wgcoxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJTAjBgNVBAsTHE1pY3Jvc29mdCBBbWVyaWNhIE9wZXJhdGlvbnMxJjAkBgNVBAsTHVRoYWxlcyBUU1MgRVNOOkREOEMtRTMzNy0yRkFFMSUwIwYDVQQDExxNaWNyb3NvZnQgVGltZS1TdGFtcCBTZXJ2aWNloiMKAQEwBwYFKw4DAhoDFQDN2Wnq3fCz9ucStub1zQz7129TQKCBgzCBgKR+MHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMA0GCSqGSIb3DQEBBQUAAgUA5adiZjAiGA8yMDIyMDIwNDE2MjkyNloYDzIwMjIwMjA1MTYyOTI2WjB3MD0GCisGAQQBhFkKBAExLzAtMAoCBQDlp2JmAgEAMAoCAQACAh92AgH/MAcCAQACAhFfMAoCBQDlqLPmAgEAMDYGCisGAQQBhFkKBAIxKDAmMAwGCisGAQQBhFkKAwKgCjAIAgEAAgMHoSChCjAIAgEAAgMBhqAwDQYJKoZIhvcNAQEFBQADgYEAbKzHReDTwgAKGSkgO/zxCAV/RcxwGpZK4M2bWgwqHVIi+/xiAKdQOSKeMz3UwRO4LI4EN4BjeK+ATpruzOtVGkyA8yzwdJFPv99j4O/Hc5kJSvovfECah62tBB3siNIFaAYNK+BF76k3j4+y9q9lF5yjQ8ZylVYbOfOjlJ2uevsxggQNMIIECQIBATCBkzB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMAITMwAAAZwPpk1h0p5LKAABAAABnDANBglghkgBZQMEAgEFAKCCAUowGgYJKoZIhvcNAQkDMQ0GCyqGSIb3DQEJEAEEMC8GCSqGSIb3DQEJBDEiBCD10XWSucJFd8jLe5PaC9h9/GHXK7ZZ0kb4BGFNqgWk9DCB+gYLKoZIhvcNAQkQAi8xgeowgecwgeQwgb0EIDcPRYUgjSzKOhF39d4QgbRZQgrPO7Lo/qE5GtvSeqa8MIGYMIGApH4wfDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEmMCQGA1UEAxMdTWljcm9zb2Z0IFRpbWUtU3RhbXAgUENBIDIwMTACEzMAAAGcD6ZNYdKeSygAAQAAAZwwIgQgPTEmgx6rn5xpH+xmlpYkcn0dL16GyYRqyG8uRajrfTYwDQYJKoZIhvcNAQELBQAEggIAxmSLisdd5cs+7liXX7vsF8zX3Y1rbtyPZ0M466WBaVULlfeuu9RMzOhWha7bkfYmKs5hXOQNBJrIHxnHbWeDu/V7EVF80IQ7/6qWiVPNcLtMnS8+uyw3Tby2gEhshO/HHOJRqllrgtUGuPl8FoAAPbPKV7X7jrg1kqOWoqpwyiSzt5cN/qijwfTcTIaPeON5wMd8wRT2l5F9e/JDBSq0WuzYb2r0kifv9wq21orm4sQkbg5GgDa/oCr3PQVCNSvPb2G7HANcqJXHeRKyoCeVhX5UaFrCQbdxGFytOc5GzptQKNn18yGg4VYrFpigUcglLbuREqqlPabwZh/QfgKdp4b4uVA6On5HIV5e9XdJ9TXhvDArEQ+Kgyao5xlgb4MkvPO9Thh9UIbw9r3JhQ+uKnCvXyiE2SpTLq1OXTjt/RrLZb+qsZW4nrDRBlBswa4G5OVW/svvI8N2/+YYrSdaCWt4unWSnSxeVUWZkugBmWdeoEJpfs5Qy8NwC+g1TyXcMjolOFTfBB6JYpHCbk6Euh2gKxq5weBb7puxVN4cvawYY2zorp6ZXtAmQGflZTl02nWld+nvDoF/zi6IUmP7PTKvbEVs8T+QkCiIM7cpLMs4NFK29f33lu1iZeGwJAxCYsV/P7geuRfA++bpDutTXjLKMqUTgf00sqJ/5HGuwLE= From webhook-mailer at python.org Fri Feb 4 11:34:22 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 04 Feb 2022 16:34:22 -0000 Subject: [Python-checkins] bpo-46629: Update classicAppCompat.sccd for new signing certificate (GH-31111) Message-ID: https://github.com/python/cpython/commit/7b5b23c31db87cfc5e09ec297147499ba1d47d31 commit: 7b5b23c31db87cfc5e09ec297147499ba1d47d31 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-04T08:34:12-08:00 summary: bpo-46629: Update classicAppCompat.sccd for new signing certificate (GH-31111) (cherry picked from commit 9b4e3d94a5746af093392ed8e977b26fcc1bfd11) Co-authored-by: Steve Dower files: M PC/classicAppCompat.can.xml M PC/classicAppCompat.sccd diff --git a/PC/classicAppCompat.can.xml b/PC/classicAppCompat.can.xml index f00475c8da312..df361f8e3e2cd 100644 --- a/PC/classicAppCompat.can.xml +++ b/PC/classicAppCompat.can.xml @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file diff --git a/PC/classicAppCompat.sccd b/PC/classicAppCompat.sccd index 97648985a2ccb..d7a70cdd0533c 100644 --- a/PC/classicAppCompat.sccd +++ b/PC/classicAppCompat.sccd @@ -1,28 +1,41 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - MIIq5AYJKoZIhvcNAQcCoIIq1TCCKtECAQExDzANBglghkgBZQMEAgEFADCCARAGCSsGAQQBgjcKAaCCAQEwgf4wDAYKKwYBBAGCNwwBAQQQaM+L42jwBUGvBczrtolMmhcNMTgxMTMwMDA1OTAzWjAOBgorBgEEAYI3DAEDBQAwgbwwKgQUWKcU3R38DGPlKK33XGIwKtVL1r4xEjAQBgorBgEEAYI3DAIDMQKCADCBjQQg3K+KBOQX7HfxjRNZC9cx8gIPkEhPRO1nJFRdWQrVEJ4xaTAQBgorBgEEAYI3DAIDMQKCADBVBgorBgEEAYI3AgEEMUcwRTAQBgorBgEEAYI3AgEZogKAADAxMA0GCWCGSAFlAwQCAQUABCDcr4oE5Bfsd/GNE1kL1zHyAg+QSE9E7WckVF1ZCtUQnqCCFFAwggZSMIIEOqADAgECAhMzAAMu49KhfNamygpWAAIAAy7jMA0GCSqGSIb3DQEBCwUAMIGMMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMQ0wCwYDVQQLEwRNT1BSMScwJQYDVQQDEx5NaWNyb3NvZnQgTWFya2V0cGxhY2UgQ0EgRyAwMTMwHhcNMTgxMTMwMDA1NTA1WhcNMTgxMjAzMDA1NTA1WjB0MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMR4wHAYDVQQDExVNaWNyb3NvZnQgQ29ycG9yYXRpb24wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCwpcimfAx3HEpba1GLL/gDaRVddHE5PXTRmwlgaz8kt6/rq5rlrPFnCnbIc5818v0xJIznastbmrq26xyCEHyMLBKnyneTKE36I7+TGjcY0D7ow+o2vY7LDKMCTGlh31fx1Tvrl+5xTbWX5jdLU/3MB5faeOGh+0Knzwx1KDoXWgPtfXnD8I5jxJieoWoCwCjKTJgBOklLy9nbOalxf0h+xQRy2p5fj+PxAwQPgHWft36AF7/IMbt9FcXMtg4xdpnTYz4OV3dFOPz4m3M8HwVgNMv89W/1Ozc7uOyZt0Ij1baT6r2L3IjYg5ftzpGqaDOFcWlyDFSdhMR6BIKW8xEpAgMBAAGjggHCMIIBvjAYBgNVHSUBAf8EDjAMBgorBgEEAYI3TBwBMB0GA1UdDgQWBBRdpGYiCytx83FYzPSl+o97YzpxGzAPBgNVHREECDAGggRNT1BSMB8GA1UdIwQYMBaAFEnYB1RFhpclHtZZcRLDcpt0OE3oMGIGA1UdHwRbMFkwV6BVoFOGUWh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY3JsL01pY3Jvc29mdCUyME1hcmtldHBsYWNlJTIwQ0ElMjBHJTIwMDEzKDIpLmNybDBvBggrBgEFBQcBAQRjMGEwXwYIKwYBBQUHMAKGU2h0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY2VydHMvTWljcm9zb2Z0JTIwTWFya2V0cGxhY2UlMjBDQSUyMEclMjAwMTMoMikuY3J0MAwGA1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgXgMDwGCSsGAQQBgjcVBwQvMC0GJSsGAQQBgjcVCIOS9kTqrxCDkY0wgqzgLIKinDE0g+6NOIaE7wACAWQCARYwIAYJKwYBBAGCNxUKAQH/BBAwDjAMBgorBgEEAYI3TBwBMA0GCSqGSIb3DQEBCwUAA4ICAQB3Dk3rXH52CDq/z1fwqn9xI5WGjGmu6oAE4HSc3sNdFrSVMMGm4gTlYGWSZ0wJUUf16mVr/rdXhxuR3MZn+m4Bhdl8KQqYjYbIvCUVj0o9nZ+yT6foeY8bKnB+K5h6rol+mjDj5IfcutC4x2Kx5RrtDtRTSoKA63iZ74DYngPpBGBBgaS2c/QzgqPRAMMRqy2KBDP0miCnpR3F4YlzHGyOZwyHhESjYd9kwF47+msuHS04JZpnGHIvBppKN9XQzH3WezNnnX3lz4AyAUMsMFuARqEnacUhrAHL9n5zMv9CzxDYN1r1/aDh/788RuGuZM+E3NtmbxJJ7j6T5/VtXNBRgKtIq8d2+11j6qvKLigOTxSC25/A70BZBEvllLFnvc1vA2LrC9drwt1KpSmWie1nvpilw7o+gHMOG9utUxGha2VuVizuVNGCywTRRjvmGS1QqTfaun1URVrLfnDINXuTgN1Vwp0J5IGpJ3D8yj01NDQ/RworE+3W/R531NBYova9QRhU/igEw/Aa/q8wjZ4Pzxr9oBIo0Ta3Tv6qIggaWXw0U9+F0J7SCqIhn0d0ATO+E1Qs/SxZIAICLwmqzoLYUAh8q153esBs4uesueqgt5ueyHK8V3WjMS4wxEyVN5ZMET3hFtEshsZC31tLDdjq750U4SgQVmoYSm3F3ZOKQDCCBtcwggS/oAMCAQICCmESRKIAAAAAAAIwDQYJKoZIhvcNAQELBQAwgYgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xMjAwBgNVBAMTKU1pY3Jvc29mdCBSb290IENlcnRpZmljYXRlIEF1dGhvcml0eSAyMDExMB4XDTExMDMyODIxMDkzOVoXDTMxMDMyODIxMTkzOVowfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAubUaSwGYVsE3MAnPfvmozUhAB3qxBABgJRW1vDp4+tVinXxD32f7k1K89JQ6zDOgS/iDgULC+yFK1K/1Qjac/0M7P6c8v5LSjnWGlERLa/qY32j46S7SLQcit3g2jgoTTO03eUG+9yHZUTGV/FJdRYB8uXhrznJBa+Y+yGwiQKF+m6XFeBH/KORoKFx+dmMoy9EWJ/m/o9IiUj2kzm9C691+vZ/I2w0Bj93W9SPPkV2PCNHlzgfIAoeajWpHmi38Wi3xZHonkzAVBHxPsCBppOoNsWvmAfUM7eBthkSPvFruekyDCPNEYhfGqgqtqLkoBebXLZCOVybF7wTQaLvse60//3P003icRcCoQYgY4NAqrF7j80o5U7DkeXxcB0xvengsaKgiAaV1DKkRbpe98wCqr1AASvm5rAJUYMU+mXmOieV2EelY2jGrenWe9FQpNXYV1NoWBh0WKoFxttoWYAnF705bIWtSZsz08ZfK6WLX4GXNLcPBlgCzfTm1sdKYASWdBbH2haaNhPapFhQQBJHKwnVW2iXErImhuPi45W3MVTZ5D9ASshZx69cLYY6xAdIa+89Kf/uRrsGOVZfahDuDw+NI183iAyzC8z/QRt2P32LYxP0xrCdqVh+DJo2i4NoE8Uk1usCdbVRuBMBQl/AwpOTq7IMvHGElf65CqzUCAwEAAaOCAUswggFHMBAGCSsGAQQBgjcVAQQDAgEAMB0GA1UdDgQWBBQPU8s/FmEl/mCJHdO5fOiQrbOU0TAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBRyLToCMZBDuRQFTuHqp8cx0SOJNDBaBgNVHR8EUzBRME+gTaBLhklodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3JsMF4GCCsGAQUFBwEBBFIwUDBOBggrBgEFBQcwAoZCaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3J0MA0GCSqGSIb3DQEBCwUAA4ICAQCjuZmM8ZVNDgp9wHsL4RY8KJ8nLinvxFTphNGCrxaLknkYG5pmMhVlX+UB/tSiW8W13W60nggz9u5xwMx7v/1t/Tgm6g2brVyOKI5A7u6/2SIJwkJKFw953K0YIKVT28w9zl8dSJnmRnyR0G86ncWbF6CLQ6A6lBQ9o2mTGVqDr4m35WKAnc6YxUUM1y74mbzFFZr63VHsCcOp3pXWnUqAY1rb6Q6NX1b3clncKqLFm0EjKHcQ56grTbwuuB7pMdh/IFCJR01MQzQbDtpEisbOeZUi43YVAAHKqI1EO9bRwg3frCjwAbml9MmI4utMW94gWFgvrMxIX+n42RBDIjf3Ot3jkT6gt3XeTTmO9bptgblZimhERdkFRUFpVtkocJeLoGuuzP93uH/Yp032wzRH+XmMgujfZv+vnfllJqxdowoQLx55FxLLeTeYfwi/xMSjZO2gNven3U/3KeSCd1kUOFS3AOrwZ0UNOXJeW5JQC6Vfd1BavFZ6FAta1fMLu3WFvNB+FqeHUaU3ya7rmtxJnzk29DeSqXgGNmVSywBS4NajI5jJIKAA6UhNJlsg8CHYwUOKf5ej8OoQCkbadUxXygAfxCfW2YBbujtI+PoyejRFxWUjYFWO5LeTI62UMyqfOEiqugoYjNxmQZla2s4YHVuqIC34R85FQlg9pKQBsDCCBxswggUDoAMCAQICEzMAAABCs21EHGjyqKYAAAAAAEIwDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMB4XDTE4MDQyMDE2NDI0NFoXDTIxMDQyMDE2NDI0NFowgYwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xDTALBgNVBAsTBE1PUFIxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAxMzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAOZ2KM9Pq1YCOiqWOivmHjUtkMgznTMP/Mr2YfzZeIIJySg1F4WxFZc4jagGHHNof9NRT+GGnktWsXkZuH1DzQEG4Ps1ln8+4vhbDglqu5ymDnd6RmsyoD+8xfc8bBIvE5o6R+ES4/GVD5TqNsOrWbwETaIZVbmTulJLoTS1WSsSjowmbc+sHqZiY8BNJNThUEmXSjuHqkQKKshuiFWYEqOTitp71mBLyH1wN7/jThRzGpolOeFusRNJdb8sEqvNzEN9Qh+Kp6ndzrnjE+t8ixXW3lShyyOOZqQMwsQn9q9T0v7Q69GuojBTFBOHKwigcCHr4xahuN+ZYMk0xGg+sm3Uj7I9mrWTSTiIRMZNIWq3sFg4+rFg48NYfRlXUpONmL7vXq6v1pIU99d2MXQ6uUrnUr1/n5ZiHGCeFcvWwqO8BYHdcTlrSOkayfFp7W9oCk9QO4Xy0h9cQRedRo2kvdTHxIuJS70Hdv6oePPF2ZFaLucUzzwsR4/XMAVKY8Vsm950omsSSOImsMtzavUdQM+wZFxvHTRqVDkF3quPdME0bCZOWB4hQJmd+o2clw+1mpwPu0/M92nA9FJg7MGPxkFaYW7g26jSqUJZ9AcX+Xa5TSIeqMZt3cRVjMTx0T/v73Sv8TpalqIQ5Fde1+hFK07sOAm3TwgzvlVJnbYgp0/rAgMBAAGjggGCMIIBfjASBgkrBgEEAYI3FQEEBQIDAgACMCMGCSsGAQQBgjcVAgQWBBSbJnDhuc3nQXuKuACsPflEbwjbozAdBgNVHQ4EFgQUSdgHVEWGlyUe1llxEsNym3Q4TegwEQYDVR0gBAowCDAGBgRVHSAAMBkGCSsGAQQBgjcUAgQMHgoAUwB1AGIAQwBBMAsGA1UdDwQEAwIBhjASBgNVHRMBAf8ECDAGAQH/AgEAMB8GA1UdIwQYMBaAFA9Tyz8WYSX+YIkd07l86JCts5TRMFcGA1UdHwRQME4wTKBKoEiGRmh0dHA6Ly9jcmwubWljcm9zb2Z0LmNvbS9wa2kvY3JsL3Byb2R1Y3RzL01pY01hclBDQTIwMTFfMjAxMS0wMy0yOC5jcmwwWwYIKwYBBQUHAQEETzBNMEsGCCsGAQUFBzAChj9odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY01hclBDQTIwMTFfMjAxMS0wMy0yOC5jcnQwDQYJKoZIhvcNAQELBQADggIBAIa2oa6kvuIHCNfz7anlL0W9tOCt8gQNkxOGRK3yliQIelNQahDJojyEFlHQ2BcHL5oZit3WeSDoYddhojx6YzJIWwfGwtVqgc0JFDKJJ2ZXRYMRsuy01Hn25xob+zRMS6VmV1axQn6uwOSMcgYmzoroh6edjPKu7qXcpt6LmhF2qFvLySA7wBCwfI/rR5/PX6I7a07Av7PpbY6/+2ujd8m1H3hwMrb4Hq3z6gcq62zJ3nDXUbC0Bp6Jt2kV9f0rEFpDK9oxE2qrGBUf8c3O2XirHOgAjRyWjWWtVms+MP8qBIA1NSLrBmToEWVP3sEkQZWMkoZWo4rYEJZpX7UIgdDc9zYNakgTCJqPhqn8AE1sgSSnpqAdMkkP41rTlFCv2ig2QVzDerjGfEv+uPDnlAT0kucbBJxHHvUC4aqUxaTSa0sy2bZ6NWFx8/u0gW8JahzxYvvvZL8SfwaA9P4ETb8pH1jw+6N/LfM2zJrNKhf5hjKa0VDOXUpkYq60OqVVnWJ6oJaSIWNkZKfzPnl/UHA8Bh4qfVrhc9H5PExPhhB9WVTsjf4r+OOVuolJldThcWQqljiPjk5rultr63G5xLyFpxNi4BCrcNQBJFB5wKgOWOyjQTVWTmh2ESaeqZ2aWBjftFHlxJ/qYc7WOGJV0+cHGkB/dvFxmKnv6tuWexiMMYIVUTCCFU0CAQEwgaQwgYwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xDTALBgNVBAsTBE1PUFIxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAxMwITMwADLuPSoXzWpsoKVgACAAMu4zANBglghkgBZQMEAgEFAKCBlTAYBgkqhkiG9w0BCQMxCwYJKwYBBAGCNwoBMC8GCSqGSIb3DQEJBDEiBCAS0d3bw2YOODvKFr0S4e3BDnaDcZXUKeBO77yvkWzVojBIBgorBgEEAYI3AgEMMTowOKAegBwATQBpAGMAcgBvAHMAbwBmAHQAIABDAG8AcgBwoRaAFGh0dHA6Ly9NaWNyb3NvZnQuY29tMA0GCSqGSIb3DQEBAQUABIIBABoap3Y+2k+zFz2cCmkc8xxHnpIygLsUSRMXeXdjPVcYx3o5cPLIixnL6p8+LIrlIagPg23mzTEmnjZaO4aaexk+3XojlHj22w/bEigEDnKyWt5bHeS0UNHJbxEFYRfd84IP1+mSH4c4+GuU9p3LsAMh6wN03MYrGmczUOnlP6YlxHNQbQxnV0sl14yOE5ni9oT4y+l+SllvbV3/Jhwpov68aoP/2MazqxR4QyGfSxhCPJ4UuDHU7IrpnTxGBTL1/oUU8ED0FxyDoH/Sc5OhTLInFqbZaVzm5Mpr12wYUBL4nE5h0Kf6BCKdgM8a+Ti3wMUsBoC79ff3jE9U/xwSneOhghLlMIIS4QYKKwYBBAGCNwMDATGCEtEwghLNBgkqhkiG9w0BBwKgghK+MIISugIBAzEPMA0GCWCGSAFlAwQCAQUAMIIBUQYLKoZIhvcNAQkQAQSgggFABIIBPDCCATgCAQEGCisGAQQBhFkKAwEwMTANBglghkgBZQMEAgEFAAQghPy22lwuCYESw8jYhb4F9ZDPJ1LPgSSZgJDkyXYzVt4CBlv98KtAoBgTMjAxODExMzAwMTA1MTkuMTM4WjAEgAIB9KCB0KSBzTCByjELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAldBMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xLTArBgNVBAsTJE1pY3Jvc29mdCBJcmVsYW5kIE9wZXJhdGlvbnMgTGltaXRlZDEmMCQGA1UECxMdVGhhbGVzIFRTUyBFU046RDA4Mi00QkZELUVFQkExJTAjBgNVBAMTHE1pY3Jvc29mdCBUaW1lLVN0YW1wIHNlcnZpY2Wggg48MIIE8TCCA9mgAwIBAgITMwAAAOIYOHtm6erB2AAAAAAA4jANBgkqhkiG9w0BAQsFADB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMDAeFw0xODA4MjMyMDI3MDNaFw0xOTExMjMyMDI3MDNaMIHKMQswCQYDVQQGEwJVUzELMAkGA1UECBMCV0ExEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEtMCsGA1UECxMkTWljcm9zb2Z0IElyZWxhbmQgT3BlcmF0aW9ucyBMaW1pdGVkMSYwJAYDVQQLEx1UaGFsZXMgVFNTIEVTTjpEMDgyLTRCRkQtRUVCQTElMCMGA1UEAxMcTWljcm9zb2Z0IFRpbWUtU3RhbXAgc2VydmljZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKirA72FF3NCLW5mfLO/D0EZ5Ycs00oiMSissXLB6WF9GNdP78QzFwAypxW/+qZSczqaHbDH8hlbxkzf3DiYgAdpQjnGkLujwKtWSaP29/lVf7jFqHy9v6eH+LdOi0LvtrPRW34MyCvpxZyOW4H1h3PkxCBL5Ra21sDqgcVL1me0osw8QTURXmI4LyeLdTH3CcI2AgNDXTjsFBf3QsO+JYyAOYWrTcLnywVN6DrigmgrDJk5w+wR4VrHfl2T9PRZbZ+UDt13wwyB9d6IURuzV8lHsAVfF8t9S0aGVPmkQ3c2waOhHpsp6VEM+T5D2Ph8xJX1r82z67WRlmGcOP2NWC0CAwEAAaOCARswggEXMB0GA1UdDgQWBBSJPpD6BsP2p+crDJL232voEtLxezAfBgNVHSMEGDAWgBTVYzpcijGQ80N7fEYbxTNoWoVtVTBWBgNVHR8ETzBNMEugSaBHhkVodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNUaW1TdGFQQ0FfMjAxMC0wNy0wMS5jcmwwWgYIKwYBBQUHAQEETjBMMEoGCCsGAQUFBzAChj5odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY1RpbVN0YVBDQV8yMDEwLTA3LTAxLmNydDAMBgNVHRMBAf8EAjAAMBMGA1UdJQQMMAoGCCsGAQUFBwMIMA0GCSqGSIb3DQEBCwUAA4IBAQARQHu7ISeBuJSHKuDRI04704cH0B7BYzeEIrD15awviMRcYIfIOHpvGzZOWQgP2Hm0Rr7kvTUu1VrSSaQ7i1gPWdhqMmw5WBnSS5bxeMhhx9UsASeE84vUu82NeZapGSjH38YAb4WT+TtiTkcoI59rA+CTCq108ttIxVfZcr3id76OETIH0HvhlnxOOWjwGy4ul6Za5RoTLG/oo2rrGmVi3FwrNWGezYLBODuEsjzG36lCRtBKC2ZAHfbOz5wtkUHbqh79mUKocjP4r3qxf5TN87yf6g1uTx+J8pdnAi5iHt+ZtangWqnVTE8PoIREWhBVlGFfQdkELUx2Or90aAqWMIIGcTCCBFmgAwIBAgIKYQmBKgAAAAAAAjANBgkqhkiG9w0BAQsFADCBiDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEyMDAGA1UEAxMpTWljcm9zb2Z0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IDIwMTAwHhcNMTAwNzAxMjEzNjU1WhcNMjUwNzAxMjE0NjU1WjB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKkdDbx3EYo6IOz8E5f1+n9plGt0VBDVpQoAgoX77XxoSyxfxcPlYcJ2tz5mK1vwFVMnBDEfQRsalR3OCROOfGEwWbEwRA/xYIiEVEMM1024OAizQt2TrNZzMFcmgqNFDdDq9UeBzb8kYDJYYEbyWEeGMoQedGFnkV+BVLHPk0ySwcSmXdFhE24oxhr5hoC732H8RsEnHSRnEnIaIYqvS2SJUGKxXf13Hz3wV3WsvYpCTUBR0Q+cBj5nf/VmwAOWRH7v0Ev9buWayrGo8noqCjHw2k4GkbaICDXoeByw6ZnNPOcvRLqn9NxkvaQBwSAJk3jN/LzAyURdXhacAQVPIk0CAwEAAaOCAeYwggHiMBAGCSsGAQQBgjcVAQQDAgEAMB0GA1UdDgQWBBTVYzpcijGQ80N7fEYbxTNoWoVtVTAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBTV9lbLj+iiXGJo0T2UkFvXzpoYxDBWBgNVHR8ETzBNMEugSaBHhkVodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNSb29DZXJBdXRfMjAxMC0wNi0yMy5jcmwwWgYIKwYBBQUHAQEETjBMMEoGCCsGAQUFBzAChj5odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY1Jvb0NlckF1dF8yMDEwLTA2LTIzLmNydDCBoAYDVR0gAQH/BIGVMIGSMIGPBgkrBgEEAYI3LgMwgYEwPQYIKwYBBQUHAgEWMWh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9QS0kvZG9jcy9DUFMvZGVmYXVsdC5odG0wQAYIKwYBBQUHAgIwNB4yIB0ATABlAGcAYQBsAF8AUABvAGwAaQBjAHkAXwBTAHQAYQB0AGUAbQBlAG4AdAAuIB0wDQYJKoZIhvcNAQELBQADggIBAAfmiFEN4sbgmD+BcQM9naOhIW+z66bM9TG+zwXiqf76V20ZMLPCxWbJat/15/B4vceoniXj+bzta1RXCCtRgkQS+7lTjMz0YBKKdsxAQEGb3FwX/1z5Xhc1mCRWS3TvQhDIr79/xn/yN31aPxzymXlKkVIArzgPF/UveYFl2am1a+THzvbKegBvSzBEJCI8z+0DpZaPWSm8tv0E4XCfMkon/VWvL/625Y4zu2JfmttXQOnxzplmkIz/amJ/3cVKC5Em4jnsGUpxY517IW3DnKOiPPp/fZZqkHimbdLhnPkd/DjYlPTGpQqWhqS9nhquBEKDuLWAmyI4ILUl5WTs9/S/fmNZJQ96LjlXdqJxqgaKD4kWumGnEcua2A5HmoDF0M2n0O99g/DhO3EJ3110mCIIYdqwUB5vvfHhAN/nMQekkzr3ZUd46PioSKv33nJ+YWtvd6mBy6cJrDm77MbL2IK0cs0d9LiFAR6A+xuJKlQ5slvayA1VmXqHczsI5pgt6o3gMy4SKfXAL1QnIffIrE7aKLixqduWsqdCosnPGUFN4Ib5KpqjEWYw07t0MkvfY3v1mYovG8chr1m1rtxEPJdQcdeh0sVV42neV8HR3jDA/czmTfsNv11P6Z0eGTgvvM9YBS7vDaBQNdrvCScc1bN+NR4Iuto229Nfj950iEkSoYICzjCCAjcCAQEwgfihgdCkgc0wgcoxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJXQTEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMS0wKwYDVQQLEyRNaWNyb3NvZnQgSXJlbGFuZCBPcGVyYXRpb25zIExpbWl0ZWQxJjAkBgNVBAsTHVRoYWxlcyBUU1MgRVNOOkQwODItNEJGRC1FRUJBMSUwIwYDVQQDExxNaWNyb3NvZnQgVGltZS1TdGFtcCBzZXJ2aWNloiMKAQEwBwYFKw4DAhoDFQByQCUheEOevaI9Zc/3QGrkX42iC6CBgzCBgKR+MHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMA0GCSqGSIb3DQEBBQUAAgUA36ppYDAiGA8yMDE4MTEyOTIxMzQyNFoYDzIwMTgxMTMwMjEzNDI0WjB3MD0GCisGAQQBhFkKBAExLzAtMAoCBQDfqmlgAgEAMAoCAQACAitfAgH/MAcCAQACAhGtMAoCBQDfq7rgAgEAMDYGCisGAQQBhFkKBAIxKDAmMAwGCisGAQQBhFkKAwKgCjAIAgEAAgMHoSChCjAIAgEAAgMBhqAwDQYJKoZIhvcNAQEFBQADgYEAbAXXPR9wy4NA0892GGqetaZF+pNClpGcfEpSuHABaZ4Gzr1nY1nmrhexTtr/U6omHALRWzkQwthk0cy+mnEHXyOZGmoEEpgrLgK3AAP5NbK/XbtHQRyZJQyhZScFbOyQycoE8QQalSVOhWxk/bbBMQaQiYVMIexNd/T0KgaDDUMxggMNMIIDCQIBATCBkzB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMAITMwAAAOIYOHtm6erB2AAAAAAA4jANBglghkgBZQMEAgEFAKCCAUowGgYJKoZIhvcNAQkDMQ0GCyqGSIb3DQEJEAEEMC8GCSqGSIb3DQEJBDEiBCCr9IiSbx6s8MLdxldRG49+4h6CbicW8hWXAicI3jNmhDCB+gYLKoZIhvcNAQkQAi8xgeowgecwgeQwgb0EIN8BpJSmQCGubWwVa4tW+aMveoHMX/nDnVN8fiDOMsrLMIGYMIGApH4wfDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEmMCQGA1UEAxMdTWljcm9zb2Z0IFRpbWUtU3RhbXAgUENBIDIwMTACEzMAAADiGDh7ZunqwdgAAAAAAOIwIgQgTkOfRvGEZNbr5/hgWclsL4/Q7SOZihE/U0lz2wEMIGcwDQYJKoZIhvcNAQELBQAEggEATlxnCfTzFfTMDvK085zlYPVCroKYW6gKFYnbAhNmrNzcxqALKmIYXpFU7B6HH/vYzkUfCyXpf5tsyEWu0oTySOjyAZ9+2vdaG8nEgjOp0L737lcitgusIjpWtta3Ik0b+mzffnvyjrgTSuKDDni3mxGfvJU77k1Ctempma4H2FJso6Bur0PRH99vIYDu4lHigOSLbeyjR5CiDciBwEVUSA0FxhoFNX1yfpxz3sukOvkaoTduREIjH5LxUjNI1ZTMK/ZkeETI8IPRpWVzAc8q7CujErHKo4sdKej/O2cfUTUHplFLVCGGExpJUCg5FH5jVUUFt75ad8503sdGplggVQ== + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + MIIucgYJKoZIhvcNAQcCoIIuYzCCLl8CAQExDzANBglghkgBZQMEAgEFADCCARAGCSsGAQQBgjcKAaCCAQEwgf4wDAYKKwYBBAGCNwwBAQQQ293slyDTMUOmVyeQqmcNSBcNMjIwMjA0MDkwNjU2WjAOBgorBgEEAYI3DAEDBQAwgbwwKgQUdgFpbn9QXi/ly4CZFKA2Eimoq6YxEjAQBgorBgEEAYI3DAIDMQKCADCBjQQg3YHMbvV3unL0mx/RJ8ihTJd1C/SYSnMHbN0yMrWijuQxaTAQBgorBgEEAYI3DAIDMQKCADBVBgorBgEEAYI3AgEEMUcwRTAQBgorBgEEAYI3AgEZogKAADAxMA0GCWCGSAFlAwQCAQUABCDdgcxu9Xe6cvSbH9EnyKFMl3UL9JhKcwds3TIytaKO5KCCE8owggYEMIID7KADAgECAhMzAA/d9uEraWrDsxZGAAAAD932MA0GCSqGSIb3DQEBCwUAMIGLMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMQwwCgYDVQQLEwNBT0MxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAyMjAeFw0yMjAyMDQwODU5MzZaFw0yMjAyMDcwODU5MzZaMC8xLTArBgNVBAMTJDQ2ZTIwYzY2LTUxNTEtNDNhOS05MWQ4LTMwNDY0ZGZhMDg4ZjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALaMR5Ajlp2POidTxT0N47z5sukiE9fqMuI1QimTXIfB2/gPdMsNUbVCD57ha8J1DUJQpO5nuunbk3SONwfDTdWKx9u7zWubpEmgG7hZ8LTKFlF+xhS6lS/FBSSHyXpdScWPg62BMnDysqHMH/AjLw26HNPRWK8A0vx9jsalFwLg15u4MPuKN3Bpawr/OL0B+7eh/dGO3PutAqJ4aZs2lUCIyODg0q3Tzhgi7SIvFacFWJ8Qj6+D3AfasOv8oanfpNLLPhPlxXGEK9sMKHOOb8mTU9V/ibERqEKTHkJ24Vu+BwrXq5eVedtwmHT2WWR7teaIvrUT9AiPmMhx4hIF/0MCAwEAAaOCAbowggG2MBgGA1UdJQEB/wQOMAwGCisGAQQBgjdMHAEwHQYDVR0OBBYEFB4NpkpeSLk/j691tPw97rB4XKDJMA0GA1UdEQQGMASCAlVTMB8GA1UdIwQYMBaAFPeC7EkBPjJR1eFAcP4hAMeQ72IPMF8GA1UdHwRYMFYwVKBSoFCGTmh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY3JsL01pY3Jvc29mdCUyME1hcmtldHBsYWNlJTIwQ0ElMjBHJTIwMDIyLmNybDBsBggrBgEFBQcBAQRgMF4wXAYIKwYBBQUHMAKGUGh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY2VydHMvTWljcm9zb2Z0JTIwTWFya2V0cGxhY2UlMjBDQSUyMEclMjAwMjIuY3J0MAwGA1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgXgMDwGCSsGAQQBgjcVBwQvMC0GJSsGAQQBgjcVCIOS9kTqrxCDkY0wgqzgLIKinDE0g+6NOIaE7wACAWQCARcwIAYJKwYBBAGCNxUKAQH/BBAwDjAMBgorBgEEAYI3TBwBMA0GCSqGSIb3DQEBCwUAA4ICAQBAiJuIJeMS+PMrxG+kwPWT7yy7A6P8trchwtXyYep/WXu7xT2tH2ys3wuP4DYG5CY4nKekjTKraQhZGuYj4r4KrUviShe8ZWMIcK4MRduXAU81UOd/fsCq5djQ6NDN4Bzq1TxjwZKiEMEWmXaunj6XAlDSX3SgyyMb/ywmur8VVKO+xFVkOxAqCjNo1VaCv3zEvr3Y6dUwO9gYQui1bAQVQzdxRYUPEJlsFBmbNL1AnZ39r4/9K/6orxavU4qwkj4cXnhtMYKDjTIIbTBrfD4glP6mnmkcZpN0ItfEAVgU9rGpXg84hQLeP83nBs2Y8DrF3bBF9867dJCl90c5rK1DMcelmH3oUAwqZ3U+jIIj3HGyfefAQ6HPL0yiY9OYLXQZUxcZF13m92l5s7dy2C21cIh0W0iTcWPIhNn9cia4Hr7FabTC0RoQkO1bg0PKvPMnoe0AqEVmPKImNDncg538AjPy2qeUxn7+0kcuZWEdKVAPwWfzh5qvSAmYKk4XNBfN2E40GQ0ruRdvMVMFeYBLXKm4SJR6zoiBa5v7qzt1uQqZf6GD4BNFZivBp0P2B9lTubEAU6SmUsazRnEVmekIO+x7gDuXbMS8tv6mb1pbGCtIV/wRcIYNcDkHKTHMbW5xR/OmGTM3mPRxoMQtaWtxIeagZkU2nl+pElA/0h7pIjCCBtcwggS/oAMCAQICCmESRKIAAAAAAAIwDQYJKoZIhvcNAQELBQAwgYgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xMjAwBgNVBAMTKU1pY3Jvc29mdCBSb290IENlcnRpZmljYXRlIEF1dGhvcml0eSAyMDExMB4XDTExMDMyODIxMDkzOVoXDTMxMDMyODIxMTkzOVowfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAubUaSwGYVsE3MAnPfvmozUhAB3qxBABgJRW1vDp4+tVinXxD32f7k1K89JQ6zDOgS/iDgULC+yFK1K/1Qjac/0M7P6c8v5LSjnWGlERLa/qY32j46S7SLQcit3g2jgoTTO03eUG+9yHZUTGV/FJdRYB8uXhrznJBa+Y+yGwiQKF+m6XFeBH/KORoKFx+dmMoy9EWJ/m/o9IiUj2kzm9C691+vZ/I2w0Bj93W9SPPkV2PCNHlzgfIAoeajWpHmi38Wi3xZHonkzAVBHxPsCBppOoNsWvmAfUM7eBthkSPvFruekyDCPNEYhfGqgqtqLkoBebXLZCOVybF7wTQaLvse60//3P003icRcCoQYgY4NAqrF7j80o5U7DkeXxcB0xvengsaKgiAaV1DKkRbpe98wCqr1AASvm5rAJUYMU+mXmOieV2EelY2jGrenWe9FQpNXYV1NoWBh0WKoFxttoWYAnF705bIWtSZsz08ZfK6WLX4GXNLcPBlgCzfTm1sdKYASWdBbH2haaNhPapFhQQBJHKwnVW2iXErImhuPi45W3MVTZ5D9ASshZx69cLYY6xAdIa+89Kf/uRrsGOVZfahDuDw+NI183iAyzC8z/QRt2P32LYxP0xrCdqVh+DJo2i4NoE8Uk1usCdbVRuBMBQl/AwpOTq7IMvHGElf65CqzUCAwEAAaOCAUswggFHMBAGCSsGAQQBgjcVAQQDAgEAMB0GA1UdDgQWBBQPU8s/FmEl/mCJHdO5fOiQrbOU0TAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBRyLToCMZBDuRQFTuHqp8cx0SOJNDBaBgNVHR8EUzBRME+gTaBLhklodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3JsMF4GCCsGAQUFBwEBBFIwUDBOBggrBgEFBQcwAoZCaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3J0MA0GCSqGSIb3DQEBCwUAA4ICAQCjuZmM8ZVNDgp9wHsL4RY8KJ8nLinvxFTphNGCrxaLknkYG5pmMhVlX+UB/tSiW8W13W60nggz9u5xwMx7v/1t/Tgm6g2brVyOKI5A7u6/2SIJwkJKFw953K0YIKVT28w9zl8dSJnmRnyR0G86ncWbF6CLQ6A6lBQ9o2mTGVqDr4m35WKAnc6YxUUM1y74mbzFFZr63VHsCcOp3pXWnUqAY1rb6Q6NX1b3clncKqLFm0EjKHcQ56grTbwuuB7pMdh/IFCJR01MQzQbDtpEisbOeZUi43YVAAHKqI1EO9bRwg3frCjwAbml9MmI4utMW94gWFgvrMxIX+n42RBDIjf3Ot3jkT6gt3XeTTmO9bptgblZimhERdkFRUFpVtkocJeLoGuuzP93uH/Yp032wzRH+XmMgujfZv+vnfllJqxdowoQLx55FxLLeTeYfwi/xMSjZO2gNven3U/3KeSCd1kUOFS3AOrwZ0UNOXJeW5JQC6Vfd1BavFZ6FAta1fMLu3WFvNB+FqeHUaU3ya7rmtxJnzk29DeSqXgGNmVSywBS4NajI5jJIKAA6UhNJlsg8CHYwUOKf5ej8OoQCkbadUxXygAfxCfW2YBbujtI+PoyejRFxWUjYFWO5LeTI62UMyqfOEiqugoYjNxmQZla2s4YHVuqIC34R85FQlg9pKQBsDCCBuMwggTLoAMCAQICEzMAAABLZz0Eed1JFCkAAAAAAEswDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMB4XDTE5MDQxODIwNDUwNFoXDTI0MDQxODIwNDUwNFowgYsxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xDDAKBgNVBAsTA0FPQzEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldHBsYWNlIENBIEcgMDIyMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAzGfV5tDxZ3oPG8aq2OwrBLSRFC61lYvWtkqOdKswlBHVncuoOIzekLIFNsMwluoK0plNZ5w/JPmwkGTGIrt9ftbWQz0k7XRfacXvQQfhCaBgjkybvvDx7CTL0UDEUiXfEEkylUcQKOdm9LApPDP4oO/V8RS4ugkqkjsuVaOyfYv0TytjVVQon72W+wR/AyrBAXgfwzzEb85403GTJRzQMlTw1YqdgS/o9SCsvH9dRZTGlWCatIl+a00eTG5zgeu0xCtqwgERhw9UT6mlPqmp6RqJ8XMgKylh9Ss+jI53EmraAFlelph16kuqF6n3vNnRBfhQ+gmNdtKi9s3jI8Di7ip8hYGPHGGhtcw0AYUg5r/VQhIfXxO45zO7SFvjujX0ji0b3WKB/xD9Sg1KO1fiTpj82ifyPqHvL+iwn5dV98C4ru0dQ0hfK7tA4K5qkW+2gym0fBjGYWq81/smrj6LbPdNqCotgTMU4dCMBhmSJOD9fSiPUybrbHJrHGR5YNs5VJbHH5u1ia8UrgsSrGTb1bh38Tb10matJpBI7NwDWTkV7kMvgNRQsGO/1+dg47PLW3X7TbiZniTPC+oAqOh1CiF+ODnem4Nw1iRw854LN3yqibLHgTi093mCBtPGWZxg2sTcHUX4fo/UHNbSh8bERJznnennm4gEJXAYtpnEFSUCAwEAAaOCAUswggFHMA4GA1UdDwEB/wQEAwIBhjAQBgkrBgEEAYI3FQEEAwIBADAdBgNVHQ4EFgQU94LsSQE+MlHV4UBw/iEAx5DvYg8wGQYJKwYBBAGCNxQCBAweCgBTAHUAYgBDAEEwEgYDVR0TAQH/BAgwBgEB/wIBADAfBgNVHSMEGDAWgBQPU8s/FmEl/mCJHdO5fOiQrbOU0TBXBgNVHR8EUDBOMEygSqBIhkZodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNNYXJQQ0EyMDExXzIwMTEtMDMtMjguY3JsMFsGCCsGAQUFBwEBBE8wTTBLBggrBgEFBQcwAoY/aHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNNYXJQQ0EyMDExXzIwMTEtMDMtMjguY3J0MA0GCSqGSIb3DQEBCwUAA4ICAQAbTWps31OnBv77EVpCaU68drubToSKtpw9fYt4MvAhorXGRrRRfS6qou5o45QZAKjwzbOrr33dWxF4P3gCGqZxCotW32p6hlXGP7BnYlVw6242nlX0PahgpyWVWBmi0ElKH8N6HYhJqAW74tUNCzY7ALNMwk/JVEMKpWq+5XqutAEmtP0SdoSZPRwPtdiRjMLS8yyjP4HVAiKJ+m0wV3bLBtCG20BttEGkly7fYa2/E0ps41PLeX3T2yKRgmnGjhD16NUcMDdRpBh7MCXktwpISDEanB8QySJNdErU+NaKukgOW68+Oyk7lGiiZcG3boCGNec9JkM7dQdpA4tneQg0FI/o0mkDZR5aVItbAjRfMgAXpm6Hfu9DqOP+/R0K9v+CwyhwEjF3+SeL8YBAijXHD5YaiWzwZzBDtNQpuOXc91NmF7ifdHVy9IE1EhuIIQsMio3l0Uo87vGdwQ50po8o1F72zi1vfvKfDmszVezk6fF7cy7fZYh8I0ceVbZ4XY4xJ76TdYMItUAoPkvVdDq5B/d4oI7/xHfomRsEw08fuLxCRE8/BcvzSlDpnxu4wL6IULqVwHE+fbQ82Lr5ZDLxa1dYUw8zBGgCnx/ObVPj1RO1CaVi13wCxm8yaHeflKJWw7ZHOLeOGQyMRT+f9cFFk+5fzCA9GcwiSlEyYj4ilzGCGWUwghlhAgEBMIGjMIGLMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMQwwCgYDVQQLEwNBT0MxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAyMgITMwAP3fbhK2lqw7MWRgAAAA/d9jANBglghkgBZQMEAgEFAKCBjzAYBgkqhkiG9w0BCQMxCwYJKwYBBAGCNwoBMC8GCSqGSIb3DQEJBDEiBCBDgOMhvMousTIzk/P1eC2qGiSJLdSm3brGanFMLa41+DBCBgorBgEEAYI3AgEMMTQwMqAUgBIATQBpAGMAcgBvAHMAbwBmAHShGoAYaHR0cDovL3d3dy5taWNyb3NvZnQuY29tMA0GCSqGSIb3DQEBAQUABIIBAJ8LEwzLh2rDfLOOOodKT10yXoCzHPaQLv9sMwei+650mqV0iHaw9bUolBZAdBnF7JBRjY+yb9AIF7M/Av9pHRKvuLHWDntlo49RrfvNucHaJqPjCMm81h/GAuimPSBMpRtTCW+HHDWrw+HoGbRWQ+N/Vpt//pzj9gAzOzs8W9gU0RMz3ub4Hfgunm/CGACjFu5Hi2sjJbgigNlyRnxjI4+h3J7dGVOfR8hlhRB9YjXDB700F0bzaKqxC9LH7kZbAjiHPgLLCWz7OQ/rTo7wV5/e6v9GCKpamydPy/0MObq4TDi0fn75aVHTJJrzlwt1BJKyVPdAsOhiFUH3Mq4WheKhghcAMIIW/AYKKwYBBAGCNwMDATGCFuwwghboBgkqhkiG9w0BBwKgghbZMIIW1QIBAzEPMA0GCWCGSAFlAwQCAQUAMIIBUQYLKoZIhvcNAQkQAQSgggFABIIBPDCCATgCAQEGCisGAQQBhFkKAwEwMTANBglghkgBZQMEAgEFAAQgr34HNu5gF3j5DAITqq38vc6sNxs6ZqLz9i9zU2oQDtoCBmH66bcTCRgTMjAyMjAyMDQwOTA5NDkuNTkxWjAEgAIB9KCB0KSBzTCByjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjElMCMGA1UECxMcTWljcm9zb2Z0IEFtZXJpY2EgT3BlcmF0aW9uczEmMCQGA1UECxMdVGhhbGVzIFRTUyBFU046REQ4Qy1FMzM3LTJGQUUxJTAjBgNVBAMTHE1pY3Jvc29mdCBUaW1lLVN0YW1wIFNlcnZpY2WgghFXMIIHDDCCBPSgAwIBAgITMwAAAZwPpk1h0p5LKAABAAABnDANBgkqhkiG9w0BAQsFADB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMDAeFw0yMTEyMDIxOTA1MTlaFw0yMzAyMjgxOTA1MTlaMIHKMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSUwIwYDVQQLExxNaWNyb3NvZnQgQW1lcmljYSBPcGVyYXRpb25zMSYwJAYDVQQLEx1UaGFsZXMgVFNTIEVTTjpERDhDLUUzMzctMkZBRTElMCMGA1UEAxMcTWljcm9zb2Z0IFRpbWUtU3RhbXAgU2VydmljZTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBANtSKgwZXUkWP6zrXazTaYq7bco9Q2zvU6MN4ka3GRMX2tJZOK4DxeBiQACL/n7YV/sKTslwpD0f9cPU4rCDX9sfcTWo7XPxdHLQ+WkaGbKKWATsqw69bw8hkJ/bjcp2V2A6vGsvwcqJCh07BK3JPmUtZikyy5PZ8fyTyiKGN7hOWlaIU9oIoucUNoAHQJzLq8h20eNgHUh7eI5k+Kyq4v6810LHuA6EHyKJOZN2xTw5JSkLy0FN5Mhg/OaFrFBl3iag2Tqp4InKLt+Jbh/Jd0etnei2aDHFrmlfPmlRSv5wSNX5zAhgEyRpjmQcz1zp0QaSAefRkMm923/ngU51IbrVbAeHj569SHC9doHgsIxkh0K3lpw582+0ONXcIfIU6nkBT+qADAZ+0dT1uu/gRTBy614QAofjo258TbSX9aOU1SHuAC+3bMoyM7jNdHEJROH+msFDBcmJRl4VKsReI5+S69KUGeLIBhhmnmQ6drF8Ip0ZiO+vhAsD3e9AnqnY7Hcge850I9oKvwuwpVwWnKnwwSGElMz7UvCocmoUMXk7Vn2aNti+bdH28+GQb5EMsqhOmvuZOCRpOWN33G+b3g5unwEP0eTiY+LnWa2AuK43z/pplURJVle29K42QPkOcglB6sjLmNpEpb9basJ72eA0Mlp1LtH3oYZGXsggTfuXAgMBAAGjggE2MIIBMjAdBgNVHQ4EFgQUu2kJZ1Ndjl2112SynL6jGMID+rIwHwYDVR0jBBgwFoAUn6cVXQBeYl2D9OXSZacbUzUZ6XIwXwYDVR0fBFgwVjBUoFKgUIZOaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9jcmwvTWljcm9zb2Z0JTIwVGltZS1TdGFtcCUyMFBDQSUyMDIwMTAoMSkuY3JsMGwGCCsGAQUFBwEBBGAwXjBcBggrBgEFBQcwAoZQaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9jZXJ0cy9NaWNyb3NvZnQlMjBUaW1lLVN0YW1wJTIwUENBJTIwMjAxMCgxKS5jcnQwDAYDVR0TAQH/BAIwADATBgNVHSUEDDAKBggrBgEFBQcDCDANBgkqhkiG9w0BAQsFAAOCAgEApwAqpiMYRzNNYyz3PSbtijbeyCpUXcvIrqA4zPtMIcAk34W9u9mRDndWS+tlR3WwTpr1OgaV1wmc6YFzqK6EGWm903UEsFE7xBJMPXjfdVOPhcJB3vfvA0PX56oobcF2OvNsOSwTB8bi/ns+Cs39Puzs+QSNQZd8iAVBCSvxNCL78dln2RGU1xyB4AKqV9vi4Y/Gfmx2FA+jF0y+YLeob0M40nlSxL0q075t7L6iFRMNr0u8ROhzhDPLl+4ePYfUmyYJoobvydel9anAEsHFlhKl+aXb2ic3yNwbsoPycZJL/vo8OVvYYxCy+/5FrQmAvoW0ZEaBiYcKkzrNWt/hX9r5KgdwL61x0ZiTZopTko6W/58UTefTbhX7Pni0MApH3Pvyt6N0IFap+/LlwFRD1zn7e6ccPTwESnuo/auCmgPznq80OATA7vufsRZPvqeX8jKtsraSNscvNQymEWlcqdXV9hYkjb4T/Qse9cUYaoXg68wFHFuslWfTdPYPLl1vqzlPMnNJpC8KtdioDgcq+y1BaSqSm8EdNfwzT37+/JFtVc3Gs915fDqgPZDgOSzKQIV+fw3aPYt2LET3AbmKKW/r13Oy8cg3+D0D362GQBAJVv0NRI5NowgaCw6oNgWOFPrN72WSEcca/8QQiTGP2XpLiGpRDJZ6sWRpRYNdydkwggdxMIIFWaADAgECAhMzAAAAFcXna54Cm0mZAAAAAAAVMA0GCSqGSIb3DQEBCwUAMIGIMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMTIwMAYDVQQDEylNaWNyb3NvZnQgUm9vdCBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkgMjAxMDAeFw0yMTA5MzAxODIyMjVaFw0zMDA5MzAxODMyMjVaMHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA5OGmTOe0ciELeaLL1yR5vQ7VgtP97pwHB9KpbE51yMo1V/YBf2xK4OK9uT4XYDP/XE/HZveVU3Fa4n5KWv64NmeFRiMMtY0Tz3cywBAY6GB9alKDRLemjkZrBxTzxXb1hlDcwUTIcVxRMTegCjhuje3XD9gmU3w5YQJ6xKr9cmmvHaus9ja+NSZk2pg7uhp7M62AW36MEBydUv626GIl3GoPz130/o5Tz9bshVZN7928jaTjkY+yOSxRnOlwaQ3KNi1wjjHINSi947SHJMPgyY9+tVSP3PoFVZhtaDuaRr3tpK56KTesy+uDRedGbsoy1cCGMFxPLOJiss254o2I5JasAUq7vnGpF1tnYN74kpEeHT39IM9zfUGaRnXNxF803RKJ1v2lIH1+/NmeRd+2ci/bfV+AutuqfjbsNkz2K26oElHovwUDo9Fzpk03dJQcNIIP8BDyt0cY7afomXw/TNuvXsLz1dhzPUNOwTM5TI4CvEJoLhDqhFFG4tG9ahhaYQFzymeiXtcodgLiMxhy16cg8ML6EgrXY28MyTZki1ugpoMhXV8wdJGUlNi5UPkLiWHzNgY1GIRH29wb0f2y1BzFa/ZcUlFdEtsluq9QBXpsxREdcu+N+VLEhReTwDwV2xo3xwgVGD94q0W29R6HXtqPnhZyacaue7e3PmriLq0CAwEAAaOCAd0wggHZMBIGCSsGAQQBgjcVAQQFAgMBAAEwIwYJKwYBBAGCNxUCBBYEFCqnUv5kxJq+gpE8RjUpzxD/LwTuMB0GA1UdDgQWBBSfpxVdAF5iXYP05dJlpxtTNRnpcjBcBgNVHSAEVTBTMFEGDCsGAQQBgjdMg30BATBBMD8GCCsGAQUFBwIBFjNodHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpb3BzL0RvY3MvUmVwb3NpdG9yeS5odG0wEwYDVR0lBAwwCgYIKwYBBQUHAwgwGQYJKwYBBAGCNxQCBAweCgBTAHUAYgBDAEEwCwYDVR0PBAQDAgGGMA8GA1UdEwEB/wQFMAMBAf8wHwYDVR0jBBgwFoAU1fZWy4/oolxiaNE9lJBb186aGMQwVgYDVR0fBE8wTTBLoEmgR4ZFaHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraS9jcmwvcHJvZHVjdHMvTWljUm9vQ2VyQXV0XzIwMTAtMDYtMjMuY3JsMFoGCCsGAQUFBwEBBE4wTDBKBggrBgEFBQcwAoY+aHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNSb29DZXJBdXRfMjAxMC0wNi0yMy5jcnQwDQYJKoZIhvcNAQELBQADggIBAJ1VffwqreEsH2cBMSRb4Z5yS/ypb+pcFLY+TkdkeLEGk5c9MTO1OdfCcTY/2mRsfNB1OW27DzHkwo/7bNGhlBgi7ulmZzpTTd2YurYeeNg2LpypglYAA7AFvonoaeC6Ce5732pvvinLbtg/SHUB2RjebYIM9W0jVOR4U3UkV7ndn/OOPcbzaN9l9qRWqveVtihVJ9AkvUCgvxm2EhIRXT0n4ECWOKz3+SmJw7wXsFSFQrP8DJ6LGYnn8AtqgcKBGUIZUnWKNsIdw2FzLixre24/LAl4FOmRsqlb30mjdAy87JGA0j3mSj5mO0+7hvoyGtmW9I/2kQH2zsZ0/fZMcm8Qq3UwxTSwethQ/gpY3UA8x1RtnWN0SCyxTkctwRQEcb9k+SS+c23Kjgm9swFXSVRk2XPXfx5bRAGOWhmRaw2fpCjcZxkoJLo4S5pu+yFUa2pFEUep8beuyOiJXk+d0tBMdrVXVAmxaQFEfnyhYWxz/gq77EFmPWn9y8FBSX5+k77L+DvktxW/tM4+pTFRhLy/AsGConsXHRWJjXD+57XQKBqJC4822rpM+Zv/Cuk0+CQ1ZyvgDbjmjJnW4SLq8CdCPSWU5nR0W2rRnj7tfqAxM328y+l7vzhwRNGQ8cirOoo6CGJ/2XBjU02N7oJtpQUQwXEGahC0HVUzWLOhcGbyoYICzjCCAjcCAQEwgfihgdCkgc0wgcoxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJTAjBgNVBAsTHE1pY3Jvc29mdCBBbWVyaWNhIE9wZXJhdGlvbnMxJjAkBgNVBAsTHVRoYWxlcyBUU1MgRVNOOkREOEMtRTMzNy0yRkFFMSUwIwYDVQQDExxNaWNyb3NvZnQgVGltZS1TdGFtcCBTZXJ2aWNloiMKAQEwBwYFKw4DAhoDFQDN2Wnq3fCz9ucStub1zQz7129TQKCBgzCBgKR+MHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMA0GCSqGSIb3DQEBBQUAAgUA5adiZjAiGA8yMDIyMDIwNDE2MjkyNloYDzIwMjIwMjA1MTYyOTI2WjB3MD0GCisGAQQBhFkKBAExLzAtMAoCBQDlp2JmAgEAMAoCAQACAh92AgH/MAcCAQACAhFfMAoCBQDlqLPmAgEAMDYGCisGAQQBhFkKBAIxKDAmMAwGCisGAQQBhFkKAwKgCjAIAgEAAgMHoSChCjAIAgEAAgMBhqAwDQYJKoZIhvcNAQEFBQADgYEAbKzHReDTwgAKGSkgO/zxCAV/RcxwGpZK4M2bWgwqHVIi+/xiAKdQOSKeMz3UwRO4LI4EN4BjeK+ATpruzOtVGkyA8yzwdJFPv99j4O/Hc5kJSvovfECah62tBB3siNIFaAYNK+BF76k3j4+y9q9lF5yjQ8ZylVYbOfOjlJ2uevsxggQNMIIECQIBATCBkzB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMAITMwAAAZwPpk1h0p5LKAABAAABnDANBglghkgBZQMEAgEFAKCCAUowGgYJKoZIhvcNAQkDMQ0GCyqGSIb3DQEJEAEEMC8GCSqGSIb3DQEJBDEiBCD10XWSucJFd8jLe5PaC9h9/GHXK7ZZ0kb4BGFNqgWk9DCB+gYLKoZIhvcNAQkQAi8xgeowgecwgeQwgb0EIDcPRYUgjSzKOhF39d4QgbRZQgrPO7Lo/qE5GtvSeqa8MIGYMIGApH4wfDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEmMCQGA1UEAxMdTWljcm9zb2Z0IFRpbWUtU3RhbXAgUENBIDIwMTACEzMAAAGcD6ZNYdKeSygAAQAAAZwwIgQgPTEmgx6rn5xpH+xmlpYkcn0dL16GyYRqyG8uRajrfTYwDQYJKoZIhvcNAQELBQAEggIAxmSLisdd5cs+7liXX7vsF8zX3Y1rbtyPZ0M466WBaVULlfeuu9RMzOhWha7bkfYmKs5hXOQNBJrIHxnHbWeDu/V7EVF80IQ7/6qWiVPNcLtMnS8+uyw3Tby2gEhshO/HHOJRqllrgtUGuPl8FoAAPbPKV7X7jrg1kqOWoqpwyiSzt5cN/qijwfTcTIaPeON5wMd8wRT2l5F9e/JDBSq0WuzYb2r0kifv9wq21orm4sQkbg5GgDa/oCr3PQVCNSvPb2G7HANcqJXHeRKyoCeVhX5UaFrCQbdxGFytOc5GzptQKNn18yGg4VYrFpigUcglLbuREqqlPabwZh/QfgKdp4b4uVA6On5HIV5e9XdJ9TXhvDArEQ+Kgyao5xlgb4MkvPO9Thh9UIbw9r3JhQ+uKnCvXyiE2SpTLq1OXTjt/RrLZb+qsZW4nrDRBlBswa4G5OVW/svvI8N2/+YYrSdaCWt4unWSnSxeVUWZkugBmWdeoEJpfs5Qy8NwC+g1TyXcMjolOFTfBB6JYpHCbk6Euh2gKxq5weBb7puxVN4cvawYY2zorp6ZXtAmQGflZTl02nWld+nvDoF/zi6IUmP7PTKvbEVs8T+QkCiIM7cpLMs4NFK29f33lu1iZeGwJAxCYsV/P7geuRfA++bpDutTXjLKMqUTgf00sqJ/5HGuwLE= From webhook-mailer at python.org Fri Feb 4 11:38:23 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 04 Feb 2022 16:38:23 -0000 Subject: [Python-checkins] bpo-46629: Update classicAppCompat.sccd for new signing certificate (GH-31111) Message-ID: https://github.com/python/cpython/commit/6da436497a79dea0f3bb756c2d78921b2838e03b commit: 6da436497a79dea0f3bb756c2d78921b2838e03b branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-04T08:38:09-08:00 summary: bpo-46629: Update classicAppCompat.sccd for new signing certificate (GH-31111) (cherry picked from commit 9b4e3d94a5746af093392ed8e977b26fcc1bfd11) Co-authored-by: Steve Dower files: M PC/classicAppCompat.can.xml M PC/classicAppCompat.sccd diff --git a/PC/classicAppCompat.can.xml b/PC/classicAppCompat.can.xml index f00475c8da312..df361f8e3e2cd 100644 --- a/PC/classicAppCompat.can.xml +++ b/PC/classicAppCompat.can.xml @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file diff --git a/PC/classicAppCompat.sccd b/PC/classicAppCompat.sccd index 97648985a2ccb..d7a70cdd0533c 100644 --- a/PC/classicAppCompat.sccd +++ b/PC/classicAppCompat.sccd @@ -1,28 +1,41 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - MIIq5AYJKoZIhvcNAQcCoIIq1TCCKtECAQExDzANBglghkgBZQMEAgEFADCCARAGCSsGAQQBgjcKAaCCAQEwgf4wDAYKKwYBBAGCNwwBAQQQaM+L42jwBUGvBczrtolMmhcNMTgxMTMwMDA1OTAzWjAOBgorBgEEAYI3DAEDBQAwgbwwKgQUWKcU3R38DGPlKK33XGIwKtVL1r4xEjAQBgorBgEEAYI3DAIDMQKCADCBjQQg3K+KBOQX7HfxjRNZC9cx8gIPkEhPRO1nJFRdWQrVEJ4xaTAQBgorBgEEAYI3DAIDMQKCADBVBgorBgEEAYI3AgEEMUcwRTAQBgorBgEEAYI3AgEZogKAADAxMA0GCWCGSAFlAwQCAQUABCDcr4oE5Bfsd/GNE1kL1zHyAg+QSE9E7WckVF1ZCtUQnqCCFFAwggZSMIIEOqADAgECAhMzAAMu49KhfNamygpWAAIAAy7jMA0GCSqGSIb3DQEBCwUAMIGMMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMQ0wCwYDVQQLEwRNT1BSMScwJQYDVQQDEx5NaWNyb3NvZnQgTWFya2V0cGxhY2UgQ0EgRyAwMTMwHhcNMTgxMTMwMDA1NTA1WhcNMTgxMjAzMDA1NTA1WjB0MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMR4wHAYDVQQDExVNaWNyb3NvZnQgQ29ycG9yYXRpb24wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCwpcimfAx3HEpba1GLL/gDaRVddHE5PXTRmwlgaz8kt6/rq5rlrPFnCnbIc5818v0xJIznastbmrq26xyCEHyMLBKnyneTKE36I7+TGjcY0D7ow+o2vY7LDKMCTGlh31fx1Tvrl+5xTbWX5jdLU/3MB5faeOGh+0Knzwx1KDoXWgPtfXnD8I5jxJieoWoCwCjKTJgBOklLy9nbOalxf0h+xQRy2p5fj+PxAwQPgHWft36AF7/IMbt9FcXMtg4xdpnTYz4OV3dFOPz4m3M8HwVgNMv89W/1Ozc7uOyZt0Ij1baT6r2L3IjYg5ftzpGqaDOFcWlyDFSdhMR6BIKW8xEpAgMBAAGjggHCMIIBvjAYBgNVHSUBAf8EDjAMBgorBgEEAYI3TBwBMB0GA1UdDgQWBBRdpGYiCytx83FYzPSl+o97YzpxGzAPBgNVHREECDAGggRNT1BSMB8GA1UdIwQYMBaAFEnYB1RFhpclHtZZcRLDcpt0OE3oMGIGA1UdHwRbMFkwV6BVoFOGUWh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY3JsL01pY3Jvc29mdCUyME1hcmtldHBsYWNlJTIwQ0ElMjBHJTIwMDEzKDIpLmNybDBvBggrBgEFBQcBAQRjMGEwXwYIKwYBBQUHMAKGU2h0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY2VydHMvTWljcm9zb2Z0JTIwTWFya2V0cGxhY2UlMjBDQSUyMEclMjAwMTMoMikuY3J0MAwGA1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgXgMDwGCSsGAQQBgjcVBwQvMC0GJSsGAQQBgjcVCIOS9kTqrxCDkY0wgqzgLIKinDE0g+6NOIaE7wACAWQCARYwIAYJKwYBBAGCNxUKAQH/BBAwDjAMBgorBgEEAYI3TBwBMA0GCSqGSIb3DQEBCwUAA4ICAQB3Dk3rXH52CDq/z1fwqn9xI5WGjGmu6oAE4HSc3sNdFrSVMMGm4gTlYGWSZ0wJUUf16mVr/rdXhxuR3MZn+m4Bhdl8KQqYjYbIvCUVj0o9nZ+yT6foeY8bKnB+K5h6rol+mjDj5IfcutC4x2Kx5RrtDtRTSoKA63iZ74DYngPpBGBBgaS2c/QzgqPRAMMRqy2KBDP0miCnpR3F4YlzHGyOZwyHhESjYd9kwF47+msuHS04JZpnGHIvBppKN9XQzH3WezNnnX3lz4AyAUMsMFuARqEnacUhrAHL9n5zMv9CzxDYN1r1/aDh/788RuGuZM+E3NtmbxJJ7j6T5/VtXNBRgKtIq8d2+11j6qvKLigOTxSC25/A70BZBEvllLFnvc1vA2LrC9drwt1KpSmWie1nvpilw7o+gHMOG9utUxGha2VuVizuVNGCywTRRjvmGS1QqTfaun1URVrLfnDINXuTgN1Vwp0J5IGpJ3D8yj01NDQ/RworE+3W/R531NBYova9QRhU/igEw/Aa/q8wjZ4Pzxr9oBIo0Ta3Tv6qIggaWXw0U9+F0J7SCqIhn0d0ATO+E1Qs/SxZIAICLwmqzoLYUAh8q153esBs4uesueqgt5ueyHK8V3WjMS4wxEyVN5ZMET3hFtEshsZC31tLDdjq750U4SgQVmoYSm3F3ZOKQDCCBtcwggS/oAMCAQICCmESRKIAAAAAAAIwDQYJKoZIhvcNAQELBQAwgYgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xMjAwBgNVBAMTKU1pY3Jvc29mdCBSb290IENlcnRpZmljYXRlIEF1dGhvcml0eSAyMDExMB4XDTExMDMyODIxMDkzOVoXDTMxMDMyODIxMTkzOVowfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAubUaSwGYVsE3MAnPfvmozUhAB3qxBABgJRW1vDp4+tVinXxD32f7k1K89JQ6zDOgS/iDgULC+yFK1K/1Qjac/0M7P6c8v5LSjnWGlERLa/qY32j46S7SLQcit3g2jgoTTO03eUG+9yHZUTGV/FJdRYB8uXhrznJBa+Y+yGwiQKF+m6XFeBH/KORoKFx+dmMoy9EWJ/m/o9IiUj2kzm9C691+vZ/I2w0Bj93W9SPPkV2PCNHlzgfIAoeajWpHmi38Wi3xZHonkzAVBHxPsCBppOoNsWvmAfUM7eBthkSPvFruekyDCPNEYhfGqgqtqLkoBebXLZCOVybF7wTQaLvse60//3P003icRcCoQYgY4NAqrF7j80o5U7DkeXxcB0xvengsaKgiAaV1DKkRbpe98wCqr1AASvm5rAJUYMU+mXmOieV2EelY2jGrenWe9FQpNXYV1NoWBh0WKoFxttoWYAnF705bIWtSZsz08ZfK6WLX4GXNLcPBlgCzfTm1sdKYASWdBbH2haaNhPapFhQQBJHKwnVW2iXErImhuPi45W3MVTZ5D9ASshZx69cLYY6xAdIa+89Kf/uRrsGOVZfahDuDw+NI183iAyzC8z/QRt2P32LYxP0xrCdqVh+DJo2i4NoE8Uk1usCdbVRuBMBQl/AwpOTq7IMvHGElf65CqzUCAwEAAaOCAUswggFHMBAGCSsGAQQBgjcVAQQDAgEAMB0GA1UdDgQWBBQPU8s/FmEl/mCJHdO5fOiQrbOU0TAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBRyLToCMZBDuRQFTuHqp8cx0SOJNDBaBgNVHR8EUzBRME+gTaBLhklodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3JsMF4GCCsGAQUFBwEBBFIwUDBOBggrBgEFBQcwAoZCaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3J0MA0GCSqGSIb3DQEBCwUAA4ICAQCjuZmM8ZVNDgp9wHsL4RY8KJ8nLinvxFTphNGCrxaLknkYG5pmMhVlX+UB/tSiW8W13W60nggz9u5xwMx7v/1t/Tgm6g2brVyOKI5A7u6/2SIJwkJKFw953K0YIKVT28w9zl8dSJnmRnyR0G86ncWbF6CLQ6A6lBQ9o2mTGVqDr4m35WKAnc6YxUUM1y74mbzFFZr63VHsCcOp3pXWnUqAY1rb6Q6NX1b3clncKqLFm0EjKHcQ56grTbwuuB7pMdh/IFCJR01MQzQbDtpEisbOeZUi43YVAAHKqI1EO9bRwg3frCjwAbml9MmI4utMW94gWFgvrMxIX+n42RBDIjf3Ot3jkT6gt3XeTTmO9bptgblZimhERdkFRUFpVtkocJeLoGuuzP93uH/Yp032wzRH+XmMgujfZv+vnfllJqxdowoQLx55FxLLeTeYfwi/xMSjZO2gNven3U/3KeSCd1kUOFS3AOrwZ0UNOXJeW5JQC6Vfd1BavFZ6FAta1fMLu3WFvNB+FqeHUaU3ya7rmtxJnzk29DeSqXgGNmVSywBS4NajI5jJIKAA6UhNJlsg8CHYwUOKf5ej8OoQCkbadUxXygAfxCfW2YBbujtI+PoyejRFxWUjYFWO5LeTI62UMyqfOEiqugoYjNxmQZla2s4YHVuqIC34R85FQlg9pKQBsDCCBxswggUDoAMCAQICEzMAAABCs21EHGjyqKYAAAAAAEIwDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMB4XDTE4MDQyMDE2NDI0NFoXDTIxMDQyMDE2NDI0NFowgYwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xDTALBgNVBAsTBE1PUFIxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAxMzCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAOZ2KM9Pq1YCOiqWOivmHjUtkMgznTMP/Mr2YfzZeIIJySg1F4WxFZc4jagGHHNof9NRT+GGnktWsXkZuH1DzQEG4Ps1ln8+4vhbDglqu5ymDnd6RmsyoD+8xfc8bBIvE5o6R+ES4/GVD5TqNsOrWbwETaIZVbmTulJLoTS1WSsSjowmbc+sHqZiY8BNJNThUEmXSjuHqkQKKshuiFWYEqOTitp71mBLyH1wN7/jThRzGpolOeFusRNJdb8sEqvNzEN9Qh+Kp6ndzrnjE+t8ixXW3lShyyOOZqQMwsQn9q9T0v7Q69GuojBTFBOHKwigcCHr4xahuN+ZYMk0xGg+sm3Uj7I9mrWTSTiIRMZNIWq3sFg4+rFg48NYfRlXUpONmL7vXq6v1pIU99d2MXQ6uUrnUr1/n5ZiHGCeFcvWwqO8BYHdcTlrSOkayfFp7W9oCk9QO4Xy0h9cQRedRo2kvdTHxIuJS70Hdv6oePPF2ZFaLucUzzwsR4/XMAVKY8Vsm950omsSSOImsMtzavUdQM+wZFxvHTRqVDkF3quPdME0bCZOWB4hQJmd+o2clw+1mpwPu0/M92nA9FJg7MGPxkFaYW7g26jSqUJZ9AcX+Xa5TSIeqMZt3cRVjMTx0T/v73Sv8TpalqIQ5Fde1+hFK07sOAm3TwgzvlVJnbYgp0/rAgMBAAGjggGCMIIBfjASBgkrBgEEAYI3FQEEBQIDAgACMCMGCSsGAQQBgjcVAgQWBBSbJnDhuc3nQXuKuACsPflEbwjbozAdBgNVHQ4EFgQUSdgHVEWGlyUe1llxEsNym3Q4TegwEQYDVR0gBAowCDAGBgRVHSAAMBkGCSsGAQQBgjcUAgQMHgoAUwB1AGIAQwBBMAsGA1UdDwQEAwIBhjASBgNVHRMBAf8ECDAGAQH/AgEAMB8GA1UdIwQYMBaAFA9Tyz8WYSX+YIkd07l86JCts5TRMFcGA1UdHwRQME4wTKBKoEiGRmh0dHA6Ly9jcmwubWljcm9zb2Z0LmNvbS9wa2kvY3JsL3Byb2R1Y3RzL01pY01hclBDQTIwMTFfMjAxMS0wMy0yOC5jcmwwWwYIKwYBBQUHAQEETzBNMEsGCCsGAQUFBzAChj9odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY01hclBDQTIwMTFfMjAxMS0wMy0yOC5jcnQwDQYJKoZIhvcNAQELBQADggIBAIa2oa6kvuIHCNfz7anlL0W9tOCt8gQNkxOGRK3yliQIelNQahDJojyEFlHQ2BcHL5oZit3WeSDoYddhojx6YzJIWwfGwtVqgc0JFDKJJ2ZXRYMRsuy01Hn25xob+zRMS6VmV1axQn6uwOSMcgYmzoroh6edjPKu7qXcpt6LmhF2qFvLySA7wBCwfI/rR5/PX6I7a07Av7PpbY6/+2ujd8m1H3hwMrb4Hq3z6gcq62zJ3nDXUbC0Bp6Jt2kV9f0rEFpDK9oxE2qrGBUf8c3O2XirHOgAjRyWjWWtVms+MP8qBIA1NSLrBmToEWVP3sEkQZWMkoZWo4rYEJZpX7UIgdDc9zYNakgTCJqPhqn8AE1sgSSnpqAdMkkP41rTlFCv2ig2QVzDerjGfEv+uPDnlAT0kucbBJxHHvUC4aqUxaTSa0sy2bZ6NWFx8/u0gW8JahzxYvvvZL8SfwaA9P4ETb8pH1jw+6N/LfM2zJrNKhf5hjKa0VDOXUpkYq60OqVVnWJ6oJaSIWNkZKfzPnl/UHA8Bh4qfVrhc9H5PExPhhB9WVTsjf4r+OOVuolJldThcWQqljiPjk5rultr63G5xLyFpxNi4BCrcNQBJFB5wKgOWOyjQTVWTmh2ESaeqZ2aWBjftFHlxJ/qYc7WOGJV0+cHGkB/dvFxmKnv6tuWexiMMYIVUTCCFU0CAQEwgaQwgYwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xDTALBgNVBAsTBE1PUFIxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAxMwITMwADLuPSoXzWpsoKVgACAAMu4zANBglghkgBZQMEAgEFAKCBlTAYBgkqhkiG9w0BCQMxCwYJKwYBBAGCNwoBMC8GCSqGSIb3DQEJBDEiBCAS0d3bw2YOODvKFr0S4e3BDnaDcZXUKeBO77yvkWzVojBIBgorBgEEAYI3AgEMMTowOKAegBwATQBpAGMAcgBvAHMAbwBmAHQAIABDAG8AcgBwoRaAFGh0dHA6Ly9NaWNyb3NvZnQuY29tMA0GCSqGSIb3DQEBAQUABIIBABoap3Y+2k+zFz2cCmkc8xxHnpIygLsUSRMXeXdjPVcYx3o5cPLIixnL6p8+LIrlIagPg23mzTEmnjZaO4aaexk+3XojlHj22w/bEigEDnKyWt5bHeS0UNHJbxEFYRfd84IP1+mSH4c4+GuU9p3LsAMh6wN03MYrGmczUOnlP6YlxHNQbQxnV0sl14yOE5ni9oT4y+l+SllvbV3/Jhwpov68aoP/2MazqxR4QyGfSxhCPJ4UuDHU7IrpnTxGBTL1/oUU8ED0FxyDoH/Sc5OhTLInFqbZaVzm5Mpr12wYUBL4nE5h0Kf6BCKdgM8a+Ti3wMUsBoC79ff3jE9U/xwSneOhghLlMIIS4QYKKwYBBAGCNwMDATGCEtEwghLNBgkqhkiG9w0BBwKgghK+MIISugIBAzEPMA0GCWCGSAFlAwQCAQUAMIIBUQYLKoZIhvcNAQkQAQSgggFABIIBPDCCATgCAQEGCisGAQQBhFkKAwEwMTANBglghkgBZQMEAgEFAAQghPy22lwuCYESw8jYhb4F9ZDPJ1LPgSSZgJDkyXYzVt4CBlv98KtAoBgTMjAxODExMzAwMTA1MTkuMTM4WjAEgAIB9KCB0KSBzTCByjELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAldBMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xLTArBgNVBAsTJE1pY3Jvc29mdCBJcmVsYW5kIE9wZXJhdGlvbnMgTGltaXRlZDEmMCQGA1UECxMdVGhhbGVzIFRTUyBFU046RDA4Mi00QkZELUVFQkExJTAjBgNVBAMTHE1pY3Jvc29mdCBUaW1lLVN0YW1wIHNlcnZpY2Wggg48MIIE8TCCA9mgAwIBAgITMwAAAOIYOHtm6erB2AAAAAAA4jANBgkqhkiG9w0BAQsFADB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMDAeFw0xODA4MjMyMDI3MDNaFw0xOTExMjMyMDI3MDNaMIHKMQswCQYDVQQGEwJVUzELMAkGA1UECBMCV0ExEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEtMCsGA1UECxMkTWljcm9zb2Z0IElyZWxhbmQgT3BlcmF0aW9ucyBMaW1pdGVkMSYwJAYDVQQLEx1UaGFsZXMgVFNTIEVTTjpEMDgyLTRCRkQtRUVCQTElMCMGA1UEAxMcTWljcm9zb2Z0IFRpbWUtU3RhbXAgc2VydmljZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKirA72FF3NCLW5mfLO/D0EZ5Ycs00oiMSissXLB6WF9GNdP78QzFwAypxW/+qZSczqaHbDH8hlbxkzf3DiYgAdpQjnGkLujwKtWSaP29/lVf7jFqHy9v6eH+LdOi0LvtrPRW34MyCvpxZyOW4H1h3PkxCBL5Ra21sDqgcVL1me0osw8QTURXmI4LyeLdTH3CcI2AgNDXTjsFBf3QsO+JYyAOYWrTcLnywVN6DrigmgrDJk5w+wR4VrHfl2T9PRZbZ+UDt13wwyB9d6IURuzV8lHsAVfF8t9S0aGVPmkQ3c2waOhHpsp6VEM+T5D2Ph8xJX1r82z67WRlmGcOP2NWC0CAwEAAaOCARswggEXMB0GA1UdDgQWBBSJPpD6BsP2p+crDJL232voEtLxezAfBgNVHSMEGDAWgBTVYzpcijGQ80N7fEYbxTNoWoVtVTBWBgNVHR8ETzBNMEugSaBHhkVodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNUaW1TdGFQQ0FfMjAxMC0wNy0wMS5jcmwwWgYIKwYBBQUHAQEETjBMMEoGCCsGAQUFBzAChj5odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY1RpbVN0YVBDQV8yMDEwLTA3LTAxLmNydDAMBgNVHRMBAf8EAjAAMBMGA1UdJQQMMAoGCCsGAQUFBwMIMA0GCSqGSIb3DQEBCwUAA4IBAQARQHu7ISeBuJSHKuDRI04704cH0B7BYzeEIrD15awviMRcYIfIOHpvGzZOWQgP2Hm0Rr7kvTUu1VrSSaQ7i1gPWdhqMmw5WBnSS5bxeMhhx9UsASeE84vUu82NeZapGSjH38YAb4WT+TtiTkcoI59rA+CTCq108ttIxVfZcr3id76OETIH0HvhlnxOOWjwGy4ul6Za5RoTLG/oo2rrGmVi3FwrNWGezYLBODuEsjzG36lCRtBKC2ZAHfbOz5wtkUHbqh79mUKocjP4r3qxf5TN87yf6g1uTx+J8pdnAi5iHt+ZtangWqnVTE8PoIREWhBVlGFfQdkELUx2Or90aAqWMIIGcTCCBFmgAwIBAgIKYQmBKgAAAAAAAjANBgkqhkiG9w0BAQsFADCBiDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEyMDAGA1UEAxMpTWljcm9zb2Z0IFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IDIwMTAwHhcNMTAwNzAxMjEzNjU1WhcNMjUwNzAxMjE0NjU1WjB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKkdDbx3EYo6IOz8E5f1+n9plGt0VBDVpQoAgoX77XxoSyxfxcPlYcJ2tz5mK1vwFVMnBDEfQRsalR3OCROOfGEwWbEwRA/xYIiEVEMM1024OAizQt2TrNZzMFcmgqNFDdDq9UeBzb8kYDJYYEbyWEeGMoQedGFnkV+BVLHPk0ySwcSmXdFhE24oxhr5hoC732H8RsEnHSRnEnIaIYqvS2SJUGKxXf13Hz3wV3WsvYpCTUBR0Q+cBj5nf/VmwAOWRH7v0Ev9buWayrGo8noqCjHw2k4GkbaICDXoeByw6ZnNPOcvRLqn9NxkvaQBwSAJk3jN/LzAyURdXhacAQVPIk0CAwEAAaOCAeYwggHiMBAGCSsGAQQBgjcVAQQDAgEAMB0GA1UdDgQWBBTVYzpcijGQ80N7fEYbxTNoWoVtVTAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBTV9lbLj+iiXGJo0T2UkFvXzpoYxDBWBgNVHR8ETzBNMEugSaBHhkVodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNSb29DZXJBdXRfMjAxMC0wNi0yMy5jcmwwWgYIKwYBBQUHAQEETjBMMEoGCCsGAQUFBzAChj5odHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpL2NlcnRzL01pY1Jvb0NlckF1dF8yMDEwLTA2LTIzLmNydDCBoAYDVR0gAQH/BIGVMIGSMIGPBgkrBgEEAYI3LgMwgYEwPQYIKwYBBQUHAgEWMWh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9QS0kvZG9jcy9DUFMvZGVmYXVsdC5odG0wQAYIKwYBBQUHAgIwNB4yIB0ATABlAGcAYQBsAF8AUABvAGwAaQBjAHkAXwBTAHQAYQB0AGUAbQBlAG4AdAAuIB0wDQYJKoZIhvcNAQELBQADggIBAAfmiFEN4sbgmD+BcQM9naOhIW+z66bM9TG+zwXiqf76V20ZMLPCxWbJat/15/B4vceoniXj+bzta1RXCCtRgkQS+7lTjMz0YBKKdsxAQEGb3FwX/1z5Xhc1mCRWS3TvQhDIr79/xn/yN31aPxzymXlKkVIArzgPF/UveYFl2am1a+THzvbKegBvSzBEJCI8z+0DpZaPWSm8tv0E4XCfMkon/VWvL/625Y4zu2JfmttXQOnxzplmkIz/amJ/3cVKC5Em4jnsGUpxY517IW3DnKOiPPp/fZZqkHimbdLhnPkd/DjYlPTGpQqWhqS9nhquBEKDuLWAmyI4ILUl5WTs9/S/fmNZJQ96LjlXdqJxqgaKD4kWumGnEcua2A5HmoDF0M2n0O99g/DhO3EJ3110mCIIYdqwUB5vvfHhAN/nMQekkzr3ZUd46PioSKv33nJ+YWtvd6mBy6cJrDm77MbL2IK0cs0d9LiFAR6A+xuJKlQ5slvayA1VmXqHczsI5pgt6o3gMy4SKfXAL1QnIffIrE7aKLixqduWsqdCosnPGUFN4Ib5KpqjEWYw07t0MkvfY3v1mYovG8chr1m1rtxEPJdQcdeh0sVV42neV8HR3jDA/czmTfsNv11P6Z0eGTgvvM9YBS7vDaBQNdrvCScc1bN+NR4Iuto229Nfj950iEkSoYICzjCCAjcCAQEwgfihgdCkgc0wgcoxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJXQTEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMS0wKwYDVQQLEyRNaWNyb3NvZnQgSXJlbGFuZCBPcGVyYXRpb25zIExpbWl0ZWQxJjAkBgNVBAsTHVRoYWxlcyBUU1MgRVNOOkQwODItNEJGRC1FRUJBMSUwIwYDVQQDExxNaWNyb3NvZnQgVGltZS1TdGFtcCBzZXJ2aWNloiMKAQEwBwYFKw4DAhoDFQByQCUheEOevaI9Zc/3QGrkX42iC6CBgzCBgKR+MHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMA0GCSqGSIb3DQEBBQUAAgUA36ppYDAiGA8yMDE4MTEyOTIxMzQyNFoYDzIwMTgxMTMwMjEzNDI0WjB3MD0GCisGAQQBhFkKBAExLzAtMAoCBQDfqmlgAgEAMAoCAQACAitfAgH/MAcCAQACAhGtMAoCBQDfq7rgAgEAMDYGCisGAQQBhFkKBAIxKDAmMAwGCisGAQQBhFkKAwKgCjAIAgEAAgMHoSChCjAIAgEAAgMBhqAwDQYJKoZIhvcNAQEFBQADgYEAbAXXPR9wy4NA0892GGqetaZF+pNClpGcfEpSuHABaZ4Gzr1nY1nmrhexTtr/U6omHALRWzkQwthk0cy+mnEHXyOZGmoEEpgrLgK3AAP5NbK/XbtHQRyZJQyhZScFbOyQycoE8QQalSVOhWxk/bbBMQaQiYVMIexNd/T0KgaDDUMxggMNMIIDCQIBATCBkzB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMAITMwAAAOIYOHtm6erB2AAAAAAA4jANBglghkgBZQMEAgEFAKCCAUowGgYJKoZIhvcNAQkDMQ0GCyqGSIb3DQEJEAEEMC8GCSqGSIb3DQEJBDEiBCCr9IiSbx6s8MLdxldRG49+4h6CbicW8hWXAicI3jNmhDCB+gYLKoZIhvcNAQkQAi8xgeowgecwgeQwgb0EIN8BpJSmQCGubWwVa4tW+aMveoHMX/nDnVN8fiDOMsrLMIGYMIGApH4wfDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEmMCQGA1UEAxMdTWljcm9zb2Z0IFRpbWUtU3RhbXAgUENBIDIwMTACEzMAAADiGDh7ZunqwdgAAAAAAOIwIgQgTkOfRvGEZNbr5/hgWclsL4/Q7SOZihE/U0lz2wEMIGcwDQYJKoZIhvcNAQELBQAEggEATlxnCfTzFfTMDvK085zlYPVCroKYW6gKFYnbAhNmrNzcxqALKmIYXpFU7B6HH/vYzkUfCyXpf5tsyEWu0oTySOjyAZ9+2vdaG8nEgjOp0L737lcitgusIjpWtta3Ik0b+mzffnvyjrgTSuKDDni3mxGfvJU77k1Ctempma4H2FJso6Bur0PRH99vIYDu4lHigOSLbeyjR5CiDciBwEVUSA0FxhoFNX1yfpxz3sukOvkaoTduREIjH5LxUjNI1ZTMK/ZkeETI8IPRpWVzAc8q7CujErHKo4sdKej/O2cfUTUHplFLVCGGExpJUCg5FH5jVUUFt75ad8503sdGplggVQ== + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + MIIucgYJKoZIhvcNAQcCoIIuYzCCLl8CAQExDzANBglghkgBZQMEAgEFADCCARAGCSsGAQQBgjcKAaCCAQEwgf4wDAYKKwYBBAGCNwwBAQQQ293slyDTMUOmVyeQqmcNSBcNMjIwMjA0MDkwNjU2WjAOBgorBgEEAYI3DAEDBQAwgbwwKgQUdgFpbn9QXi/ly4CZFKA2Eimoq6YxEjAQBgorBgEEAYI3DAIDMQKCADCBjQQg3YHMbvV3unL0mx/RJ8ihTJd1C/SYSnMHbN0yMrWijuQxaTAQBgorBgEEAYI3DAIDMQKCADBVBgorBgEEAYI3AgEEMUcwRTAQBgorBgEEAYI3AgEZogKAADAxMA0GCWCGSAFlAwQCAQUABCDdgcxu9Xe6cvSbH9EnyKFMl3UL9JhKcwds3TIytaKO5KCCE8owggYEMIID7KADAgECAhMzAA/d9uEraWrDsxZGAAAAD932MA0GCSqGSIb3DQEBCwUAMIGLMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMQwwCgYDVQQLEwNBT0MxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAyMjAeFw0yMjAyMDQwODU5MzZaFw0yMjAyMDcwODU5MzZaMC8xLTArBgNVBAMTJDQ2ZTIwYzY2LTUxNTEtNDNhOS05MWQ4LTMwNDY0ZGZhMDg4ZjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALaMR5Ajlp2POidTxT0N47z5sukiE9fqMuI1QimTXIfB2/gPdMsNUbVCD57ha8J1DUJQpO5nuunbk3SONwfDTdWKx9u7zWubpEmgG7hZ8LTKFlF+xhS6lS/FBSSHyXpdScWPg62BMnDysqHMH/AjLw26HNPRWK8A0vx9jsalFwLg15u4MPuKN3Bpawr/OL0B+7eh/dGO3PutAqJ4aZs2lUCIyODg0q3Tzhgi7SIvFacFWJ8Qj6+D3AfasOv8oanfpNLLPhPlxXGEK9sMKHOOb8mTU9V/ibERqEKTHkJ24Vu+BwrXq5eVedtwmHT2WWR7teaIvrUT9AiPmMhx4hIF/0MCAwEAAaOCAbowggG2MBgGA1UdJQEB/wQOMAwGCisGAQQBgjdMHAEwHQYDVR0OBBYEFB4NpkpeSLk/j691tPw97rB4XKDJMA0GA1UdEQQGMASCAlVTMB8GA1UdIwQYMBaAFPeC7EkBPjJR1eFAcP4hAMeQ72IPMF8GA1UdHwRYMFYwVKBSoFCGTmh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY3JsL01pY3Jvc29mdCUyME1hcmtldHBsYWNlJTIwQ0ElMjBHJTIwMDIyLmNybDBsBggrBgEFBQcBAQRgMF4wXAYIKwYBBQUHMAKGUGh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2lvcHMvY2VydHMvTWljcm9zb2Z0JTIwTWFya2V0cGxhY2UlMjBDQSUyMEclMjAwMjIuY3J0MAwGA1UdEwEB/wQCMAAwDgYDVR0PAQH/BAQDAgXgMDwGCSsGAQQBgjcVBwQvMC0GJSsGAQQBgjcVCIOS9kTqrxCDkY0wgqzgLIKinDE0g+6NOIaE7wACAWQCARcwIAYJKwYBBAGCNxUKAQH/BBAwDjAMBgorBgEEAYI3TBwBMA0GCSqGSIb3DQEBCwUAA4ICAQBAiJuIJeMS+PMrxG+kwPWT7yy7A6P8trchwtXyYep/WXu7xT2tH2ys3wuP4DYG5CY4nKekjTKraQhZGuYj4r4KrUviShe8ZWMIcK4MRduXAU81UOd/fsCq5djQ6NDN4Bzq1TxjwZKiEMEWmXaunj6XAlDSX3SgyyMb/ywmur8VVKO+xFVkOxAqCjNo1VaCv3zEvr3Y6dUwO9gYQui1bAQVQzdxRYUPEJlsFBmbNL1AnZ39r4/9K/6orxavU4qwkj4cXnhtMYKDjTIIbTBrfD4glP6mnmkcZpN0ItfEAVgU9rGpXg84hQLeP83nBs2Y8DrF3bBF9867dJCl90c5rK1DMcelmH3oUAwqZ3U+jIIj3HGyfefAQ6HPL0yiY9OYLXQZUxcZF13m92l5s7dy2C21cIh0W0iTcWPIhNn9cia4Hr7FabTC0RoQkO1bg0PKvPMnoe0AqEVmPKImNDncg538AjPy2qeUxn7+0kcuZWEdKVAPwWfzh5qvSAmYKk4XNBfN2E40GQ0ruRdvMVMFeYBLXKm4SJR6zoiBa5v7qzt1uQqZf6GD4BNFZivBp0P2B9lTubEAU6SmUsazRnEVmekIO+x7gDuXbMS8tv6mb1pbGCtIV/wRcIYNcDkHKTHMbW5xR/OmGTM3mPRxoMQtaWtxIeagZkU2nl+pElA/0h7pIjCCBtcwggS/oAMCAQICCmESRKIAAAAAAAIwDQYJKoZIhvcNAQELBQAwgYgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xMjAwBgNVBAMTKU1pY3Jvc29mdCBSb290IENlcnRpZmljYXRlIEF1dGhvcml0eSAyMDExMB4XDTExMDMyODIxMDkzOVoXDTMxMDMyODIxMTkzOVowfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAubUaSwGYVsE3MAnPfvmozUhAB3qxBABgJRW1vDp4+tVinXxD32f7k1K89JQ6zDOgS/iDgULC+yFK1K/1Qjac/0M7P6c8v5LSjnWGlERLa/qY32j46S7SLQcit3g2jgoTTO03eUG+9yHZUTGV/FJdRYB8uXhrznJBa+Y+yGwiQKF+m6XFeBH/KORoKFx+dmMoy9EWJ/m/o9IiUj2kzm9C691+vZ/I2w0Bj93W9SPPkV2PCNHlzgfIAoeajWpHmi38Wi3xZHonkzAVBHxPsCBppOoNsWvmAfUM7eBthkSPvFruekyDCPNEYhfGqgqtqLkoBebXLZCOVybF7wTQaLvse60//3P003icRcCoQYgY4NAqrF7j80o5U7DkeXxcB0xvengsaKgiAaV1DKkRbpe98wCqr1AASvm5rAJUYMU+mXmOieV2EelY2jGrenWe9FQpNXYV1NoWBh0WKoFxttoWYAnF705bIWtSZsz08ZfK6WLX4GXNLcPBlgCzfTm1sdKYASWdBbH2haaNhPapFhQQBJHKwnVW2iXErImhuPi45W3MVTZ5D9ASshZx69cLYY6xAdIa+89Kf/uRrsGOVZfahDuDw+NI183iAyzC8z/QRt2P32LYxP0xrCdqVh+DJo2i4NoE8Uk1usCdbVRuBMBQl/AwpOTq7IMvHGElf65CqzUCAwEAAaOCAUswggFHMBAGCSsGAQQBgjcVAQQDAgEAMB0GA1UdDgQWBBQPU8s/FmEl/mCJHdO5fOiQrbOU0TAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBRyLToCMZBDuRQFTuHqp8cx0SOJNDBaBgNVHR8EUzBRME+gTaBLhklodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3JsMF4GCCsGAQUFBwEBBFIwUDBOBggrBgEFBQcwAoZCaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNSb29DZXJBdXQyMDExXzIwMTFfMDNfMjIuY3J0MA0GCSqGSIb3DQEBCwUAA4ICAQCjuZmM8ZVNDgp9wHsL4RY8KJ8nLinvxFTphNGCrxaLknkYG5pmMhVlX+UB/tSiW8W13W60nggz9u5xwMx7v/1t/Tgm6g2brVyOKI5A7u6/2SIJwkJKFw953K0YIKVT28w9zl8dSJnmRnyR0G86ncWbF6CLQ6A6lBQ9o2mTGVqDr4m35WKAnc6YxUUM1y74mbzFFZr63VHsCcOp3pXWnUqAY1rb6Q6NX1b3clncKqLFm0EjKHcQ56grTbwuuB7pMdh/IFCJR01MQzQbDtpEisbOeZUi43YVAAHKqI1EO9bRwg3frCjwAbml9MmI4utMW94gWFgvrMxIX+n42RBDIjf3Ot3jkT6gt3XeTTmO9bptgblZimhERdkFRUFpVtkocJeLoGuuzP93uH/Yp032wzRH+XmMgujfZv+vnfllJqxdowoQLx55FxLLeTeYfwi/xMSjZO2gNven3U/3KeSCd1kUOFS3AOrwZ0UNOXJeW5JQC6Vfd1BavFZ6FAta1fMLu3WFvNB+FqeHUaU3ya7rmtxJnzk29DeSqXgGNmVSywBS4NajI5jJIKAA6UhNJlsg8CHYwUOKf5ej8OoQCkbadUxXygAfxCfW2YBbujtI+PoyejRFxWUjYFWO5LeTI62UMyqfOEiqugoYjNxmQZla2s4YHVuqIC34R85FQlg9pKQBsDCCBuMwggTLoAMCAQICEzMAAABLZz0Eed1JFCkAAAAAAEswDQYJKoZIhvcNAQELBQAwfTELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldFBsYWNlIFBDQSAyMDExMB4XDTE5MDQxODIwNDUwNFoXDTI0MDQxODIwNDUwNFowgYsxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xDDAKBgNVBAsTA0FPQzEnMCUGA1UEAxMeTWljcm9zb2Z0IE1hcmtldHBsYWNlIENBIEcgMDIyMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAzGfV5tDxZ3oPG8aq2OwrBLSRFC61lYvWtkqOdKswlBHVncuoOIzekLIFNsMwluoK0plNZ5w/JPmwkGTGIrt9ftbWQz0k7XRfacXvQQfhCaBgjkybvvDx7CTL0UDEUiXfEEkylUcQKOdm9LApPDP4oO/V8RS4ugkqkjsuVaOyfYv0TytjVVQon72W+wR/AyrBAXgfwzzEb85403GTJRzQMlTw1YqdgS/o9SCsvH9dRZTGlWCatIl+a00eTG5zgeu0xCtqwgERhw9UT6mlPqmp6RqJ8XMgKylh9Ss+jI53EmraAFlelph16kuqF6n3vNnRBfhQ+gmNdtKi9s3jI8Di7ip8hYGPHGGhtcw0AYUg5r/VQhIfXxO45zO7SFvjujX0ji0b3WKB/xD9Sg1KO1fiTpj82ifyPqHvL+iwn5dV98C4ru0dQ0hfK7tA4K5qkW+2gym0fBjGYWq81/smrj6LbPdNqCotgTMU4dCMBhmSJOD9fSiPUybrbHJrHGR5YNs5VJbHH5u1ia8UrgsSrGTb1bh38Tb10matJpBI7NwDWTkV7kMvgNRQsGO/1+dg47PLW3X7TbiZniTPC+oAqOh1CiF+ODnem4Nw1iRw854LN3yqibLHgTi093mCBtPGWZxg2sTcHUX4fo/UHNbSh8bERJznnennm4gEJXAYtpnEFSUCAwEAAaOCAUswggFHMA4GA1UdDwEB/wQEAwIBhjAQBgkrBgEEAYI3FQEEAwIBADAdBgNVHQ4EFgQU94LsSQE+MlHV4UBw/iEAx5DvYg8wGQYJKwYBBAGCNxQCBAweCgBTAHUAYgBDAEEwEgYDVR0TAQH/BAgwBgEB/wIBADAfBgNVHSMEGDAWgBQPU8s/FmEl/mCJHdO5fOiQrbOU0TBXBgNVHR8EUDBOMEygSqBIhkZodHRwOi8vY3JsLm1pY3Jvc29mdC5jb20vcGtpL2NybC9wcm9kdWN0cy9NaWNNYXJQQ0EyMDExXzIwMTEtMDMtMjguY3JsMFsGCCsGAQUFBwEBBE8wTTBLBggrBgEFBQcwAoY/aHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNNYXJQQ0EyMDExXzIwMTEtMDMtMjguY3J0MA0GCSqGSIb3DQEBCwUAA4ICAQAbTWps31OnBv77EVpCaU68drubToSKtpw9fYt4MvAhorXGRrRRfS6qou5o45QZAKjwzbOrr33dWxF4P3gCGqZxCotW32p6hlXGP7BnYlVw6242nlX0PahgpyWVWBmi0ElKH8N6HYhJqAW74tUNCzY7ALNMwk/JVEMKpWq+5XqutAEmtP0SdoSZPRwPtdiRjMLS8yyjP4HVAiKJ+m0wV3bLBtCG20BttEGkly7fYa2/E0ps41PLeX3T2yKRgmnGjhD16NUcMDdRpBh7MCXktwpISDEanB8QySJNdErU+NaKukgOW68+Oyk7lGiiZcG3boCGNec9JkM7dQdpA4tneQg0FI/o0mkDZR5aVItbAjRfMgAXpm6Hfu9DqOP+/R0K9v+CwyhwEjF3+SeL8YBAijXHD5YaiWzwZzBDtNQpuOXc91NmF7ifdHVy9IE1EhuIIQsMio3l0Uo87vGdwQ50po8o1F72zi1vfvKfDmszVezk6fF7cy7fZYh8I0ceVbZ4XY4xJ76TdYMItUAoPkvVdDq5B/d4oI7/xHfomRsEw08fuLxCRE8/BcvzSlDpnxu4wL6IULqVwHE+fbQ82Lr5ZDLxa1dYUw8zBGgCnx/ObVPj1RO1CaVi13wCxm8yaHeflKJWw7ZHOLeOGQyMRT+f9cFFk+5fzCA9GcwiSlEyYj4ilzGCGWUwghlhAgEBMIGjMIGLMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMQwwCgYDVQQLEwNBT0MxJzAlBgNVBAMTHk1pY3Jvc29mdCBNYXJrZXRwbGFjZSBDQSBHIDAyMgITMwAP3fbhK2lqw7MWRgAAAA/d9jANBglghkgBZQMEAgEFAKCBjzAYBgkqhkiG9w0BCQMxCwYJKwYBBAGCNwoBMC8GCSqGSIb3DQEJBDEiBCBDgOMhvMousTIzk/P1eC2qGiSJLdSm3brGanFMLa41+DBCBgorBgEEAYI3AgEMMTQwMqAUgBIATQBpAGMAcgBvAHMAbwBmAHShGoAYaHR0cDovL3d3dy5taWNyb3NvZnQuY29tMA0GCSqGSIb3DQEBAQUABIIBAJ8LEwzLh2rDfLOOOodKT10yXoCzHPaQLv9sMwei+650mqV0iHaw9bUolBZAdBnF7JBRjY+yb9AIF7M/Av9pHRKvuLHWDntlo49RrfvNucHaJqPjCMm81h/GAuimPSBMpRtTCW+HHDWrw+HoGbRWQ+N/Vpt//pzj9gAzOzs8W9gU0RMz3ub4Hfgunm/CGACjFu5Hi2sjJbgigNlyRnxjI4+h3J7dGVOfR8hlhRB9YjXDB700F0bzaKqxC9LH7kZbAjiHPgLLCWz7OQ/rTo7wV5/e6v9GCKpamydPy/0MObq4TDi0fn75aVHTJJrzlwt1BJKyVPdAsOhiFUH3Mq4WheKhghcAMIIW/AYKKwYBBAGCNwMDATGCFuwwghboBgkqhkiG9w0BBwKgghbZMIIW1QIBAzEPMA0GCWCGSAFlAwQCAQUAMIIBUQYLKoZIhvcNAQkQAQSgggFABIIBPDCCATgCAQEGCisGAQQBhFkKAwEwMTANBglghkgBZQMEAgEFAAQgr34HNu5gF3j5DAITqq38vc6sNxs6ZqLz9i9zU2oQDtoCBmH66bcTCRgTMjAyMjAyMDQwOTA5NDkuNTkxWjAEgAIB9KCB0KSBzTCByjELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjElMCMGA1UECxMcTWljcm9zb2Z0IEFtZXJpY2EgT3BlcmF0aW9uczEmMCQGA1UECxMdVGhhbGVzIFRTUyBFU046REQ4Qy1FMzM3LTJGQUUxJTAjBgNVBAMTHE1pY3Jvc29mdCBUaW1lLVN0YW1wIFNlcnZpY2WgghFXMIIHDDCCBPSgAwIBAgITMwAAAZwPpk1h0p5LKAABAAABnDANBgkqhkiG9w0BAQsFADB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMDAeFw0yMTEyMDIxOTA1MTlaFw0yMzAyMjgxOTA1MTlaMIHKMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSUwIwYDVQQLExxNaWNyb3NvZnQgQW1lcmljYSBPcGVyYXRpb25zMSYwJAYDVQQLEx1UaGFsZXMgVFNTIEVTTjpERDhDLUUzMzctMkZBRTElMCMGA1UEAxMcTWljcm9zb2Z0IFRpbWUtU3RhbXAgU2VydmljZTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBANtSKgwZXUkWP6zrXazTaYq7bco9Q2zvU6MN4ka3GRMX2tJZOK4DxeBiQACL/n7YV/sKTslwpD0f9cPU4rCDX9sfcTWo7XPxdHLQ+WkaGbKKWATsqw69bw8hkJ/bjcp2V2A6vGsvwcqJCh07BK3JPmUtZikyy5PZ8fyTyiKGN7hOWlaIU9oIoucUNoAHQJzLq8h20eNgHUh7eI5k+Kyq4v6810LHuA6EHyKJOZN2xTw5JSkLy0FN5Mhg/OaFrFBl3iag2Tqp4InKLt+Jbh/Jd0etnei2aDHFrmlfPmlRSv5wSNX5zAhgEyRpjmQcz1zp0QaSAefRkMm923/ngU51IbrVbAeHj569SHC9doHgsIxkh0K3lpw582+0ONXcIfIU6nkBT+qADAZ+0dT1uu/gRTBy614QAofjo258TbSX9aOU1SHuAC+3bMoyM7jNdHEJROH+msFDBcmJRl4VKsReI5+S69KUGeLIBhhmnmQ6drF8Ip0ZiO+vhAsD3e9AnqnY7Hcge850I9oKvwuwpVwWnKnwwSGElMz7UvCocmoUMXk7Vn2aNti+bdH28+GQb5EMsqhOmvuZOCRpOWN33G+b3g5unwEP0eTiY+LnWa2AuK43z/pplURJVle29K42QPkOcglB6sjLmNpEpb9basJ72eA0Mlp1LtH3oYZGXsggTfuXAgMBAAGjggE2MIIBMjAdBgNVHQ4EFgQUu2kJZ1Ndjl2112SynL6jGMID+rIwHwYDVR0jBBgwFoAUn6cVXQBeYl2D9OXSZacbUzUZ6XIwXwYDVR0fBFgwVjBUoFKgUIZOaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9jcmwvTWljcm9zb2Z0JTIwVGltZS1TdGFtcCUyMFBDQSUyMDIwMTAoMSkuY3JsMGwGCCsGAQUFBwEBBGAwXjBcBggrBgEFBQcwAoZQaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraW9wcy9jZXJ0cy9NaWNyb3NvZnQlMjBUaW1lLVN0YW1wJTIwUENBJTIwMjAxMCgxKS5jcnQwDAYDVR0TAQH/BAIwADATBgNVHSUEDDAKBggrBgEFBQcDCDANBgkqhkiG9w0BAQsFAAOCAgEApwAqpiMYRzNNYyz3PSbtijbeyCpUXcvIrqA4zPtMIcAk34W9u9mRDndWS+tlR3WwTpr1OgaV1wmc6YFzqK6EGWm903UEsFE7xBJMPXjfdVOPhcJB3vfvA0PX56oobcF2OvNsOSwTB8bi/ns+Cs39Puzs+QSNQZd8iAVBCSvxNCL78dln2RGU1xyB4AKqV9vi4Y/Gfmx2FA+jF0y+YLeob0M40nlSxL0q075t7L6iFRMNr0u8ROhzhDPLl+4ePYfUmyYJoobvydel9anAEsHFlhKl+aXb2ic3yNwbsoPycZJL/vo8OVvYYxCy+/5FrQmAvoW0ZEaBiYcKkzrNWt/hX9r5KgdwL61x0ZiTZopTko6W/58UTefTbhX7Pni0MApH3Pvyt6N0IFap+/LlwFRD1zn7e6ccPTwESnuo/auCmgPznq80OATA7vufsRZPvqeX8jKtsraSNscvNQymEWlcqdXV9hYkjb4T/Qse9cUYaoXg68wFHFuslWfTdPYPLl1vqzlPMnNJpC8KtdioDgcq+y1BaSqSm8EdNfwzT37+/JFtVc3Gs915fDqgPZDgOSzKQIV+fw3aPYt2LET3AbmKKW/r13Oy8cg3+D0D362GQBAJVv0NRI5NowgaCw6oNgWOFPrN72WSEcca/8QQiTGP2XpLiGpRDJZ6sWRpRYNdydkwggdxMIIFWaADAgECAhMzAAAAFcXna54Cm0mZAAAAAAAVMA0GCSqGSIb3DQEBCwUAMIGIMQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMTIwMAYDVQQDEylNaWNyb3NvZnQgUm9vdCBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkgMjAxMDAeFw0yMTA5MzAxODIyMjVaFw0zMDA5MzAxODMyMjVaMHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA5OGmTOe0ciELeaLL1yR5vQ7VgtP97pwHB9KpbE51yMo1V/YBf2xK4OK9uT4XYDP/XE/HZveVU3Fa4n5KWv64NmeFRiMMtY0Tz3cywBAY6GB9alKDRLemjkZrBxTzxXb1hlDcwUTIcVxRMTegCjhuje3XD9gmU3w5YQJ6xKr9cmmvHaus9ja+NSZk2pg7uhp7M62AW36MEBydUv626GIl3GoPz130/o5Tz9bshVZN7928jaTjkY+yOSxRnOlwaQ3KNi1wjjHINSi947SHJMPgyY9+tVSP3PoFVZhtaDuaRr3tpK56KTesy+uDRedGbsoy1cCGMFxPLOJiss254o2I5JasAUq7vnGpF1tnYN74kpEeHT39IM9zfUGaRnXNxF803RKJ1v2lIH1+/NmeRd+2ci/bfV+AutuqfjbsNkz2K26oElHovwUDo9Fzpk03dJQcNIIP8BDyt0cY7afomXw/TNuvXsLz1dhzPUNOwTM5TI4CvEJoLhDqhFFG4tG9ahhaYQFzymeiXtcodgLiMxhy16cg8ML6EgrXY28MyTZki1ugpoMhXV8wdJGUlNi5UPkLiWHzNgY1GIRH29wb0f2y1BzFa/ZcUlFdEtsluq9QBXpsxREdcu+N+VLEhReTwDwV2xo3xwgVGD94q0W29R6HXtqPnhZyacaue7e3PmriLq0CAwEAAaOCAd0wggHZMBIGCSsGAQQBgjcVAQQFAgMBAAEwIwYJKwYBBAGCNxUCBBYEFCqnUv5kxJq+gpE8RjUpzxD/LwTuMB0GA1UdDgQWBBSfpxVdAF5iXYP05dJlpxtTNRnpcjBcBgNVHSAEVTBTMFEGDCsGAQQBgjdMg30BATBBMD8GCCsGAQUFBwIBFjNodHRwOi8vd3d3Lm1pY3Jvc29mdC5jb20vcGtpb3BzL0RvY3MvUmVwb3NpdG9yeS5odG0wEwYDVR0lBAwwCgYIKwYBBQUHAwgwGQYJKwYBBAGCNxQCBAweCgBTAHUAYgBDAEEwCwYDVR0PBAQDAgGGMA8GA1UdEwEB/wQFMAMBAf8wHwYDVR0jBBgwFoAU1fZWy4/oolxiaNE9lJBb186aGMQwVgYDVR0fBE8wTTBLoEmgR4ZFaHR0cDovL2NybC5taWNyb3NvZnQuY29tL3BraS9jcmwvcHJvZHVjdHMvTWljUm9vQ2VyQXV0XzIwMTAtMDYtMjMuY3JsMFoGCCsGAQUFBwEBBE4wTDBKBggrBgEFBQcwAoY+aHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9jZXJ0cy9NaWNSb29DZXJBdXRfMjAxMC0wNi0yMy5jcnQwDQYJKoZIhvcNAQELBQADggIBAJ1VffwqreEsH2cBMSRb4Z5yS/ypb+pcFLY+TkdkeLEGk5c9MTO1OdfCcTY/2mRsfNB1OW27DzHkwo/7bNGhlBgi7ulmZzpTTd2YurYeeNg2LpypglYAA7AFvonoaeC6Ce5732pvvinLbtg/SHUB2RjebYIM9W0jVOR4U3UkV7ndn/OOPcbzaN9l9qRWqveVtihVJ9AkvUCgvxm2EhIRXT0n4ECWOKz3+SmJw7wXsFSFQrP8DJ6LGYnn8AtqgcKBGUIZUnWKNsIdw2FzLixre24/LAl4FOmRsqlb30mjdAy87JGA0j3mSj5mO0+7hvoyGtmW9I/2kQH2zsZ0/fZMcm8Qq3UwxTSwethQ/gpY3UA8x1RtnWN0SCyxTkctwRQEcb9k+SS+c23Kjgm9swFXSVRk2XPXfx5bRAGOWhmRaw2fpCjcZxkoJLo4S5pu+yFUa2pFEUep8beuyOiJXk+d0tBMdrVXVAmxaQFEfnyhYWxz/gq77EFmPWn9y8FBSX5+k77L+DvktxW/tM4+pTFRhLy/AsGConsXHRWJjXD+57XQKBqJC4822rpM+Zv/Cuk0+CQ1ZyvgDbjmjJnW4SLq8CdCPSWU5nR0W2rRnj7tfqAxM328y+l7vzhwRNGQ8cirOoo6CGJ/2XBjU02N7oJtpQUQwXEGahC0HVUzWLOhcGbyoYICzjCCAjcCAQEwgfihgdCkgc0wgcoxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJTAjBgNVBAsTHE1pY3Jvc29mdCBBbWVyaWNhIE9wZXJhdGlvbnMxJjAkBgNVBAsTHVRoYWxlcyBUU1MgRVNOOkREOEMtRTMzNy0yRkFFMSUwIwYDVQQDExxNaWNyb3NvZnQgVGltZS1TdGFtcCBTZXJ2aWNloiMKAQEwBwYFKw4DAhoDFQDN2Wnq3fCz9ucStub1zQz7129TQKCBgzCBgKR+MHwxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdSZWRtb25kMR4wHAYDVQQKExVNaWNyb3NvZnQgQ29ycG9yYXRpb24xJjAkBgNVBAMTHU1pY3Jvc29mdCBUaW1lLVN0YW1wIFBDQSAyMDEwMA0GCSqGSIb3DQEBBQUAAgUA5adiZjAiGA8yMDIyMDIwNDE2MjkyNloYDzIwMjIwMjA1MTYyOTI2WjB3MD0GCisGAQQBhFkKBAExLzAtMAoCBQDlp2JmAgEAMAoCAQACAh92AgH/MAcCAQACAhFfMAoCBQDlqLPmAgEAMDYGCisGAQQBhFkKBAIxKDAmMAwGCisGAQQBhFkKAwKgCjAIAgEAAgMHoSChCjAIAgEAAgMBhqAwDQYJKoZIhvcNAQEFBQADgYEAbKzHReDTwgAKGSkgO/zxCAV/RcxwGpZK4M2bWgwqHVIi+/xiAKdQOSKeMz3UwRO4LI4EN4BjeK+ATpruzOtVGkyA8yzwdJFPv99j4O/Hc5kJSvovfECah62tBB3siNIFaAYNK+BF76k3j4+y9q9lF5yjQ8ZylVYbOfOjlJ2uevsxggQNMIIECQIBATCBkzB8MQswCQYDVQQGEwJVUzETMBEGA1UECBMKV2FzaGluZ3RvbjEQMA4GA1UEBxMHUmVkbW9uZDEeMBwGA1UEChMVTWljcm9zb2Z0IENvcnBvcmF0aW9uMSYwJAYDVQQDEx1NaWNyb3NvZnQgVGltZS1TdGFtcCBQQ0EgMjAxMAITMwAAAZwPpk1h0p5LKAABAAABnDANBglghkgBZQMEAgEFAKCCAUowGgYJKoZIhvcNAQkDMQ0GCyqGSIb3DQEJEAEEMC8GCSqGSIb3DQEJBDEiBCD10XWSucJFd8jLe5PaC9h9/GHXK7ZZ0kb4BGFNqgWk9DCB+gYLKoZIhvcNAQkQAi8xgeowgecwgeQwgb0EIDcPRYUgjSzKOhF39d4QgbRZQgrPO7Lo/qE5GtvSeqa8MIGYMIGApH4wfDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCldhc2hpbmd0b24xEDAOBgNVBAcTB1JlZG1vbmQxHjAcBgNVBAoTFU1pY3Jvc29mdCBDb3Jwb3JhdGlvbjEmMCQGA1UEAxMdTWljcm9zb2Z0IFRpbWUtU3RhbXAgUENBIDIwMTACEzMAAAGcD6ZNYdKeSygAAQAAAZwwIgQgPTEmgx6rn5xpH+xmlpYkcn0dL16GyYRqyG8uRajrfTYwDQYJKoZIhvcNAQELBQAEggIAxmSLisdd5cs+7liXX7vsF8zX3Y1rbtyPZ0M466WBaVULlfeuu9RMzOhWha7bkfYmKs5hXOQNBJrIHxnHbWeDu/V7EVF80IQ7/6qWiVPNcLtMnS8+uyw3Tby2gEhshO/HHOJRqllrgtUGuPl8FoAAPbPKV7X7jrg1kqOWoqpwyiSzt5cN/qijwfTcTIaPeON5wMd8wRT2l5F9e/JDBSq0WuzYb2r0kifv9wq21orm4sQkbg5GgDa/oCr3PQVCNSvPb2G7HANcqJXHeRKyoCeVhX5UaFrCQbdxGFytOc5GzptQKNn18yGg4VYrFpigUcglLbuREqqlPabwZh/QfgKdp4b4uVA6On5HIV5e9XdJ9TXhvDArEQ+Kgyao5xlgb4MkvPO9Thh9UIbw9r3JhQ+uKnCvXyiE2SpTLq1OXTjt/RrLZb+qsZW4nrDRBlBswa4G5OVW/svvI8N2/+YYrSdaCWt4unWSnSxeVUWZkugBmWdeoEJpfs5Qy8NwC+g1TyXcMjolOFTfBB6JYpHCbk6Euh2gKxq5weBb7puxVN4cvawYY2zorp6ZXtAmQGflZTl02nWld+nvDoF/zi6IUmP7PTKvbEVs8T+QkCiIM7cpLMs4NFK29f33lu1iZeGwJAxCYsV/P7geuRfA++bpDutTXjLKMqUTgf00sqJ/5HGuwLE= From webhook-mailer at python.org Fri Feb 4 12:57:13 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 04 Feb 2022 17:57:13 -0000 Subject: [Python-checkins] bpo-46608: exclude marshalled-frozen data if deep-freezing to save 300 KB space (GH-31074) Message-ID: https://github.com/python/cpython/commit/bf95ff91f2c1fc5a57190491f9ccdc63458b089e commit: bf95ff91f2c1fc5a57190491f9ccdc63458b089e branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-04T09:57:03-08:00 summary: bpo-46608: exclude marshalled-frozen data if deep-freezing to save 300 KB space (GH-31074) This reduces the size of the data segment by **300 KB** of the executable because if the modules are deep-frozen then the marshalled frozen data just wastes space. This was inspired by comment by @gvanrossum in https://github.com/python/cpython/pull/29118#issuecomment-958521863. Note: There is a new option `--deepfreeze-only` in `freeze_modules.py` to change this behavior, it is on be default to save disk space. ```console # du -s ./python before 27892 ./python # du -s ./python after 27524 ./python ``` Automerge-Triggered-By: GH:ericsnowcurrently files: A Misc/NEWS.d/next/Build/2022-02-02-11-26-46.bpo-46608.cXH9po.rst M Doc/c-api/import.rst M Doc/whatsnew/3.11.rst M Include/cpython/import.h M Lib/ctypes/test/test_values.py M Python/frozen.c M Python/import.c M Tools/freeze/makefreeze.py M Tools/scripts/freeze_modules.py diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index d2ae6b6d4e471..5e2333a74ce64 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -256,8 +256,12 @@ Importing Modules const char *name; const unsigned char *code; int size; + bool is_package; }; + .. versionchanged:: 3.11 + The new ``is_package`` field indicates whether the module is a package or not. + This replaces setting the ``size`` field to a negative value. .. c:var:: const struct _frozen* PyImport_FrozenModules diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 18d4652fb4293..1558d67d9a89f 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -650,6 +650,11 @@ C API Changes fields of the result from the exception instance (the ``value`` field). (Contributed by Irit Katriel in :issue:`45711`.) +* :c:struct:`_frozen` has a new ``is_package`` field to indicate whether + or not the frozen module is a package. Previously, a negative value + in the ``size`` field was the indicator. Now only non-negative values + be used for ``size``. + (Contributed by Kumar Aditya in :issue:`46608`.) New Features ------------ diff --git a/Include/cpython/import.h b/Include/cpython/import.h index 5ec637e7ab3b8..da9fb770a9245 100644 --- a/Include/cpython/import.h +++ b/Include/cpython/import.h @@ -32,6 +32,7 @@ struct _frozen { const char *name; /* ASCII encoded string */ const unsigned char *code; int size; + bool is_package; PyObject *(*get_code)(void); }; diff --git a/Lib/ctypes/test/test_values.py b/Lib/ctypes/test/test_values.py index 3e8b13768b421..b2db426d10f43 100644 --- a/Lib/ctypes/test/test_values.py +++ b/Lib/ctypes/test/test_values.py @@ -54,6 +54,7 @@ class struct_frozen(Structure): _fields_ = [("name", c_char_p), ("code", POINTER(c_ubyte)), ("size", c_int), + ("is_package", c_bool), ("get_code", POINTER(c_ubyte)), # Function ptr ] FrozenTable = POINTER(struct_frozen) @@ -71,13 +72,14 @@ class struct_frozen(Structure): modname = entry.name.decode("ascii") modules.append(modname) with self.subTest(modname): - # Do a sanity check on entry.size and entry.code. - self.assertGreater(abs(entry.size), 10) - self.assertTrue([entry.code[i] for i in range(abs(entry.size))]) + if entry.size != 0: + # Do a sanity check on entry.size and entry.code. + self.assertGreater(abs(entry.size), 10) + self.assertTrue([entry.code[i] for i in range(abs(entry.size))]) # Check the module's package-ness. with import_helper.frozen_modules(): spec = importlib.util.find_spec(modname) - if entry.size < 0: + if entry.is_package: # It's a package. self.assertIsNotNone(spec.submodule_search_locations) else: diff --git a/Misc/NEWS.d/next/Build/2022-02-02-11-26-46.bpo-46608.cXH9po.rst b/Misc/NEWS.d/next/Build/2022-02-02-11-26-46.bpo-46608.cXH9po.rst new file mode 100644 index 0000000000000..13c73a614e5e8 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2022-02-02-11-26-46.bpo-46608.cXH9po.rst @@ -0,0 +1,2 @@ +Exclude marshalled-frozen data if deep-freezing to save 300 KB disk space. This includes adding +a new ``is_package`` field to :c:struct:`_frozen`. Patch by Kumar Aditya. diff --git a/Python/frozen.c b/Python/frozen.c index 25cf0a8d37c78..c5b36f73b4a47 100644 --- a/Python/frozen.c +++ b/Python/frozen.c @@ -39,29 +39,6 @@ #include "pycore_import.h" /* Includes for frozen modules: */ -#include "frozen_modules/importlib._bootstrap.h" -#include "frozen_modules/importlib._bootstrap_external.h" -#include "frozen_modules/zipimport.h" -#include "frozen_modules/abc.h" -#include "frozen_modules/codecs.h" -#include "frozen_modules/io.h" -#include "frozen_modules/_collections_abc.h" -#include "frozen_modules/_sitebuiltins.h" -#include "frozen_modules/genericpath.h" -#include "frozen_modules/ntpath.h" -#include "frozen_modules/posixpath.h" -#include "frozen_modules/os.h" -#include "frozen_modules/site.h" -#include "frozen_modules/stat.h" -#include "frozen_modules/importlib.util.h" -#include "frozen_modules/importlib.machinery.h" -#include "frozen_modules/runpy.h" -#include "frozen_modules/__hello__.h" -#include "frozen_modules/__phello__.h" -#include "frozen_modules/__phello__.ham.h" -#include "frozen_modules/__phello__.ham.eggs.h" -#include "frozen_modules/__phello__.spam.h" -#include "frozen_modules/frozen_only.h" /* End includes */ #define GET_CODE(name) _Py_get_##name##_toplevel @@ -98,49 +75,47 @@ extern PyObject *_Py_get___phello___spam_toplevel(void); extern PyObject *_Py_get_frozen_only_toplevel(void); /* End extern declarations */ -/* Note that a negative size indicates a package. */ - static const struct _frozen bootstrap_modules[] = { - {"_frozen_importlib", _Py_M__importlib__bootstrap, (int)sizeof(_Py_M__importlib__bootstrap), GET_CODE(importlib__bootstrap)}, - {"_frozen_importlib_external", _Py_M__importlib__bootstrap_external, (int)sizeof(_Py_M__importlib__bootstrap_external), GET_CODE(importlib__bootstrap_external)}, - {"zipimport", _Py_M__zipimport, (int)sizeof(_Py_M__zipimport), GET_CODE(zipimport)}, + {"_frozen_importlib", NULL, 0, false, GET_CODE(importlib__bootstrap)}, + {"_frozen_importlib_external", NULL, 0, false, GET_CODE(importlib__bootstrap_external)}, + {"zipimport", NULL, 0, false, GET_CODE(zipimport)}, {0, 0, 0} /* bootstrap sentinel */ }; static const struct _frozen stdlib_modules[] = { /* stdlib - startup, without site (python -S) */ - {"abc", _Py_M__abc, (int)sizeof(_Py_M__abc), GET_CODE(abc)}, - {"codecs", _Py_M__codecs, (int)sizeof(_Py_M__codecs), GET_CODE(codecs)}, - {"io", _Py_M__io, (int)sizeof(_Py_M__io), GET_CODE(io)}, + {"abc", NULL, 0, false, GET_CODE(abc)}, + {"codecs", NULL, 0, false, GET_CODE(codecs)}, + {"io", NULL, 0, false, GET_CODE(io)}, /* stdlib - startup, with site */ - {"_collections_abc", _Py_M___collections_abc, (int)sizeof(_Py_M___collections_abc), GET_CODE(_collections_abc)}, - {"_sitebuiltins", _Py_M___sitebuiltins, (int)sizeof(_Py_M___sitebuiltins), GET_CODE(_sitebuiltins)}, - {"genericpath", _Py_M__genericpath, (int)sizeof(_Py_M__genericpath), GET_CODE(genericpath)}, - {"ntpath", _Py_M__ntpath, (int)sizeof(_Py_M__ntpath), GET_CODE(ntpath)}, - {"posixpath", _Py_M__posixpath, (int)sizeof(_Py_M__posixpath), GET_CODE(posixpath)}, - {"os.path", _Py_M__posixpath, (int)sizeof(_Py_M__posixpath), GET_CODE(posixpath)}, - {"os", _Py_M__os, (int)sizeof(_Py_M__os), GET_CODE(os)}, - {"site", _Py_M__site, (int)sizeof(_Py_M__site), GET_CODE(site)}, - {"stat", _Py_M__stat, (int)sizeof(_Py_M__stat), GET_CODE(stat)}, + {"_collections_abc", NULL, 0, false, GET_CODE(_collections_abc)}, + {"_sitebuiltins", NULL, 0, false, GET_CODE(_sitebuiltins)}, + {"genericpath", NULL, 0, false, GET_CODE(genericpath)}, + {"ntpath", NULL, 0, false, GET_CODE(ntpath)}, + {"posixpath", NULL, 0, false, GET_CODE(posixpath)}, + {"os.path", NULL, 0, false, GET_CODE(posixpath)}, + {"os", NULL, 0, false, GET_CODE(os)}, + {"site", NULL, 0, false, GET_CODE(site)}, + {"stat", NULL, 0, false, GET_CODE(stat)}, /* runpy - run module with -m */ - {"importlib.util", _Py_M__importlib_util, (int)sizeof(_Py_M__importlib_util), GET_CODE(importlib_util)}, - {"importlib.machinery", _Py_M__importlib_machinery, (int)sizeof(_Py_M__importlib_machinery), GET_CODE(importlib_machinery)}, - {"runpy", _Py_M__runpy, (int)sizeof(_Py_M__runpy), GET_CODE(runpy)}, + {"importlib.util", NULL, 0, false, GET_CODE(importlib_util)}, + {"importlib.machinery", NULL, 0, false, GET_CODE(importlib_machinery)}, + {"runpy", NULL, 0, false, GET_CODE(runpy)}, {0, 0, 0} /* stdlib sentinel */ }; static const struct _frozen test_modules[] = { - {"__hello__", _Py_M____hello__, (int)sizeof(_Py_M____hello__), GET_CODE(__hello__)}, - {"__hello_alias__", _Py_M____hello__, (int)sizeof(_Py_M____hello__), GET_CODE(__hello__)}, - {"__phello_alias__", _Py_M____hello__, -(int)sizeof(_Py_M____hello__), GET_CODE(__hello__)}, - {"__phello_alias__.spam", _Py_M____hello__, (int)sizeof(_Py_M____hello__), GET_CODE(__hello__)}, - {"__phello__", _Py_M____phello__, -(int)sizeof(_Py_M____phello__), GET_CODE(__phello__)}, - {"__phello__.__init__", _Py_M____phello__, (int)sizeof(_Py_M____phello__), GET_CODE(__phello__)}, - {"__phello__.ham", _Py_M____phello___ham, -(int)sizeof(_Py_M____phello___ham), GET_CODE(__phello___ham)}, - {"__phello__.ham.__init__", _Py_M____phello___ham, (int)sizeof(_Py_M____phello___ham), GET_CODE(__phello___ham)}, - {"__phello__.ham.eggs", _Py_M____phello___ham_eggs, (int)sizeof(_Py_M____phello___ham_eggs), GET_CODE(__phello___ham_eggs)}, - {"__phello__.spam", _Py_M____phello___spam, (int)sizeof(_Py_M____phello___spam), GET_CODE(__phello___spam)}, - {"__hello_only__", _Py_M__frozen_only, (int)sizeof(_Py_M__frozen_only), GET_CODE(frozen_only)}, + {"__hello__", NULL, 0, false, GET_CODE(__hello__)}, + {"__hello_alias__", NULL, 0, false, GET_CODE(__hello__)}, + {"__phello_alias__", NULL, 0, true, GET_CODE(__hello__)}, + {"__phello_alias__.spam", NULL, 0, false, GET_CODE(__hello__)}, + {"__phello__", NULL, 0, true, GET_CODE(__phello__)}, + {"__phello__.__init__", NULL, 0, false, GET_CODE(__phello__)}, + {"__phello__.ham", NULL, 0, true, GET_CODE(__phello___ham)}, + {"__phello__.ham.__init__", NULL, 0, false, GET_CODE(__phello___ham)}, + {"__phello__.ham.eggs", NULL, 0, false, GET_CODE(__phello___ham_eggs)}, + {"__phello__.spam", NULL, 0, false, GET_CODE(__phello___spam)}, + {"__hello_only__", NULL, 0, false, GET_CODE(frozen_only)}, {0, 0, 0} /* test sentinel */ }; const struct _frozen *_PyImport_FrozenBootstrap = bootstrap_modules; diff --git a/Python/import.c b/Python/import.c index 51b779ca17c52..be60c431f7ef8 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1297,13 +1297,21 @@ find_frozen(PyObject *nameobj, struct frozen_info *info) info->nameobj = nameobj; // borrowed info->data = (const char *)p->code; info->get_code = p->get_code; - info->size = p->size < 0 ? -(p->size) : p->size; - info->is_package = p->size < 0 ? true : false; + info->size = p->size; + info->is_package = p->is_package; + if (p->size < 0) { + // backward compatibility with negative size values + info->size = -(p->size); + info->is_package = true; + } info->origname = name; info->is_alias = resolve_module_alias(name, _PyImport_FrozenAliases, &info->origname); } - + if (p->code == NULL && p->size == 0 && p->get_code != NULL) { + /* It is only deepfrozen. */ + return FROZEN_OKAY; + } if (p->code == NULL) { /* It is frozen but marked as un-importable. */ return FROZEN_EXCLUDED; @@ -2224,7 +2232,7 @@ _imp_get_frozen_object_impl(PyObject *module, PyObject *name, if (info.nameobj == NULL) { info.nameobj = name; } - if (info.size == 0) { + if (info.size == 0 && info.get_code == NULL) { /* Does not contain executable code. */ set_frozen_error(FROZEN_INVALID, name); return NULL; diff --git a/Tools/freeze/makefreeze.py b/Tools/freeze/makefreeze.py index d7d05db88a96c..bc5f8567448bf 100644 --- a/Tools/freeze/makefreeze.py +++ b/Tools/freeze/makefreeze.py @@ -45,19 +45,19 @@ def makefreeze(base, dict, debug=0, entry_point=None, fail_import=()): print("freezing", mod, "...") str = marshal.dumps(m.__code__) size = len(str) + is_package = 'false' if m.__path__: - # Indicate package by negative size - size = -size - done.append((mod, mangled, size)) + is_package = 'true' + done.append((mod, mangled, size, is_package)) writecode(outfp, mangled, str) if debug: print("generating table of frozen modules") with bkfile.open(base + 'frozen.c', 'w') as outfp: - for mod, mangled, size in done: + for mod, mangled, size, _ in done: outfp.write('extern unsigned char M_%s[];\n' % mangled) outfp.write(header) - for mod, mangled, size in done: - outfp.write('\t{"%s", M_%s, %d},\n' % (mod, mangled, size)) + for mod, mangled, size, is_package in done: + outfp.write('\t{"%s", M_%s, %d, %s},\n' % (mod, mangled, size, is_package)) outfp.write('\n') # The following modules have a NULL code pointer, indicating # that the frozen program should not search for them on the host diff --git a/Tools/scripts/freeze_modules.py b/Tools/scripts/freeze_modules.py index 6d10758b5285c..03dcf939f978e 100644 --- a/Tools/scripts/freeze_modules.py +++ b/Tools/scripts/freeze_modules.py @@ -9,7 +9,7 @@ import ntpath import posixpath import sys - +import argparse from update_file import updating_file_with_tmpfile @@ -463,14 +463,15 @@ def replace_block(lines, start_marker, end_marker, replacements, file): return lines[:start_pos + 1] + replacements + lines[end_pos:] -def regen_frozen(modules): +def regen_frozen(modules, deepfreeze_only: bool): headerlines = [] parentdir = os.path.dirname(FROZEN_FILE) - for src in _iter_sources(modules): - # Adding a comment to separate sections here doesn't add much, - # so we don't. - header = relpath_for_posix_display(src.frozenfile, parentdir) - headerlines.append(f'#include "{header}"') + if not deepfreeze_only: + for src in _iter_sources(modules): + # Adding a comment to separate sections here doesn't add much, + # so we don't. + header = relpath_for_posix_display(src.frozenfile, parentdir) + headerlines.append(f'#include "{header}"') externlines = [] bootstraplines = [] @@ -500,9 +501,13 @@ def regen_frozen(modules): externlines.append("extern PyObject *%s(void);" % get_code_name) symbol = mod.symbol - pkg = '-' if mod.ispkg else '' - line = ('{"%s", %s, %s(int)sizeof(%s), GET_CODE(%s)},' - ) % (mod.name, symbol, pkg, symbol, code_name) + pkg = 'true' if mod.ispkg else 'false' + if deepfreeze_only: + line = ('{"%s", NULL, 0, %s, GET_CODE(%s)},' + ) % (mod.name, pkg, code_name) + else: + line = ('{"%s", %s, (int)sizeof(%s), %s, GET_CODE(%s)},' + ) % (mod.name, symbol, symbol, pkg, code_name) lines.append(line) if mod.isalias: @@ -710,18 +715,19 @@ def regen_pcbuild(modules): ####################################### # the script -def main(): +def main(deepfreeze_only: bool): # Expand the raw specs, preserving order. modules = list(parse_frozen_specs()) # Regen build-related files. regen_makefile(modules) regen_pcbuild(modules) - regen_frozen(modules) + regen_frozen(modules, deepfreeze_only) if __name__ == '__main__': - argv = sys.argv[1:] - if argv: - sys.exit(f'ERROR: got unexpected args {argv}') - main() + parser = argparse.ArgumentParser() + parser.add_argument("--deepfreeze-only", action="store_true", + help="Only use deepfrozen modules", default=True) + args = parser.parse_args() + main(args.deepfreeze_only) From webhook-mailer at python.org Fri Feb 4 14:59:48 2022 From: webhook-mailer at python.org (terryjreedy) Date: Fri, 04 Feb 2022 19:59:48 -0000 Subject: [Python-checkins] [3.10] bpo-46609: Update asyncio-task coroutine doc (GH-31132) Message-ID: https://github.com/python/cpython/commit/5603db43ba7ba5568b7516d0e28730a2bc1e1f26 commit: 5603db43ba7ba5568b7516d0e28730a2bc1e1f26 branch: 3.10 author: Terry Jan Reedy committer: terryjreedy date: 2022-02-04T14:59:23-05:00 summary: [3.10] bpo-46609: Update asyncio-task coroutine doc (GH-31132) @coroutine in removed in 3.11, not 3.10. files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 1175b0537c051..7f4cc70386157 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -1057,7 +1057,7 @@ Generator-based Coroutines .. note:: Support for generator-based coroutines is **deprecated** and - is scheduled for removal in Python 3.10. + is removed in Python 3.11. Generator-based coroutines predate async/await syntax. They are Python generators that use ``yield from`` expressions to await From webhook-mailer at python.org Fri Feb 4 15:46:17 2022 From: webhook-mailer at python.org (terryjreedy) Date: Fri, 04 Feb 2022 20:46:17 -0000 Subject: [Python-checkins] [3.9] bpo-46609: Update asyncio-task coroutine doc (GH-31132) Message-ID: https://github.com/python/cpython/commit/459e26f0987a12a19238baba422e13a8f7fcfca3 commit: 459e26f0987a12a19238baba422e13a8f7fcfca3 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2022-02-04T15:46:09-05:00 summary: [3.9] bpo-46609: Update asyncio-task coroutine doc (GH-31132) @coroutine in removed in 3.11, not 3.10. (cherry picked from commit 5603db43ba7ba5568b7516d0e28730a2bc1e1f26) Co-authored-by: Terry Jan Reedy files: M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index e8bee20bc76bc..38b12e44b6de2 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -991,7 +991,7 @@ Generator-based Coroutines .. note:: Support for generator-based coroutines is **deprecated** and - is scheduled for removal in Python 3.10. + is removed in Python 3.11. Generator-based coroutines predate async/await syntax. They are Python generators that use ``yield from`` expressions to await @@ -1019,7 +1019,7 @@ enforced. This decorator should not be used for :keyword:`async def` coroutines. - .. deprecated-removed:: 3.8 3.10 + .. deprecated-removed:: 3.8 3.11 Use :keyword:`async def` instead. From webhook-mailer at python.org Fri Feb 4 22:54:34 2022 From: webhook-mailer at python.org (ethanfurman) Date: Sat, 05 Feb 2022 03:54:34 -0000 Subject: [Python-checkins] bpo-31369: include ``RegexFlag`` in ``re.__all__`` (GH-30279) Message-ID: https://github.com/python/cpython/commit/fea7290a0ecee09bbce571d4d10f5881b7ea3485 commit: fea7290a0ecee09bbce571d4d10f5881b7ea3485 branch: main author: andrei kulakov committer: ethanfurman date: 2022-02-04T19:54:28-08:00 summary: bpo-31369: include ``RegexFlag`` in ``re.__all__`` (GH-30279) * added RegexFlag to re.__all__; added RegexFlag.NOFLAG Co-authored-by: Jelle Zijlstra files: A Misc/NEWS.d/next/Library/2021-12-27-18-28-44.bpo-31369.b9yM94.rst M Doc/library/re.rst M Lib/re.py diff --git a/Doc/library/re.rst b/Doc/library/re.rst index b12ce4b9744f9..8d62e3bf4d8d8 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -637,6 +637,11 @@ form. programs that use only a few regular expressions at a time needn't worry about compiling regular expressions. +.. class:: RegexFlag + + An :class:`enum.IntFlag` class containing the regex options listed below. + + .. versionadded:: 3.11 - added to ``__all__`` .. data:: A ASCII @@ -710,6 +715,17 @@ form. string and immediately before the newline (if any) at the end of the string. Corresponds to the inline flag ``(?m)``. +.. data:: NOFLAG + + Indicates no flag being applied, the value is ``0``. This flag may be used + as a default value for a function keyword argument or as a base value that + will be conditionally ORed with other flags. Example of use as a default + value:: + + def myfunc(text, flag=re.NOFLAG): + return re.match(text, flag) + + .. versionadded:: 3.11 .. data:: S DOTALL diff --git a/Lib/re.py b/Lib/re.py index a7ab9b3706748..e9a745dc581a6 100644 --- a/Lib/re.py +++ b/Lib/re.py @@ -137,7 +137,7 @@ "findall", "finditer", "compile", "purge", "template", "escape", "error", "Pattern", "Match", "A", "I", "L", "M", "S", "X", "U", "ASCII", "IGNORECASE", "LOCALE", "MULTILINE", "DOTALL", "VERBOSE", - "UNICODE", + "UNICODE", "NOFLAG", "RegexFlag", ] __version__ = "2.2.1" @@ -145,6 +145,7 @@ @enum.global_enum @enum._simple_enum(enum.IntFlag, boundary=enum.KEEP) class RegexFlag: + NOFLAG = 0 ASCII = A = sre_compile.SRE_FLAG_ASCII # assume ascii "locale" IGNORECASE = I = sre_compile.SRE_FLAG_IGNORECASE # ignore case LOCALE = L = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale diff --git a/Misc/NEWS.d/next/Library/2021-12-27-18-28-44.bpo-31369.b9yM94.rst b/Misc/NEWS.d/next/Library/2021-12-27-18-28-44.bpo-31369.b9yM94.rst new file mode 100644 index 0000000000000..2bb9e62de1f40 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-12-27-18-28-44.bpo-31369.b9yM94.rst @@ -0,0 +1,2 @@ +Add :class:`~re.RegexFlag` to ``re.__all__`` and documented it. Add +:data:`~re.RegexFlag.NOFLAG` to indicate no flags being set. From webhook-mailer at python.org Sat Feb 5 10:50:11 2022 From: webhook-mailer at python.org (gvanrossum) Date: Sat, 05 Feb 2022 15:50:11 -0000 Subject: [Python-checkins] Fix __init_subclass__ using self instead of class (#31135) Message-ID: https://github.com/python/cpython/commit/2f077b6991f59c51989b65618317297c1eb0fb95 commit: 2f077b6991f59c51989b65618317297c1eb0fb95 branch: main author: Gregory Beauregard committer: gvanrossum date: 2022-02-05T07:50:00-08:00 summary: Fix __init_subclass__ using self instead of class (#31135) files: M Lib/typing.py diff --git a/Lib/typing.py b/Lib/typing.py index 0cf9755022e9b..e4e32b5b320d0 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -349,7 +349,7 @@ class _Final: __slots__ = ('__weakref__',) - def __init_subclass__(self, /, *args, **kwds): + def __init_subclass__(cls, /, *args, **kwds): if '_root' not in kwds: raise TypeError("Cannot subclass special typing classes") From webhook-mailer at python.org Sat Feb 5 10:59:58 2022 From: webhook-mailer at python.org (gvanrossum) Date: Sat, 05 Feb 2022 15:59:58 -0000 Subject: [Python-checkins] bpo-46608: Fix argument parsing in freeze_modules.py (GH-31131) Message-ID: https://github.com/python/cpython/commit/9d4161a60ca8b470148ffd6c73e3110a0aa6d66f commit: 9d4161a60ca8b470148ffd6c73e3110a0aa6d66f branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: gvanrossum date: 2022-02-05T07:59:48-08:00 summary: bpo-46608: Fix argument parsing in freeze_modules.py (GH-31131) files: M Tools/scripts/freeze_modules.py diff --git a/Tools/scripts/freeze_modules.py b/Tools/scripts/freeze_modules.py index 03dcf939f978e..0dc61e2fe32b2 100644 --- a/Tools/scripts/freeze_modules.py +++ b/Tools/scripts/freeze_modules.py @@ -463,10 +463,10 @@ def replace_block(lines, start_marker, end_marker, replacements, file): return lines[:start_pos + 1] + replacements + lines[end_pos:] -def regen_frozen(modules, deepfreeze_only: bool): +def regen_frozen(modules, frozen_modules: bool): headerlines = [] parentdir = os.path.dirname(FROZEN_FILE) - if not deepfreeze_only: + if frozen_modules: for src in _iter_sources(modules): # Adding a comment to separate sections here doesn't add much, # so we don't. @@ -502,7 +502,7 @@ def regen_frozen(modules, deepfreeze_only: bool): symbol = mod.symbol pkg = 'true' if mod.ispkg else 'false' - if deepfreeze_only: + if not frozen_modules: line = ('{"%s", NULL, 0, %s, GET_CODE(%s)},' ) % (mod.name, pkg, code_name) else: @@ -715,19 +715,21 @@ def regen_pcbuild(modules): ####################################### # the script -def main(deepfreeze_only: bool): +parser = argparse.ArgumentParser() +parser.add_argument("--frozen-modules", action="store_true", + help="Use both frozen and deepfrozen modules. (default: uses only deepfrozen modules)") + +def main(): + args = parser.parse_args() + frozen_modules: bool = args.frozen_modules # Expand the raw specs, preserving order. modules = list(parse_frozen_specs()) # Regen build-related files. regen_makefile(modules) regen_pcbuild(modules) - regen_frozen(modules, deepfreeze_only) + regen_frozen(modules, frozen_modules) if __name__ == '__main__': - parser = argparse.ArgumentParser() - parser.add_argument("--deepfreeze-only", action="store_true", - help="Only use deepfrozen modules", default=True) - args = parser.parse_args() - main(args.deepfreeze_only) + main() From webhook-mailer at python.org Sat Feb 5 14:52:29 2022 From: webhook-mailer at python.org (tiran) Date: Sat, 05 Feb 2022 19:52:29 -0000 Subject: [Python-checkins] bpo-40280: Address more test failures on Emscripten (GH-31050) Message-ID: https://github.com/python/cpython/commit/96b344c2f15cb09251018f57f19643fe20637392 commit: 96b344c2f15cb09251018f57f19643fe20637392 branch: main author: Christian Heimes committer: tiran date: 2022-02-05T20:52:01+01:00 summary: bpo-40280: Address more test failures on Emscripten (GH-31050) Co-authored-by: Brett Cannon files: A Misc/NEWS.d/next/Build/2022-01-31-15-15-08.bpo-40280.r1AYNW.rst M Lib/test/support/__init__.py M Lib/test/support/os_helper.py M Lib/test/test_builtin.py M Lib/test/test_capi.py M Lib/test/test_faulthandler.py M Lib/test/test_fileio.py M Lib/test/test_genericalias.py M Lib/test/test_getpass.py M Lib/test/test_inspect.py M Lib/test/test_interpreters.py M Lib/test/test_io.py M Lib/test/test_os.py M Lib/test/test_posix.py M Lib/test/test_pwd.py M Lib/test/test_pyexpat.py M Lib/test/test_resource.py M Lib/test/test_zipfile.py M Modules/clinic/resource.c.h M Modules/posixmodule.c M Modules/resource.c M Modules/timemodule.c M Tools/wasm/README.md M Tools/wasm/config.site-wasm32-emscripten M configure M configure.ac M pyconfig.h.in diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index e5eb66e9068e7..9d0d3d9c9b619 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1278,6 +1278,8 @@ def reap_children(): # Need os.waitpid(-1, os.WNOHANG): Windows is not supported if not (hasattr(os, 'waitpid') and hasattr(os, 'WNOHANG')): return + elif not has_subprocess_support: + return # Reap all our dead child processes so we don't leave zombies around. # These hog resources and might be causing some of the buildbots to die. diff --git a/Lib/test/support/os_helper.py b/Lib/test/support/os_helper.py index 50aa7a7176c0a..c761d7bd3d2b6 100644 --- a/Lib/test/support/os_helper.py +++ b/Lib/test/support/os_helper.py @@ -502,7 +502,7 @@ def __fspath__(self): def fd_count(): """Count the number of open file descriptors. """ - if sys.platform.startswith(('linux', 'freebsd')): + if sys.platform.startswith(('linux', 'freebsd', 'emscripten')): try: names = os.listdir("/proc/self/fd") # Subtract one because listdir() internally opens a file diff --git a/Lib/test/test_builtin.py b/Lib/test/test_builtin.py index c6e67cc2910cf..a601a524d6eb7 100644 --- a/Lib/test/test_builtin.py +++ b/Lib/test/test_builtin.py @@ -393,6 +393,7 @@ def test_compile_top_level_await_no_coro(self): msg=f"source={source} mode={mode}") + @unittest.skipIf(support.is_emscripten, "socket.accept is broken") def test_compile_top_level_await(self): """Test whether code some top level await can be compiled. @@ -1213,6 +1214,7 @@ def test_open_default_encoding(self): os.environ.clear() os.environ.update(old_environ) + @support.requires_subprocess() def test_open_non_inheritable(self): fileobj = open(__file__, encoding="utf-8") with fileobj: diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py index ccf8ceda49831..089088d97a66e 100644 --- a/Lib/test/test_capi.py +++ b/Lib/test/test_capi.py @@ -611,6 +611,7 @@ def check_fatal_error(self, code, expected, not_expected=()): self.assertNotIn(name, modules) self.assertEqual(len(modules), total) + @support.requires_subprocess() def test_fatal_error(self): # By default, stdlib extension modules are ignored, # but not test modules. @@ -880,6 +881,7 @@ class Test_testinternalcapi(unittest.TestCase): if name.startswith('test_')) + at support.requires_subprocess() class PyMemDebugTests(unittest.TestCase): PYTHONMALLOC = 'debug' # '0x04c06e0' or '04C06E0' diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py index daacdeef5bc80..8d106daaf6520 100644 --- a/Lib/test/test_faulthandler.py +++ b/Lib/test/test_faulthandler.py @@ -19,6 +19,9 @@ except ImportError: _testcapi = None +if not support.has_subprocess_support: + raise unittest.SkipTest("test module requires subprocess") + TIMEOUT = 0.5 MS_WINDOWS = (os.name == 'nt') diff --git a/Lib/test/test_fileio.py b/Lib/test/test_fileio.py index 4269b0e53f56d..e4984d3cd559e 100644 --- a/Lib/test/test_fileio.py +++ b/Lib/test/test_fileio.py @@ -9,7 +9,7 @@ from weakref import proxy from functools import wraps -from test.support import cpython_only, swap_attr, gc_collect +from test.support import cpython_only, swap_attr, gc_collect, is_emscripten from test.support.os_helper import (TESTFN, TESTFN_UNICODE, make_bad_fd) from test.support.warnings_helper import check_warnings from collections import UserList @@ -373,7 +373,7 @@ def testAbles(self): self.assertEqual(f.isatty(), False) f.close() - if sys.platform != "win32": + if sys.platform != "win32" and not is_emscripten: try: f = self.FileIO("/dev/tty", "a") except OSError: diff --git a/Lib/test/test_genericalias.py b/Lib/test/test_genericalias.py index 706cc5ea1af2f..d311281c578a2 100644 --- a/Lib/test/test_genericalias.py +++ b/Lib/test/test_genericalias.py @@ -24,14 +24,20 @@ from fileinput import FileInput from itertools import chain from http.cookies import Morsel -from multiprocessing.managers import ValueProxy -from multiprocessing.pool import ApplyResult +try: + from multiprocessing.managers import ValueProxy + from multiprocessing.pool import ApplyResult + from multiprocessing.queues import SimpleQueue as MPSimpleQueue +except ImportError: + # _multiprocessing module is optional + ValueProxy = None + ApplyResult = None + MPSimpleQueue = None try: from multiprocessing.shared_memory import ShareableList except ImportError: # multiprocessing.shared_memory is not available on e.g. Android ShareableList = None -from multiprocessing.queues import SimpleQueue as MPSimpleQueue from os import DirEntry from re import Pattern, Match from types import GenericAlias, MappingProxyType, AsyncGeneratorType @@ -79,13 +85,14 @@ class BaseTest(unittest.TestCase): Queue, SimpleQueue, _AssertRaisesContext, SplitResult, ParseResult, - ValueProxy, ApplyResult, WeakSet, ReferenceType, ref, - ShareableList, MPSimpleQueue, + ShareableList, Future, _WorkItem, Morsel] if ctypes is not None: generic_types.extend((ctypes.Array, ctypes.LibraryLoader)) + if ValueProxy is not None: + generic_types.extend((ValueProxy, ApplyResult, MPSimpleQueue)) def test_subscriptable(self): for t in self.generic_types: diff --git a/Lib/test/test_getpass.py b/Lib/test/test_getpass.py index 3452e46213a76..98ecec94336e3 100644 --- a/Lib/test/test_getpass.py +++ b/Lib/test/test_getpass.py @@ -28,6 +28,9 @@ def test_username_priorities_of_env_values(self, environ): getpass.getuser() except ImportError: # in case there's no pwd module pass + except KeyError: + # current user has no pwd entry + pass self.assertEqual( environ.get.call_args_list, [mock.call(x) for x in ('LOGNAME', 'USER', 'LNAME', 'USERNAME')]) diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py index a553431bdccfb..29589a726768f 100644 --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -788,6 +788,7 @@ def test_nested_class_definition_inside_function(self): self.assertSourceEqual(mod2.cls213, 218, 222) self.assertSourceEqual(mod2.cls213().func219(), 220, 221) + @unittest.skipIf(support.is_emscripten, "socket.accept is broken") def test_nested_class_definition_inside_async_function(self): import asyncio self.addCleanup(asyncio.set_event_loop_policy, None) diff --git a/Lib/test/test_interpreters.py b/Lib/test/test_interpreters.py index 6266aa7c33b32..48c7119fff1cf 100644 --- a/Lib/test/test_interpreters.py +++ b/Lib/test/test_interpreters.py @@ -5,7 +5,8 @@ import unittest import time -import _xxsubinterpreters as _interpreters +from test.support import import_helper +_interpreters = import_helper.import_module('_xxsubinterpreters') from test.support import interpreters diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py index a10611abb13f4..e9abd153a3e8c 100644 --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -76,6 +76,10 @@ def _default_chunk_size(): with open(__file__, "r", encoding="latin-1") as f: return f._CHUNK_SIZE +requires_alarm = unittest.skipUnless( + hasattr(signal, "alarm"), "test requires signal.alarm()" +) + class MockRawIOWithoutRead: """A RawIO implementation without read(), so as to exercise the default @@ -4435,12 +4439,15 @@ def _read(): if e.errno != errno.EBADF: raise + @requires_alarm def test_interrupted_write_unbuffered(self): self.check_interrupted_write(b"xy", b"xy", mode="wb", buffering=0) + @requires_alarm def test_interrupted_write_buffered(self): self.check_interrupted_write(b"xy", b"xy", mode="wb") + @requires_alarm def test_interrupted_write_text(self): self.check_interrupted_write("xy", b"xy", mode="w", encoding="ascii") @@ -4472,9 +4479,11 @@ def on_alarm(*args): wio.close() os.close(r) + @requires_alarm def test_reentrant_write_buffered(self): self.check_reentrant_write(b"xy", mode="wb") + @requires_alarm def test_reentrant_write_text(self): self.check_reentrant_write("xy", mode="w", encoding="ascii") @@ -4502,10 +4511,12 @@ def alarm_handler(sig, frame): os.close(w) os.close(r) + @requires_alarm def test_interrupted_read_retry_buffered(self): self.check_interrupted_read_retry(lambda x: x.decode('latin1'), mode="rb") + @requires_alarm def test_interrupted_read_retry_text(self): self.check_interrupted_read_retry(lambda x: x, mode="r", encoding="latin1") @@ -4578,9 +4589,11 @@ def alarm2(sig, frame): if e.errno != errno.EBADF: raise + @requires_alarm def test_interrupted_write_retry_buffered(self): self.check_interrupted_write_retry(b"x", mode="wb") + @requires_alarm def test_interrupted_write_retry_text(self): self.check_interrupted_write_retry("x", mode="w", encoding="latin1") diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index 84c27f346c340..660691579c163 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -992,6 +992,7 @@ def _empty_mapping(self): @unittest.skipUnless(unix_shell and os.path.exists(unix_shell), 'requires a shell') @unittest.skipUnless(hasattr(os, 'popen'), "needs os.popen()") + @support.requires_subprocess() def test_update2(self): os.environ.clear() os.environ.update(HELLO="World") @@ -1002,6 +1003,7 @@ def test_update2(self): @unittest.skipUnless(unix_shell and os.path.exists(unix_shell), 'requires a shell') @unittest.skipUnless(hasattr(os, 'popen'), "needs os.popen()") + @support.requires_subprocess() def test_os_popen_iter(self): with os.popen("%s -c 'echo \"line1\nline2\nline3\"'" % unix_shell) as popen: @@ -1173,6 +1175,8 @@ def test_iter_error_when_changing_os_environ_values(self): def _test_underlying_process_env(self, var, expected): if not (unix_shell and os.path.exists(unix_shell)): return + elif not support.has_subprocess_support: + return with os.popen(f"{unix_shell} -c 'echo ${var}'") as popen: value = popen.read().strip() diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py index 974edd766cc80..5cc04fd46dddb 100644 --- a/Lib/test/test_posix.py +++ b/Lib/test/test_posix.py @@ -184,7 +184,7 @@ def test_truncate(self): posix.truncate(os_helper.TESTFN, 0) @unittest.skipUnless(getattr(os, 'execve', None) in os.supports_fd, "test needs execve() to support the fd parameter") - @unittest.skipUnless(hasattr(os, 'fork'), "test needs os.fork()") + @support.requires_fork() def test_fexecve(self): fp = os.open(sys.executable, os.O_RDONLY) try: @@ -199,7 +199,7 @@ def test_fexecve(self): @unittest.skipUnless(hasattr(posix, 'waitid'), "test needs posix.waitid()") - @unittest.skipUnless(hasattr(os, 'fork'), "test needs os.fork()") + @support.requires_fork() def test_waitid(self): pid = os.fork() if pid == 0: @@ -209,7 +209,7 @@ def test_waitid(self): res = posix.waitid(posix.P_PID, pid, posix.WEXITED) self.assertEqual(pid, res.si_pid) - @unittest.skipUnless(hasattr(os, 'fork'), "test needs os.fork()") + @support.requires_fork() def test_register_at_fork(self): with self.assertRaises(TypeError, msg="Positional args not allowed"): os.register_at_fork(lambda: None) @@ -1056,6 +1056,7 @@ def test_getgrouplist(self): @unittest.skipUnless(hasattr(os, 'getegid'), "test needs os.getegid()") @unittest.skipUnless(hasattr(os, 'popen'), "test needs os.popen()") + @support.requires_subprocess() def test_getgroups(self): with os.popen('id -G 2>/dev/null') as idg: groups = idg.read().strip() @@ -1481,7 +1482,7 @@ def test_unlink_dir_fd(self): self.addCleanup(posix.unlink, fullname) raise - @unittest.skipUnless(os.mkfifo in os.supports_dir_fd, "test needs dir_fd support in os.mkfifo()") + @unittest.skipUnless(hasattr(os, 'mkfifo') and os.mkfifo in os.supports_dir_fd, "test needs dir_fd support in os.mkfifo()") def test_mkfifo_dir_fd(self): with self.prepare() as (dir_fd, name, fullname): try: diff --git a/Lib/test/test_pwd.py b/Lib/test/test_pwd.py index f8f12571ca90e..c789326425be3 100644 --- a/Lib/test/test_pwd.py +++ b/Lib/test/test_pwd.py @@ -69,7 +69,7 @@ def test_errors(self): allnames = list(bynames.keys()) namei = 0 - fakename = allnames[namei] + fakename = allnames[namei] if allnames else "invaliduser" while fakename in bynames: chars = list(fakename) for i in range(len(chars)): diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py index b2b4dea060532..6e578458a2509 100644 --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -12,7 +12,7 @@ from xml.parsers import expat from xml.parsers.expat import errors -from test.support import sortdict +from test.support import sortdict, is_emscripten class SetAttributeTest(unittest.TestCase): @@ -466,7 +466,10 @@ def test_exception(self): "pyexpat.c", "StartElement") self.check_traceback_entry(entries[2], "test_pyexpat.py", "StartElementHandler") - if sysconfig.is_python_build() and not (sys.platform == 'win32' and platform.machine() == 'ARM'): + if (sysconfig.is_python_build() + and not (sys.platform == 'win32' and platform.machine() == 'ARM') + and not is_emscripten + ): self.assertIn('call_with_frame("StartElement"', entries[1][3]) diff --git a/Lib/test/test_resource.py b/Lib/test/test_resource.py index f2642c6ba181d..317e7ca8f8c85 100644 --- a/Lib/test/test_resource.py +++ b/Lib/test/test_resource.py @@ -98,6 +98,7 @@ def test_fsize_toobig(self): except (OverflowError, ValueError): pass + @unittest.skipUnless(hasattr(resource, "getrusage"), "needs getrusage") def test_getrusage(self): self.assertRaises(TypeError, resource.getrusage) self.assertRaises(TypeError, resource.getrusage, 42, 42) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index e226dd741d7a7..de2dd33f43660 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -22,7 +22,9 @@ from test.support import script_helper from test.support import (findfile, requires_zlib, requires_bz2, requires_lzma, captured_stdout, requires_subprocess) -from test.support.os_helper import TESTFN, unlink, rmtree, temp_dir, temp_cwd +from test.support.os_helper import ( + TESTFN, unlink, rmtree, temp_dir, temp_cwd, fd_count +) TESTFN2 = TESTFN + "2" @@ -2539,14 +2541,14 @@ def test_write_after_read(self): def test_many_opens(self): # Verify that read() and open() promptly close the file descriptor, # and don't rely on the garbage collector to free resources. + startcount = fd_count() self.make_test_archive(TESTFN2) with zipfile.ZipFile(TESTFN2, mode="r") as zipf: for x in range(100): zipf.read('ones') with zipf.open('ones') as zopen1: pass - with open(os.devnull, "rb") as f: - self.assertLess(f.fileno(), 100) + self.assertEqual(startcount, fd_count()) def test_write_while_reading(self): with zipfile.ZipFile(TESTFN2, 'w', zipfile.ZIP_DEFLATED) as zipf: diff --git a/Misc/NEWS.d/next/Build/2022-01-31-15-15-08.bpo-40280.r1AYNW.rst b/Misc/NEWS.d/next/Build/2022-01-31-15-15-08.bpo-40280.r1AYNW.rst new file mode 100644 index 0000000000000..bb4878c6b0ac2 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2022-01-31-15-15-08.bpo-40280.r1AYNW.rst @@ -0,0 +1,9 @@ +Fix wasm32-emscripten test failures and platform issues. +- Disable syscalls that are not supported or don't work, e.g. + wait, getrusage, prlimit, mkfifo, mknod, setres[gu]id, setgroups. +- Use fd_count to cound open fds. +- Add more checks for subprocess and fork. +- Add workarounds for missing _multiprocessing and failing socket.accept(). +- Enable bzip2. +- Disable large file support. +- Disable signal.alarm. diff --git a/Modules/clinic/resource.c.h b/Modules/clinic/resource.c.h index 32c092ad7a94a..f31f7e8265e79 100644 --- a/Modules/clinic/resource.c.h +++ b/Modules/clinic/resource.c.h @@ -2,6 +2,8 @@ preserve [clinic start generated code]*/ +#if defined(HAVE_GETRUSAGE) + PyDoc_STRVAR(resource_getrusage__doc__, "getrusage($module, who, /)\n" "--\n" @@ -29,6 +31,8 @@ resource_getrusage(PyObject *module, PyObject *arg) return return_value; } +#endif /* defined(HAVE_GETRUSAGE) */ + PyDoc_STRVAR(resource_getrlimit__doc__, "getrlimit($module, resource, /)\n" "--\n" @@ -160,7 +164,11 @@ resource_getpagesize(PyObject *module, PyObject *Py_UNUSED(ignored)) return return_value; } +#ifndef RESOURCE_GETRUSAGE_METHODDEF + #define RESOURCE_GETRUSAGE_METHODDEF +#endif /* !defined(RESOURCE_GETRUSAGE_METHODDEF) */ + #ifndef RESOURCE_PRLIMIT_METHODDEF #define RESOURCE_PRLIMIT_METHODDEF #endif /* !defined(RESOURCE_PRLIMIT_METHODDEF) */ -/*[clinic end generated code: output=ad190fb33d647d1e input=a9049054013a1b77]*/ +/*[clinic end generated code: output=9ce1886c129eb2f3 input=a9049054013a1b77]*/ diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 7b5c3ef575565..ea0435d7d52e6 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -881,7 +881,7 @@ _Py_Gid_Converter(PyObject *obj, gid_t *p) #define _PyLong_FromDev PyLong_FromLongLong -#if defined(HAVE_MKNOD) && defined(HAVE_MAKEDEV) +#if (defined(HAVE_MKNOD) && defined(HAVE_MAKEDEV)) || defined(HAVE_DEVICE_MACROS) static int _Py_Dev_Converter(PyObject *obj, void *p) { @@ -890,7 +890,7 @@ _Py_Dev_Converter(PyObject *obj, void *p) return 0; return 1; } -#endif /* HAVE_MKNOD && HAVE_MAKEDEV */ +#endif /* (HAVE_MKNOD && HAVE_MAKEDEV) || HAVE_DEVICE_MACROS */ #ifdef AT_FDCWD diff --git a/Modules/resource.c b/Modules/resource.c index 0d69c2983b4d1..d8bba2e39847a 100644 --- a/Modules/resource.c +++ b/Modules/resource.c @@ -78,6 +78,7 @@ get_resource_state(PyObject *module) static struct PyModuleDef resourcemodule; +#ifdef HAVE_GETRUSAGE /*[clinic input] resource.getrusage @@ -134,6 +135,7 @@ resource_getrusage_impl(PyObject *module, int who) return result; } +#endif static int py2rlimit(PyObject *limits, struct rlimit *rl_out) diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 35b8e14e82711..4b7500aabded0 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1282,7 +1282,7 @@ _PyTime_GetProcessTimeWithInfo(_PyTime_t *tp, _Py_clock_info_t *info) #endif /* getrusage(RUSAGE_SELF) */ -#if defined(HAVE_SYS_RESOURCE_H) +#if defined(HAVE_SYS_RESOURCE_H) && defined(HAVE_GETRUSAGE) struct rusage ru; if (getrusage(RUSAGE_SELF, &ru) == 0) { diff --git a/Tools/wasm/README.md b/Tools/wasm/README.md index f59b876b11a74..1cdaa4efffb89 100644 --- a/Tools/wasm/README.md +++ b/Tools/wasm/README.md @@ -1,7 +1,11 @@ # Python WebAssembly (WASM) build +**WARNING: WASM support is highly experimental! Lots of features are not working yet.** + This directory contains configuration and helpers to facilitate cross -compilation of CPython to WebAssembly (WASM). +compilation of CPython to WebAssembly (WASM). For now we support +*wasm32-emscripten* builds for modern browser and for *Node.js*. It's not +possible to build for *wasm32-wasi* out-of-the-box yet. ## wasm32-emscripten build @@ -22,16 +26,14 @@ popd ### Fetch and build additional emscripten ports ```shell -embuilder build zlib +embuilder build zlib bzip2 ``` -### Cross compile to wasm32-emscripten - -For browser: +### Cross compile to wasm32-emscripten for browser ```shell -mkdir -p builddir/emscripten -pushd builddir/emscripten +mkdir -p builddir/emscripten-browser +pushd builddir/emscripten-browser CONFIG_SITE=../../Tools/wasm/config.site-wasm32-emscripten \ emconfigure ../../configure -C \ @@ -41,11 +43,27 @@ CONFIG_SITE=../../Tools/wasm/config.site-wasm32-emscripten \ --with-build-python=$(pwd)/../build/python emmake make -j$(nproc) +popd +``` + +Serve `python.html` with a local webserver and open the file in a browser. + +```shell +emrun builddir/emscripten-browser/python.html +``` + +or + +```shell +python3 -m http.server ``` -For node: +### Cross compile to wasm32-emscripten for node ``` +mkdir -p builddir/emscripten-node +pushd builddir/emscripten-node + CONFIG_SITE=../../Tools/wasm/config.site-wasm32-emscripten \ emconfigure ../../configure -C \ --host=wasm32-unknown-emscripten \ @@ -54,18 +72,70 @@ CONFIG_SITE=../../Tools/wasm/config.site-wasm32-emscripten \ --with-build-python=$(pwd)/../build/python emmake make -j$(nproc) +popd ``` -### Test in browser - -Serve `python.html` with a local webserver and open the file in a browser. - -```shell -emrun python.html ``` - -or - -```shell -python3 -m http.server +node --experimental-wasm-threads --experimental-wasm-bulk-memory builddir/emscripten-node/python.js ``` + +## wasm32-emscripten limitations and issues + +- Most stdlib modules with a dependency on external libraries are missing: + ``ctypes``, ``readline``, ``sqlite3``, ``ssl``, and more. +- Shared extension modules are not implemented yet. All extension modules + are statically linked into the main binary. +- Processes are not supported. System calls like fork, popen, and subprocess + fail with ``ENOSYS`` or ``ENOSUP``. +- Blocking sockets are not available and non-blocking sockets don't work + correctly, e.g. ``socket.accept`` crashes the runtime. ``gethostbyname`` + does not resolve to a real IP address. IPv6 is not available. +- The ``select`` module is limited. ``select.select()`` crashes the runtime + due to lack of exectfd support. +- The ``*at`` variants of functions (e.g. ``openat``) are not available. + The ``dir_fd`` argument of *os* module functions can't be used. +- Signal support is limited. ``signal.alarm``, ``itimer``, ``sigaction`` + are not available or do not work correctly. ``SIGTERM`` exits the runtime. +- Most user, group, and permission related function and modules are not + supported or don't work as expected, e.g.``pwd`` module, ``grp`` module, + ``os.setgroups``, ``os.chown``, and so on. +- Offset and iovec I/O functions (e.g. ``os.pread``, ``os.preadv``) are not + available. +- ``os.mknod`` and ``os.mkfifo`` + [don't work](https://github.com/emscripten-core/emscripten/issues/16158) + and are disabled. +- Large file support crashes the runtime and is disabled. +- ``mmap`` module is unstable. flush (``msync``) can crash the runtime. +- Resource-related functions like ``os.nice`` and most functions of the + ``resource`` module are not available. +- Some time and datetime features are broken. ``strftime`` and ``strptime`` + have known bugs, e.g. + [%% quoting](https://github.com/emscripten-core/emscripten/issues/16155), + [%U off-by-one](https://github.com/emscripten-core/emscripten/issues/16156). + Extended glibc formatting features are not available. +- ``locales`` module is affected by musl libc issues, + [bpo-46390](https://bugs.python.org/issue46390). +- ``uuid`` module is affected by + [memory leak](https://github.com/emscripten-core/emscripten/issues/16081) + and crasher in Emscripten's ``freeaddrinfo``, +- Recursive ``glob`` leaks file descriptors. +- Python's object allocator ``obmalloc`` is disabled by default. +- ``ensurepip`` is not available. + +### wasm32-emscripten in browsers + +- The bundled stdlib is limited. Network-related modules, + distutils, multiprocessing, dbm, tests and similar modules + are not shipped. All other modules are bundled as pre-compiled + ``pyc`` files. +- Threading is not supported. + +### wasm32-emscripten in node + +Node builds use ``NODERAWFS``, ``USE_PTHREADS`` and ``PROXY_TO_PTHREAD`` +linker options. + +- Node RawFS allows direct access to the host file system. +- pthread support requires WASM threads and SharedArrayBuffer (bulk memory). + The runtime keeps a pool of web workers around. Each web worker uses + several file descriptors (eventfd, epoll, pipe). diff --git a/Tools/wasm/config.site-wasm32-emscripten b/Tools/wasm/config.site-wasm32-emscripten index 413506bbc9abd..98991b462446f 100644 --- a/Tools/wasm/config.site-wasm32-emscripten +++ b/Tools/wasm/config.site-wasm32-emscripten @@ -27,9 +27,6 @@ ac_cv_func_prlimit=no # unsupported syscall, https://github.com/emscripten-core/emscripten/issues/13393 ac_cv_func_shutdown=no -# breaks build, see https://github.com/ethanhs/python-wasm/issues/16 -ac_cv_lib_bz2_BZ2_bzCompress=no - # clock_nanosleep() causes time.sleep() to sleep forever. # nanosleep() works correctly ac_cv_func_clock_nanosleep=no @@ -66,6 +63,11 @@ ac_cv_func_pwritev=no ac_cv_func_pipe2=no ac_cv_func_nice=no ac_cv_func_setitimer=no +# unsupported syscall: __syscall_prlimit64 +ac_cv_func_prlimit=no +# unsupported syscall: __syscall_getrusage +ac_cv_func_getrusage=no +ac_cv_func_posix_fallocate=no # Syscalls that resulted in a segfault ac_cv_func_utimensat=no @@ -78,6 +80,20 @@ ac_cv_header_sys_ioctl_h=no ac_cv_func_openpty=no ac_cv_func_forkpty=no +# mkfifo and mknod are broken, create regular file +ac_cv_func_mkfifo=no +ac_cv_func_mkfifoat=no +ac_cv_func_mknod=no +ac_cv_func_mknodat=no + +# always fails with permission error +ac_cv_func_setgroups=no +ac_cv_func_setresuid=no +ac_cv_func_setresgid=no + +# alarm signal is not delivered, may need a callback into the event loop? +ac_cv_func_alarm=no + # To use dlopen, you need to use Emscripten's linking support, -# see https://github.com/emscripten-core/emscripten/wiki/Linking) +# see https://emscripten.org/docs/compiling/Dynamic-Linking.html ac_cv_func_dlopen=no diff --git a/configure b/configure index 9097c0514b57c..49d5abeac3024 100755 --- a/configure +++ b/configure @@ -9844,14 +9844,30 @@ _ACEOF $as_echo_n "checking whether to enable large file support... " >&6; } if test "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \ "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then + have_largefile_support="yes" +else + have_largefile_support="no" +fi +case $ac_sys_system in #( + Emscripten) : + have_largefile_support="no" + ;; #( + *) : + ;; +esac +if test "x$have_largefile_support" = xyes; then : + $as_echo "#define HAVE_LARGEFILE_SUPPORT 1" >>confdefs.h { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } + fi # The cast to long int works around a bug in the HP C Compiler @@ -13751,7 +13767,7 @@ for ac_func in \ gai_strerror getegid getentropy geteuid getgid getgrgid getgrgid_r \ getgrnam_r getgrouplist getgroups getitimer getloadavg getlogin \ getpeername getpgid getpid getppid getpriority _getpty \ - getpwent getpwnam_r getpwuid_r getresgid getresuid getsid getspent \ + getpwent getpwnam_r getpwuid_r getresgid getresuid getrusage getsid getspent \ getspnam getuid getwd if_nameindex initgroups kill killpg lchown linkat \ lockf lstat lutimes madvise mbrtowc memrchr mkdirat mkfifo mkfifoat \ mknod mknodat mktime mmap mremap nice openat opendir pathconf pause pipe \ diff --git a/configure.ac b/configure.ac index b613c18dbb3c5..83dd854248b42 100644 --- a/configure.ac +++ b/configure.ac @@ -2554,15 +2554,24 @@ AC_CHECK_SIZEOF(off_t, [], [ AC_MSG_CHECKING(whether to enable large file support) if test "$ac_cv_sizeof_off_t" -gt "$ac_cv_sizeof_long" -a \ "$ac_cv_sizeof_long_long" -ge "$ac_cv_sizeof_off_t"; then + have_largefile_support="yes" +else + have_largefile_support="no" +fi +dnl LFS does not work with Emscripten 3.1 +AS_CASE([$ac_sys_system], + [Emscripten], [have_largefile_support="no"] +) +AS_VAR_IF([have_largefile_support], [yes], [ AC_DEFINE(HAVE_LARGEFILE_SUPPORT, 1, [Defined to enable large file support when an off_t is bigger than a long and long long is at least as big as an off_t. You may need to add some flags for configuration and compilation to enable this mode. (For Solaris and Linux, the necessary defines are already defined.)]) AC_MSG_RESULT(yes) -else +], [ AC_MSG_RESULT(no) -fi +]) AC_CHECK_SIZEOF(time_t, [], [ #ifdef HAVE_SYS_TYPES_H @@ -4144,7 +4153,7 @@ AC_CHECK_FUNCS([ \ gai_strerror getegid getentropy geteuid getgid getgrgid getgrgid_r \ getgrnam_r getgrouplist getgroups getitimer getloadavg getlogin \ getpeername getpgid getpid getppid getpriority _getpty \ - getpwent getpwnam_r getpwuid_r getresgid getresuid getsid getspent \ + getpwent getpwnam_r getpwuid_r getresgid getresuid getrusage getsid getspent \ getspnam getuid getwd if_nameindex initgroups kill killpg lchown linkat \ lockf lstat lutimes madvise mbrtowc memrchr mkdirat mkfifo mkfifoat \ mknod mknodat mktime mmap mremap nice openat opendir pathconf pause pipe \ diff --git a/pyconfig.h.in b/pyconfig.h.in index 02569c49543b6..a1bf9502e9268 100644 --- a/pyconfig.h.in +++ b/pyconfig.h.in @@ -537,6 +537,9 @@ /* Define to 1 if you have the `getresuid' function. */ #undef HAVE_GETRESUID +/* Define to 1 if you have the `getrusage' function. */ +#undef HAVE_GETRUSAGE + /* Define to 1 if you have the `getsid' function. */ #undef HAVE_GETSID From webhook-mailer at python.org Sun Feb 6 05:22:33 2022 From: webhook-mailer at python.org (JulienPalard) Date: Sun, 06 Feb 2022 10:22:33 -0000 Subject: [Python-checkins] Allows to specify the port on which serve documentation (GH-31145) Message-ID: https://github.com/python/cpython/commit/f1e29cea8516d04c16d94bcb7bf24d4e2d32ffce commit: f1e29cea8516d04c16d94bcb7bf24d4e2d32ffce branch: main author: Christophe Nanteuil <35002064+christopheNan at users.noreply.github.com> committer: JulienPalard date: 2022-02-06T11:22:06+01:00 summary: Allows to specify the port on which serve documentation (GH-31145) [user at localhost]$ make serve # default configuration, no change python3 ../Tools/scripts/serve.py build/html Serving build/html on port 8000, control-C to stop ^CShutting down. [user at localhost]$ make serve SERVE_PORT=8080 # new option python3 ../Tools/scripts/serve.py build/html 8080 Serving build/html on port 8080, control-C to stop files: M Doc/Makefile diff --git a/Doc/Makefile b/Doc/Makefile index 19ddafc5f2f9f..e60aa3427fea2 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -12,6 +12,7 @@ PAPER = SOURCES = DISTVERSION = $(shell $(PYTHON) tools/extensions/patchlevel.py) SPHINXERRORHANDLING = -W +SERVE_PORT = # Internal variables. PAPEROPT_a4 = -D latex_elements.papersize=a4paper @@ -217,7 +218,7 @@ check: $(PYTHON) tools/rstlint.py ../Misc/NEWS.d/next/ serve: - $(PYTHON) ../Tools/scripts/serve.py build/html + $(PYTHON) ../Tools/scripts/serve.py build/html $(SERVE_PORT) # Targets for daily automated doc build # By default, Sphinx only rebuilds pages where the page content has changed. From webhook-mailer at python.org Sun Feb 6 07:13:17 2022 From: webhook-mailer at python.org (vstinner) Date: Sun, 06 Feb 2022 12:13:17 -0000 Subject: [Python-checkins] bpo-46640: Py_NAN now uses the C99 NAN constant (GH-31134) Message-ID: https://github.com/python/cpython/commit/54842e4311bb0e34012d1984b42eab41eeeaea6a commit: 54842e4311bb0e34012d1984b42eab41eeeaea6a branch: main author: Victor Stinner committer: vstinner date: 2022-02-06T13:13:04+01:00 summary: bpo-46640: Py_NAN now uses the C99 NAN constant (GH-31134) Building Python now requires a C99 header file providing a NAN constant, or the __builtin_nan() built-in function. If a platform does not support Not-a-Number (NaN), the Py_NO_NAN macro can be defined in the pyconfig.h file. files: A Misc/NEWS.d/next/Build/2022-02-04-21-26-50.bpo-46640.HXUmQp.rst M Doc/whatsnew/3.11.rst M Include/pymath.h diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 1558d67d9a89f..7b5e7da8f77aa 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -604,6 +604,12 @@ Build Changes ``isinf()``, ``isnan()``, ``round()``. (Contributed by Victor Stinner in :issue:`45440`.) +* Building Python now requires a C99 ```` header file providing + a ``NAN`` constant, or the ``__builtin_nan()`` built-in function. If a + platform does not support Not-a-Number (NaN), the ``Py_NO_NAN`` macro can be + defined in the ``pyconfig.h`` file. + (Contributed by Victor Stinner in :issue:`46640`.) + * Freelists for object structs can now be disabled. A new :program:`configure` option :option:`!--without-freelists` can be used to disable all freelists except empty tuple singleton. diff --git a/Include/pymath.h b/Include/pymath.h index 57310fc097e73..edd0841be9259 100644 --- a/Include/pymath.h +++ b/Include/pymath.h @@ -51,29 +51,18 @@ #endif /* Py_NAN - * A value that evaluates to a NaN. On IEEE 754 platforms INF*0 or - * INF/INF works. Define Py_NO_NAN in pyconfig.h if your platform - * doesn't support NaNs. + * A value that evaluates to a quiet Not-a-Number (NaN). + * Define Py_NO_NAN in pyconfig.h if your platform doesn't support NaNs. */ #if !defined(Py_NAN) && !defined(Py_NO_NAN) -# if !defined(__INTEL_COMPILER) -# define Py_NAN (Py_HUGE_VAL * 0.) -# else /* __INTEL_COMPILER */ -# if defined(ICC_NAN_STRICT) - #pragma float_control(push) - #pragma float_control(precise, on) - #pragma float_control(except, on) - Py_NO_INLINE static double __icc_nan() - { - return sqrt(-1.0); - } - #pragma float_control (pop) -# define Py_NAN __icc_nan() -# else /* ICC_NAN_RELAXED as default for Intel Compiler */ - static const union { unsigned char buf[8]; double __icc_nan; } __nan_store = {0,0,0,0,0,0,0xf8,0x7f}; -# define Py_NAN (__nan_store.__icc_nan) -# endif /* ICC_NAN_STRICT */ -# endif /* __INTEL_COMPILER */ +# if _Py__has_builtin(__builtin_nan) + // Built-in implementation of the ISO C99 function nan(): quiet NaN. +# define Py_NAN (__builtin_nan("")) +#else + // Use C99 NAN constant: quiet Not-A-Number. + // NAN is a float, Py_NAN is a double: cast to double. +# define Py_NAN ((double)NAN) +# endif #endif #endif /* Py_PYMATH_H */ diff --git a/Misc/NEWS.d/next/Build/2022-02-04-21-26-50.bpo-46640.HXUmQp.rst b/Misc/NEWS.d/next/Build/2022-02-04-21-26-50.bpo-46640.HXUmQp.rst new file mode 100644 index 0000000000000..c7381110ebbcb --- /dev/null +++ b/Misc/NEWS.d/next/Build/2022-02-04-21-26-50.bpo-46640.HXUmQp.rst @@ -0,0 +1,4 @@ +Building Python now requires a C99 ```` header file providing a ``NAN`` +constant, or the ``__builtin_nan()`` built-in function. If a platform does not +support Not-a-Number (NaN), the ``Py_NO_NAN`` macro can be defined in the +``pyconfig.h`` file. Patch by Victor Stinner. From webhook-mailer at python.org Sun Feb 6 07:44:26 2022 From: webhook-mailer at python.org (JulienPalard) Date: Sun, 06 Feb 2022 12:44:26 -0000 Subject: [Python-checkins] [doc] Enhance readability by avoiding big blocks for small numbers. (GH-31157) Message-ID: https://github.com/python/cpython/commit/fa90e48c53384889d28f171170ab79d777d46949 commit: fa90e48c53384889d28f171170ab79d777d46949 branch: main author: Julien Palard committer: JulienPalard date: 2022-02-06T13:44:04+01:00 summary: [doc] Enhance readability by avoiding big blocks for small numbers. (GH-31157) Initially reported by Gregory Jacob on the docs@ mailing list: https://mail.python.org/archives/list/docs at python.org/thread/VPSFGLOZOHSPF7TGPOI65AOH25TCPSVR/ files: M Doc/tutorial/floatingpoint.rst diff --git a/Doc/tutorial/floatingpoint.rst b/Doc/tutorial/floatingpoint.rst index 7212b40be8377..e1cd7f9ece75d 100644 --- a/Doc/tutorial/floatingpoint.rst +++ b/Doc/tutorial/floatingpoint.rst @@ -12,15 +12,9 @@ Floating Point Arithmetic: Issues and Limitations Floating-point numbers are represented in computer hardware as base 2 (binary) -fractions. For example, the decimal fraction :: - - 0.125 - -has value 1/10 + 2/100 + 5/1000, and in the same way the binary fraction :: - - 0.001 - -has value 0/2 + 0/4 + 1/8. These two fractions have identical values, the only +fractions. For example, the **decimal** fraction ``0.125`` +has value 1/10 + 2/100 + 5/1000, and in the same way the **binary** fraction ``0.001`` +has value 0/2 + 0/4 + 1/8. These two fractions have identical values, the only real difference being that the first is written in base 10 fractional notation, and the second in base 2. From webhook-mailer at python.org Sun Feb 6 08:59:38 2022 From: webhook-mailer at python.org (ethanfurman) Date: Sun, 06 Feb 2022 13:59:38 -0000 Subject: [Python-checkins] bpo-23952: Document cgi module's maxlen variable (GH-30338) Message-ID: https://github.com/python/cpython/commit/6c4e44ef8ab550f846ba056d4561efb8256b8eab commit: 6c4e44ef8ab550f846ba056d4561efb8256b8eab branch: main author: Hugo van Kemenade committer: ethanfurman date: 2022-02-06T05:59:32-08:00 summary: bpo-23952: Document cgi module's maxlen variable (GH-30338) files: M Doc/library/cgi.rst M Lib/cgi.py diff --git a/Doc/library/cgi.rst b/Doc/library/cgi.rst index c151f04dfb8d0..a2957ba5a68ad 100644 --- a/Doc/library/cgi.rst +++ b/Doc/library/cgi.rst @@ -21,6 +21,11 @@ Support module for Common Gateway Interface (CGI) scripts. This module defines a number of utilities for use by CGI scripts written in Python. +The global variable ``maxlen`` can be set to an integer indicating the maximum +size of a POST request. POST requests larger than this size will result in a +:exc:`ValueError` being raised during parsing. The default value of this +variable is ``0``, meaning the request size is unlimited. + Introduction ------------ diff --git a/Lib/cgi.py b/Lib/cgi.py index 6cb8cf28bd664..22897a14a9c12 100755 --- a/Lib/cgi.py +++ b/Lib/cgi.py @@ -13,6 +13,11 @@ This module defines a number of utilities for use by CGI scripts written in Python. + +The global variable maxlen can be set to an integer indicating the maximum size +of a POST request. POST requests larger than this size will result in a +ValueError being raised during parsing. The default value of this variable is 0, +meaning the request size is unlimited. """ # History From webhook-mailer at python.org Sun Feb 6 09:09:09 2022 From: webhook-mailer at python.org (vstinner) Date: Sun, 06 Feb 2022 14:09:09 -0000 Subject: [Python-checkins] bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) Message-ID: https://github.com/python/cpython/commit/1578de2fcd685c71f9c84e09bac32901dea192c1 commit: 1578de2fcd685c71f9c84e09bac32901dea192c1 branch: main author: Victor Stinner committer: vstinner date: 2022-02-06T15:08:54+01:00 summary: bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) POST requests to http://www.example.com/ fail randomly. files: M Lib/test/test_urllib2.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index a2b1340e0bf5b..0936e435a4870 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1790,6 +1790,8 @@ class MyOtherHTTPHandler(urllib.request.HTTPHandler): @unittest.skipUnless(support.is_resource_enabled('network'), 'test requires network access') + # bpo-46648: test fails randomly with "http://www.example.com/" URL + @unittest.skipIf(True, "POST request to http://www.example.com/ fail randomly") def test_issue16464(self): with socket_helper.transient_internet("http://www.example.com/"): opener = urllib.request.build_opener() From webhook-mailer at python.org Sun Feb 6 09:30:33 2022 From: webhook-mailer at python.org (miss-islington) Date: Sun, 06 Feb 2022 14:30:33 -0000 Subject: [Python-checkins] bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) Message-ID: https://github.com/python/cpython/commit/1b8a34ae65688cfadc81e0174b5aea979b264b3e commit: 1b8a34ae65688cfadc81e0174b5aea979b264b3e branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-06T06:30:18-08:00 summary: bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) POST requests to http://www.example.com/ fail randomly. (cherry picked from commit 1578de2fcd685c71f9c84e09bac32901dea192c1) Co-authored-by: Victor Stinner files: M Lib/test/test_urllib2.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 12c3f4dbd5bf0..d8aab53089bc4 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1788,6 +1788,8 @@ class MyOtherHTTPHandler(urllib.request.HTTPHandler): @unittest.skipUnless(support.is_resource_enabled('network'), 'test requires network access') + # bpo-46648: test fails randomly with "http://www.example.com/" URL + @unittest.skipIf(True, "POST request to http://www.example.com/ fail randomly") def test_issue16464(self): with socket_helper.transient_internet("http://www.example.com/"): opener = urllib.request.build_opener() From webhook-mailer at python.org Sun Feb 6 09:33:12 2022 From: webhook-mailer at python.org (miss-islington) Date: Sun, 06 Feb 2022 14:33:12 -0000 Subject: [Python-checkins] bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) Message-ID: https://github.com/python/cpython/commit/c88407ccf5e72d00e909c2399ff7163501aa7089 commit: c88407ccf5e72d00e909c2399ff7163501aa7089 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-06T06:33:01-08:00 summary: bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) POST requests to http://www.example.com/ fail randomly. (cherry picked from commit 1578de2fcd685c71f9c84e09bac32901dea192c1) Co-authored-by: Victor Stinner files: M Lib/test/test_urllib2.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 9db23e6ce04bd..4d341e357063a 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1790,6 +1790,8 @@ class MyOtherHTTPHandler(urllib.request.HTTPHandler): @unittest.skipUnless(support.is_resource_enabled('network'), 'test requires network access') + # bpo-46648: test fails randomly with "http://www.example.com/" URL + @unittest.skipIf(True, "POST request to http://www.example.com/ fail randomly") def test_issue16464(self): with socket_helper.transient_internet("http://www.example.com/"): opener = urllib.request.build_opener() From webhook-mailer at python.org Sun Feb 6 14:52:47 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 06 Feb 2022 19:52:47 -0000 Subject: [Python-checkins] Add more tests for variable substitution in generics (GH-31170) Message-ID: https://github.com/python/cpython/commit/3da5526136034188185d6a3fdba71e2b56577ee2 commit: 3da5526136034188185d6a3fdba71e2b56577ee2 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2022-02-06T21:52:39+02:00 summary: Add more tests for variable substitution in generics (GH-31170) files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 85f74064458f2..14f49b01aa9f0 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -255,6 +255,15 @@ def test_no_bivariant(self): with self.assertRaises(ValueError): TypeVar('T', covariant=True, contravariant=True) + def test_bad_var_substitution(self): + T = TypeVar('T') + for arg in (), (int, str): + with self.subTest(arg=arg): + with self.assertRaises(TypeError): + List[T][arg] + with self.assertRaises(TypeError): + list[T][arg] + class UnionTests(BaseTestCase): @@ -568,8 +577,11 @@ def test_var_substitution(self): C2 = Callable[[KT, T], VT] C3 = Callable[..., T] self.assertEqual(C1[str], Callable[[int, str], str]) + if Callable is typing.Callable: + self.assertEqual(C1[None], Callable[[int, type(None)], type(None)]) self.assertEqual(C2[int, float, str], Callable[[int, float], str]) self.assertEqual(C3[int], Callable[..., int]) + self.assertEqual(C3[NoReturn], Callable[..., NoReturn]) # multi chaining C4 = C2[int, VT, str] @@ -4981,6 +4993,17 @@ class X(Generic[P, P2]): self.assertEqual(G1.__args__, ((int, str), (bytes,))) self.assertEqual(G2.__args__, ((int,), (str, bytes))) + def test_bad_var_substitution(self): + T = TypeVar('T') + P = ParamSpec('P') + bad_args = (42, int, None, T, int|str, Union[int, str]) + for arg in bad_args: + with self.subTest(arg=arg): + with self.assertRaises(TypeError): + typing.Callable[P, T][arg, str] + with self.assertRaises(TypeError): + collections.abc.Callable[P, T][arg, str] + def test_no_paramspec_in__parameters__(self): # ParamSpec should not be found in __parameters__ # of generics. Usages outside Callable, Concatenate @@ -5010,6 +5033,10 @@ def test_paramspec_in_nested_generics(self): self.assertEqual(G1.__parameters__, (P, T)) self.assertEqual(G2.__parameters__, (P, T)) self.assertEqual(G3.__parameters__, (P, T)) + C = Callable[[int, str], float] + self.assertEqual(G1[[int, str], float], List[C]) + self.assertEqual(G2[[int, str], float], list[C]) + self.assertEqual(G3[[int, str], float], list[C] | int) class ConcatenateTests(BaseTestCase): From webhook-mailer at python.org Sun Feb 6 15:15:37 2022 From: webhook-mailer at python.org (miss-islington) Date: Sun, 06 Feb 2022 20:15:37 -0000 Subject: [Python-checkins] Add more tests for variable substitution in generics (GH-31170) Message-ID: https://github.com/python/cpython/commit/cb789944b22311b5a6de3c1c4f8bc4044ddb949a commit: cb789944b22311b5a6de3c1c4f8bc4044ddb949a branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-06T12:15:29-08:00 summary: Add more tests for variable substitution in generics (GH-31170) (cherry picked from commit 3da5526136034188185d6a3fdba71e2b56577ee2) Co-authored-by: Serhiy Storchaka files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 366f7d8eb31d1..895805fb2a260 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -247,6 +247,15 @@ def test_no_bivariant(self): with self.assertRaises(ValueError): TypeVar('T', covariant=True, contravariant=True) + def test_bad_var_substitution(self): + T = TypeVar('T') + for arg in (), (int, str): + with self.subTest(arg=arg): + with self.assertRaises(TypeError): + List[T][arg] + with self.assertRaises(TypeError): + list[T][arg] + class UnionTests(BaseTestCase): @@ -560,8 +569,11 @@ def test_var_substitution(self): C2 = Callable[[KT, T], VT] C3 = Callable[..., T] self.assertEqual(C1[str], Callable[[int, str], str]) + if Callable is typing.Callable: + self.assertEqual(C1[None], Callable[[int, type(None)], type(None)]) self.assertEqual(C2[int, float, str], Callable[[int, float], str]) self.assertEqual(C3[int], Callable[..., int]) + self.assertEqual(C3[NoReturn], Callable[..., NoReturn]) # multi chaining C4 = C2[int, VT, str] @@ -4880,6 +4892,17 @@ class X(Generic[P, P2]): self.assertEqual(G1.__args__, ((int, str), (bytes,))) self.assertEqual(G2.__args__, ((int,), (str, bytes))) + def test_bad_var_substitution(self): + T = TypeVar('T') + P = ParamSpec('P') + bad_args = (42, int, None, T, int|str, Union[int, str]) + for arg in bad_args: + with self.subTest(arg=arg): + with self.assertRaises(TypeError): + typing.Callable[P, T][arg, str] + with self.assertRaises(TypeError): + collections.abc.Callable[P, T][arg, str] + def test_no_paramspec_in__parameters__(self): # ParamSpec should not be found in __parameters__ # of generics. Usages outside Callable, Concatenate @@ -4909,6 +4932,10 @@ def test_paramspec_in_nested_generics(self): self.assertEqual(G1.__parameters__, (P, T)) self.assertEqual(G2.__parameters__, (P, T)) self.assertEqual(G3.__parameters__, (P, T)) + C = Callable[[int, str], float] + self.assertEqual(G1[[int, str], float], List[C]) + self.assertEqual(G2[[int, str], float], list[C]) + self.assertEqual(G3[[int, str], float], list[C] | int) class ConcatenateTests(BaseTestCase): From webhook-mailer at python.org Sun Feb 6 15:50:18 2022 From: webhook-mailer at python.org (vstinner) Date: Sun, 06 Feb 2022 20:50:18 -0000 Subject: [Python-checkins] bpo-46659: Update the test on the mbcs codec alias (GH-31168) Message-ID: https://github.com/python/cpython/commit/04dd60e50cd3da48fd19cdab4c0e4cc600d6af30 commit: 04dd60e50cd3da48fd19cdab4c0e4cc600d6af30 branch: main author: Victor Stinner committer: vstinner date: 2022-02-06T21:50:09+01:00 summary: bpo-46659: Update the test on the mbcs codec alias (GH-31168) encodings registers the _alias_mbcs() codec search function before the search_function() codec search function. Previously, the _alias_mbcs() was never used. Fix the test_codecs.test_mbcs_alias() test: use the current ANSI code page, not a fake ANSI code page number. Remove the test_site.test_aliasing_mbcs() test: the alias is now implemented in the encodings module, no longer in the site module. files: M Lib/encodings/__init__.py M Lib/test/test_codecs.py M Lib/test/test_site.py diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py index 4b37d3321c903..dff22a4b27f45 100644 --- a/Lib/encodings/__init__.py +++ b/Lib/encodings/__init__.py @@ -152,9 +152,6 @@ def search_function(encoding): # Return the registry entry return entry -# Register the search_function in the Python codec registry -codecs.register(search_function) - if sys.platform == 'win32': def _alias_mbcs(encoding): try: @@ -167,4 +164,8 @@ def _alias_mbcs(encoding): # Imports may fail while we are shutting down pass + # It must be registered before search_function() codecs.register(_alias_mbcs) + +# Register the search_function in the Python codec registry +codecs.register(search_function) diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index 4ad24dbb9a924..d30ff8f82db9a 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -1904,7 +1904,10 @@ def test_basics(self): name += "_codec" elif encoding == "latin_1": name = "latin_1" - self.assertEqual(encoding.replace("_", "-"), name.replace("_", "-")) + # Skip the mbcs alias on Windows + if name != "mbcs": + self.assertEqual(encoding.replace("_", "-"), + name.replace("_", "-")) (b, size) = codecs.getencoder(encoding)(s) self.assertEqual(size, len(s), "encoding=%r" % encoding) @@ -3188,11 +3191,13 @@ def test_incremental(self): self.assertEqual(decoded, ('abc', 3)) def test_mbcs_alias(self): - # Check that looking up our 'default' codepage will return - # mbcs when we don't have a more specific one available - with mock.patch('_winapi.GetACP', return_value=123): - codec = codecs.lookup('cp123') - self.assertEqual(codec.name, 'mbcs') + # On Windows, the encoding name must be the ANSI code page + encoding = locale.getpreferredencoding(False) + self.assertTrue(encoding.startswith('cp'), encoding) + + # The encodings module create a "mbcs" alias to the ANSI code page + codec = codecs.lookup(encoding) + self.assertEqual(codec.name, "mbcs") @support.bigmemtest(size=2**31, memuse=7, dry_run=False) def test_large_input(self, size): diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py index 032a1be3aa529..a67cfec72aee4 100644 --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -456,16 +456,6 @@ def test_setting_help(self): # 'help' should be set in builtins self.assertTrue(hasattr(builtins, "help")) - def test_aliasing_mbcs(self): - if sys.platform == "win32": - import locale - if locale.getdefaultlocale()[1].startswith('cp'): - for value in encodings.aliases.aliases.values(): - if value == "mbcs": - break - else: - self.fail("did not alias mbcs") - def test_sitecustomize_executed(self): # If sitecustomize is available, it should have been imported. if "sitecustomize" not in sys.modules: From webhook-mailer at python.org Sun Feb 6 15:52:04 2022 From: webhook-mailer at python.org (vstinner) Date: Sun, 06 Feb 2022 20:52:04 -0000 Subject: [Python-checkins] bpo-46659: test.support avoids locale.getdefaultlocale() (GH-31167) Message-ID: https://github.com/python/cpython/commit/06b8f1615b09099fae5c5393334b8716a4144d20 commit: 06b8f1615b09099fae5c5393334b8716a4144d20 branch: main author: Victor Stinner committer: vstinner date: 2022-02-06T21:51:56+01:00 summary: bpo-46659: test.support avoids locale.getdefaultlocale() (GH-31167) skip_if_buggy_ucrt_strfptime() of test.support now uses locale.getpreferredencoding(False) instead of locale.getdefaultlocale() to get the Windows code page. files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 9d0d3d9c9b619..d80472d8776f6 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1421,7 +1421,7 @@ def skip_if_buggy_ucrt_strfptime(test): global _buggy_ucrt if _buggy_ucrt is None: if(sys.platform == 'win32' and - locale.getdefaultlocale()[1] == 'cp65001' and + locale.getpreferredencoding(False) == 'cp65001' and time.localtime().tm_zone == ''): _buggy_ucrt = True else: From webhook-mailer at python.org Sun Feb 6 18:16:29 2022 From: webhook-mailer at python.org (gvanrossum) Date: Sun, 06 Feb 2022 23:16:29 -0000 Subject: [Python-checkins] bpo-46655: allow stringized TypeAlias with get_type_hints (GH-31156) Message-ID: https://github.com/python/cpython/commit/77b025be4a4cd5a3bfc1b1af560cc57e8e956c98 commit: 77b025be4a4cd5a3bfc1b1af560cc57e8e956c98 branch: main author: Gregory Beauregard committer: gvanrossum date: 2022-02-06T15:16:22-08:00 summary: bpo-46655: allow stringized TypeAlias with get_type_hints (GH-31156) files: A Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 14f49b01aa9f0..2aee5c33b6674 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4866,6 +4866,11 @@ def test_no_isinstance(self): with self.assertRaises(TypeError): isinstance(42, TypeAlias) + def test_stringized_usage(self): + class A: + a: "TypeAlias" + self.assertEqual(get_type_hints(A), {'a': TypeAlias}) + def test_no_issubclass(self): with self.assertRaises(TypeError): issubclass(Employee, TypeAlias) diff --git a/Lib/typing.py b/Lib/typing.py index e4e32b5b320d0..44c239c43ee4e 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -174,7 +174,7 @@ def _type_check(arg, msg, is_argument=True, module=None, *, allow_special_forms= if (isinstance(arg, _GenericAlias) and arg.__origin__ in invalid_generic_forms): raise TypeError(f"{arg} is not valid as type argument") - if arg in (Any, NoReturn, ClassVar, Final): + if arg in (Any, NoReturn, ClassVar, Final, TypeAlias): return arg if isinstance(arg, _SpecialForm) or arg in (Generic, Protocol): raise TypeError(f"Plain {arg} is not valid as type argument") diff --git a/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst b/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst new file mode 100644 index 0000000000000..4f0de9519a00e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst @@ -0,0 +1 @@ +In :func:`typing.get_type_hints`, support evaluating bare stringified ``TypeAlias`` annotations. Patch by Gregory Beauregard. \ No newline at end of file From webhook-mailer at python.org Sun Feb 6 19:25:48 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 00:25:48 -0000 Subject: [Python-checkins] bpo-46670: Remove unused get_frame_state() function (GH-31177) Message-ID: https://github.com/python/cpython/commit/16f96a4cf9ab1e91e6e8e18232378bc4b42bb796 commit: 16f96a4cf9ab1e91e6e8e18232378bc4b42bb796 branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T01:25:44+01:00 summary: bpo-46670: Remove unused get_frame_state() function (GH-31177) The PyFrame_MAXFREELIST macro is no longer defined. files: M Objects/frameobject.c diff --git a/Objects/frameobject.c b/Objects/frameobject.c index 78f3894111bc3..dd29541f04b0e 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -20,15 +20,6 @@ static PyMemberDef frame_memberlist[] = { {NULL} /* Sentinel */ }; -#if PyFrame_MAXFREELIST > 0 -static struct _Py_frame_state * -get_frame_state(void) -{ - PyInterpreterState *interp = _PyInterpreterState_GET(); - return &interp->frame; -} -#endif - static PyObject * frame_getlocals(PyFrameObject *f, void *closure) From webhook-mailer at python.org Sun Feb 6 19:27:02 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 00:27:02 -0000 Subject: [Python-checkins] bpo-46670: Define all macros for stringlib (GH-31176) Message-ID: https://github.com/python/cpython/commit/097f74a5a37e2a8a26d529cede456ede7011b66f commit: 097f74a5a37e2a8a26d529cede456ede7011b66f branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T01:26:58+01:00 summary: bpo-46670: Define all macros for stringlib (GH-31176) bytesobject.c, bytearrayobject.c and unicodeobject.c now define all macros used by stringlib, to avoid using undefined macros. Fix "gcc -Wundef" warnings. files: M Objects/bytearrayobject.c M Objects/bytesobject.c M Objects/stringlib/asciilib.h M Objects/stringlib/join.h M Objects/stringlib/split.h M Objects/stringlib/ucs1lib.h M Objects/stringlib/ucs2lib.h M Objects/stringlib/ucs4lib.h M Objects/stringlib/undef.h M Objects/stringlib/unicodedefs.h diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c index a6009854221ff..8fce83dbfb50b 100644 --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -1074,6 +1074,7 @@ bytearray_dealloc(PyByteArrayObject *self) /* -------------------------------------------------------------------- */ /* Methods */ +#define STRINGLIB_IS_UNICODE 0 #define FASTSEARCH fastsearch #define STRINGLIB(F) stringlib_##F #define STRINGLIB_CHAR char diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c index b6edfb9acb2db..cfc358f4deba2 100644 --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1235,6 +1235,7 @@ PyBytes_AsStringAndSize(PyObject *obj, #define STRINGLIB_GET_EMPTY() bytes_get_empty() #include "stringlib/stringdefs.h" +#define STRINGLIB_MUTABLE 0 #include "stringlib/fastsearch.h" #include "stringlib/count.h" diff --git a/Objects/stringlib/asciilib.h b/Objects/stringlib/asciilib.h index 7749e8fb33982..eebe888e411e0 100644 --- a/Objects/stringlib/asciilib.h +++ b/Objects/stringlib/asciilib.h @@ -20,6 +20,7 @@ #define STRINGLIB_NEW(STR,LEN) _PyUnicode_FromASCII((const char*)(STR),(LEN)) #define STRINGLIB_CHECK PyUnicode_Check #define STRINGLIB_CHECK_EXACT PyUnicode_CheckExact +#define STRINGLIB_MUTABLE 0 #define STRINGLIB_TOSTR PyObject_Str #define STRINGLIB_TOASCII PyObject_ASCII diff --git a/Objects/stringlib/join.h b/Objects/stringlib/join.h index 62e4c98de7f25..bb011f7db796d 100644 --- a/Objects/stringlib/join.h +++ b/Objects/stringlib/join.h @@ -32,7 +32,7 @@ STRINGLIB(bytes_join)(PyObject *sep, PyObject *iterable) Py_DECREF(seq); return STRINGLIB_NEW(NULL, 0); } -#ifndef STRINGLIB_MUTABLE +#if !STRINGLIB_MUTABLE if (seqlen == 1) { item = PySequence_Fast_GET_ITEM(seq, 0); if (STRINGLIB_CHECK_EXACT(item)) { diff --git a/Objects/stringlib/split.h b/Objects/stringlib/split.h index 068047f9874a0..0c11b7214e9b0 100644 --- a/Objects/stringlib/split.h +++ b/Objects/stringlib/split.h @@ -70,7 +70,7 @@ STRINGLIB(split_whitespace)(PyObject* str_obj, j = i; i++; while (i < str_len && !STRINGLIB_ISSPACE(str[i])) i++; -#ifndef STRINGLIB_MUTABLE +#if !STRINGLIB_MUTABLE if (j == 0 && i == str_len && STRINGLIB_CHECK_EXACT(str_obj)) { /* No whitespace in str_obj, so just use it as list[0] */ Py_INCREF(str_obj); @@ -122,7 +122,7 @@ STRINGLIB(split_char)(PyObject* str_obj, } } } -#ifndef STRINGLIB_MUTABLE +#if !STRINGLIB_MUTABLE if (count == 0 && STRINGLIB_CHECK_EXACT(str_obj)) { /* ch not in str_obj, so just use str_obj as list[0] */ Py_INCREF(str_obj); @@ -170,7 +170,7 @@ STRINGLIB(split)(PyObject* str_obj, SPLIT_ADD(str, i, j); i = j + sep_len; } -#ifndef STRINGLIB_MUTABLE +#if !STRINGLIB_MUTABLE if (count == 0 && STRINGLIB_CHECK_EXACT(str_obj)) { /* No match in str_obj, so just use it as list[0] */ Py_INCREF(str_obj); @@ -209,7 +209,7 @@ STRINGLIB(rsplit_whitespace)(PyObject* str_obj, j = i; i--; while (i >= 0 && !STRINGLIB_ISSPACE(str[i])) i--; -#ifndef STRINGLIB_MUTABLE +#if !STRINGLIB_MUTABLE if (j == str_len - 1 && i < 0 && STRINGLIB_CHECK_EXACT(str_obj)) { /* No whitespace in str_obj, so just use it as list[0] */ Py_INCREF(str_obj); @@ -262,7 +262,7 @@ STRINGLIB(rsplit_char)(PyObject* str_obj, } } } -#ifndef STRINGLIB_MUTABLE +#if !STRINGLIB_MUTABLE if (count == 0 && STRINGLIB_CHECK_EXACT(str_obj)) { /* ch not in str_obj, so just use str_obj as list[0] */ Py_INCREF(str_obj); @@ -311,7 +311,7 @@ STRINGLIB(rsplit)(PyObject* str_obj, SPLIT_ADD(str, pos + sep_len, j); j = pos; } -#ifndef STRINGLIB_MUTABLE +#if !STRINGLIB_MUTABLE if (count == 0 && STRINGLIB_CHECK_EXACT(str_obj)) { /* No match in str_obj, so just use it as list[0] */ Py_INCREF(str_obj); @@ -370,7 +370,7 @@ STRINGLIB(splitlines)(PyObject* str_obj, if (keepends) eol = i; } -#ifndef STRINGLIB_MUTABLE +#if !STRINGLIB_MUTABLE if (j == 0 && eol == str_len && STRINGLIB_CHECK_EXACT(str_obj)) { /* No linebreak in str_obj, so just use it as list[0] */ if (PyList_Append(list, str_obj)) diff --git a/Objects/stringlib/ucs1lib.h b/Objects/stringlib/ucs1lib.h index 5b0b8a025e808..026ab11f1f7b8 100644 --- a/Objects/stringlib/ucs1lib.h +++ b/Objects/stringlib/ucs1lib.h @@ -20,6 +20,7 @@ #define STRINGLIB_NEW _PyUnicode_FromUCS1 #define STRINGLIB_CHECK PyUnicode_Check #define STRINGLIB_CHECK_EXACT PyUnicode_CheckExact +#define STRINGLIB_MUTABLE 0 #define STRINGLIB_TOSTR PyObject_Str #define STRINGLIB_TOASCII PyObject_ASCII diff --git a/Objects/stringlib/ucs2lib.h b/Objects/stringlib/ucs2lib.h index 6af01511c5f8a..75f11bc290508 100644 --- a/Objects/stringlib/ucs2lib.h +++ b/Objects/stringlib/ucs2lib.h @@ -20,6 +20,7 @@ #define STRINGLIB_NEW _PyUnicode_FromUCS2 #define STRINGLIB_CHECK PyUnicode_Check #define STRINGLIB_CHECK_EXACT PyUnicode_CheckExact +#define STRINGLIB_MUTABLE 0 #define STRINGLIB_TOSTR PyObject_Str #define STRINGLIB_TOASCII PyObject_ASCII diff --git a/Objects/stringlib/ucs4lib.h b/Objects/stringlib/ucs4lib.h index 39071a0cdf0cd..57344f235b659 100644 --- a/Objects/stringlib/ucs4lib.h +++ b/Objects/stringlib/ucs4lib.h @@ -20,6 +20,7 @@ #define STRINGLIB_NEW _PyUnicode_FromUCS4 #define STRINGLIB_CHECK PyUnicode_Check #define STRINGLIB_CHECK_EXACT PyUnicode_CheckExact +#define STRINGLIB_MUTABLE 0 #define STRINGLIB_TOSTR PyObject_Str #define STRINGLIB_TOASCII PyObject_ASCII diff --git a/Objects/stringlib/undef.h b/Objects/stringlib/undef.h index c41e254fde6f4..bf32298505ed7 100644 --- a/Objects/stringlib/undef.h +++ b/Objects/stringlib/undef.h @@ -1,10 +1,10 @@ -#undef FASTSEARCH -#undef STRINGLIB -#undef STRINGLIB_SIZEOF_CHAR -#undef STRINGLIB_MAX_CHAR -#undef STRINGLIB_CHAR -#undef STRINGLIB_STR -#undef STRINGLIB_LEN -#undef STRINGLIB_NEW +#undef FASTSEARCH +#undef STRINGLIB +#undef STRINGLIB_SIZEOF_CHAR +#undef STRINGLIB_MAX_CHAR +#undef STRINGLIB_CHAR +#undef STRINGLIB_STR +#undef STRINGLIB_LEN +#undef STRINGLIB_NEW #undef STRINGLIB_IS_UNICODE - +#undef STRINGLIB_MUTABLE diff --git a/Objects/stringlib/unicodedefs.h b/Objects/stringlib/unicodedefs.h index 5ea79cd4f50ac..ba2ce0aeea1cc 100644 --- a/Objects/stringlib/unicodedefs.h +++ b/Objects/stringlib/unicodedefs.h @@ -22,6 +22,7 @@ #define STRINGLIB_NEW PyUnicode_FromUnicode #define STRINGLIB_CHECK PyUnicode_Check #define STRINGLIB_CHECK_EXACT PyUnicode_CheckExact +#define STRINGLIB_MUTABLE 0 #define STRINGLIB_TOSTR PyObject_Str #define STRINGLIB_TOASCII PyObject_ASCII From webhook-mailer at python.org Sun Feb 6 19:46:56 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 00:46:56 -0000 Subject: [Python-checkins] bpo-46670: Test if a macro is defined, not its value (GH-31178) Message-ID: https://github.com/python/cpython/commit/b556f53785cb9ad8cc088ad8c10efee91fe3da58 commit: b556f53785cb9ad8cc088ad8c10efee91fe3da58 branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T01:46:51+01:00 summary: bpo-46670: Test if a macro is defined, not its value (GH-31178) * audioop.c: #ifdef WORDS_BIGENDIAN * ctypes.h: #ifdef USING_MALLOC_CLOSURE_DOT_C * _ctypes/malloc_closure.c: #ifdef HAVE_FFI_CLOSURE_ALLOC and #ifdef USING_APPLE_OS_LIBFFI * pytime.c: #ifdef __APPLE__ * unicodeobject.c: #ifdef HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION files: M Modules/_ctypes/callbacks.c M Modules/_ctypes/callproc.c M Modules/_ctypes/ctypes.h M Modules/_ctypes/malloc_closure.c M Modules/audioop.c M Objects/unicodeobject.c M Python/pytime.c diff --git a/Modules/_ctypes/callbacks.c b/Modules/_ctypes/callbacks.c index 01d703745bc60..b4079eee44667 100644 --- a/Modules/_ctypes/callbacks.c +++ b/Modules/_ctypes/callbacks.c @@ -418,7 +418,7 @@ CThunkObject *_ctypes_alloc_callback(PyObject *callable, goto error; } #if HAVE_FFI_PREP_CLOSURE_LOC -# if USING_APPLE_OS_LIBFFI +# ifdef USING_APPLE_OS_LIBFFI # define HAVE_FFI_PREP_CLOSURE_LOC_RUNTIME __builtin_available(macos 10.15, ios 13, watchos 6, tvos 13, *) # else # define HAVE_FFI_PREP_CLOSURE_LOC_RUNTIME 1 @@ -430,7 +430,7 @@ CThunkObject *_ctypes_alloc_callback(PyObject *callable, } else #endif { -#if USING_APPLE_OS_LIBFFI && defined(__arm64__) +#if defined(USING_APPLE_OS_LIBFFI) && defined(__arm64__) PyErr_Format(PyExc_NotImplementedError, "ffi_prep_closure_loc() is missing"); goto error; #else diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index e2204961070db..928737edb09b1 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -826,7 +826,7 @@ static int _call_function_pointer(int flags, cc = FFI_STDCALL; #endif -# if USING_APPLE_OS_LIBFFI +# ifdef USING_APPLE_OS_LIBFFI # define HAVE_FFI_PREP_CIF_VAR_RUNTIME __builtin_available(macos 10.15, ios 13, watchos 6, tvos 13, *) # elif HAVE_FFI_PREP_CIF_VAR # define HAVE_FFI_PREP_CIF_VAR_RUNTIME true diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h index 9600ddc7413b2..9e82ce8574714 100644 --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -362,7 +362,7 @@ PyObject *_ctypes_get_errobj(int **pspace); extern PyObject *ComError; #endif -#if USING_MALLOC_CLOSURE_DOT_C +#ifdef USING_MALLOC_CLOSURE_DOT_C void Py_ffi_closure_free(void *p); void *Py_ffi_closure_alloc(size_t size, void** codeloc); #else diff --git a/Modules/_ctypes/malloc_closure.c b/Modules/_ctypes/malloc_closure.c index 788bae6a96c7f..38edc90e70763 100644 --- a/Modules/_ctypes/malloc_closure.c +++ b/Modules/_ctypes/malloc_closure.c @@ -91,13 +91,13 @@ static void more_core(void) /* put the item back into the free list */ void Py_ffi_closure_free(void *p) { -#if HAVE_FFI_CLOSURE_ALLOC -#if USING_APPLE_OS_LIBFFI +#ifdef HAVE_FFI_CLOSURE_ALLOC +#ifdef USING_APPLE_OS_LIBFFI if (__builtin_available(macos 10.15, ios 13, watchos 6, tvos 13, *)) { #endif ffi_closure_free(p); return; -#if USING_APPLE_OS_LIBFFI +#ifdef USING_APPLE_OS_LIBFFI } #endif #endif @@ -109,12 +109,12 @@ void Py_ffi_closure_free(void *p) /* return one item from the free list, allocating more if needed */ void *Py_ffi_closure_alloc(size_t size, void** codeloc) { -#if HAVE_FFI_CLOSURE_ALLOC -#if USING_APPLE_OS_LIBFFI +#ifdef HAVE_FFI_CLOSURE_ALLOC +#ifdef USING_APPLE_OS_LIBFFI if (__builtin_available(macos 10.15, ios 13, watchos 6, tvos 13, *)) { #endif return ffi_closure_alloc(size, codeloc); -#if USING_APPLE_OS_LIBFFI +#ifdef USING_APPLE_OS_LIBFFI } #endif #endif diff --git a/Modules/audioop.c b/Modules/audioop.c index 2a5d805c053c7..32237ca6177cd 100644 --- a/Modules/audioop.c +++ b/Modules/audioop.c @@ -297,7 +297,7 @@ static const int stepsizeTable[89] = { #define GETINT16(cp, i) GETINTX(int16_t, (cp), (i)) #define GETINT32(cp, i) GETINTX(int32_t, (cp), (i)) -#if WORDS_BIGENDIAN +#ifdef WORDS_BIGENDIAN #define GETINT24(cp, i) ( \ ((unsigned char *)(cp) + (i))[2] + \ (((unsigned char *)(cp) + (i))[1] << 8) + \ @@ -314,7 +314,7 @@ static const int stepsizeTable[89] = { #define SETINT16(cp, i, val) SETINTX(int16_t, (cp), (i), (val)) #define SETINT32(cp, i, val) SETINTX(int32_t, (cp), (i), (val)) -#if WORDS_BIGENDIAN +#ifdef WORDS_BIGENDIAN #define SETINT24(cp, i, val) do { \ ((unsigned char *)(cp) + (i))[2] = (int)(val); \ ((unsigned char *)(cp) + (i))[1] = (int)(val) >> 8; \ diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 4cea0d8e62e85..888871a90c875 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -61,7 +61,7 @@ OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. #endif #ifdef HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION -#include "pycore_fileutils.h" // _Py_LocaleUsesNonUnicodeWchar() +# include "pycore_fileutils.h" // _Py_LocaleUsesNonUnicodeWchar() #endif /* Uncomment to display statistics on interned strings at exit @@ -3344,7 +3344,7 @@ PyUnicode_AsWideChar(PyObject *unicode, } unicode_copy_as_widechar(unicode, w, size); -#if HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION +#ifdef HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION /* Oracle Solaris uses non-Unicode internal wchar_t form for non-Unicode locales and hence needs conversion first. */ if (_Py_LocaleUsesNonUnicodeWchar()) { @@ -3381,7 +3381,7 @@ PyUnicode_AsWideCharString(PyObject *unicode, } unicode_copy_as_widechar(unicode, buffer, buflen + 1); -#if HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION +#ifdef HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION /* Oracle Solaris uses non-Unicode internal wchar_t form for non-Unicode locales and hence needs conversion first. */ if (_Py_LocaleUsesNonUnicodeWchar()) { diff --git a/Python/pytime.c b/Python/pytime.c index 9653662b0fb14..aff20c6e5f1d4 100644 --- a/Python/pytime.c +++ b/Python/pytime.c @@ -962,7 +962,7 @@ _PyTime_GetSystemClockWithInfo(_PyTime_t *t, _Py_clock_info_t *info) } -#if __APPLE__ +#ifdef __APPLE__ static int py_mach_timebase_info(_PyTime_t *pnumer, _PyTime_t *pdenom, int raise) { From webhook-mailer at python.org Sun Feb 6 20:35:55 2022 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 07 Feb 2022 01:35:55 -0000 Subject: [Python-checkins] bpo-46611: add coverage to instance and class checks in `typing.py` (GH-31078) Message-ID: https://github.com/python/cpython/commit/067c03bf40d13393209f0138fa9c4d5980c4ff8a commit: 067c03bf40d13393209f0138fa9c4d5980c4ff8a branch: main author: Nikita Sobolev committer: gvanrossum date: 2022-02-06T17:35:48-08:00 summary: bpo-46611: add coverage to instance and class checks in `typing.py` (GH-31078) files: M Lib/test/test_types.py M Lib/test/test_typing.py diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py index c54854eeb5ad2..b8b1ce96f93c3 100644 --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -703,22 +703,62 @@ def test_hash(self): self.assertEqual(hash(int | str), hash(str | int)) self.assertEqual(hash(int | str), hash(typing.Union[int, str])) - def test_instancecheck(self): - x = int | str - self.assertIsInstance(1, x) - self.assertIsInstance(True, x) - self.assertIsInstance('a', x) - self.assertNotIsInstance(None, x) - self.assertTrue(issubclass(int, x)) - self.assertTrue(issubclass(bool, x)) - self.assertTrue(issubclass(str, x)) - self.assertFalse(issubclass(type(None), x)) - x = int | None - self.assertIsInstance(None, x) - self.assertTrue(issubclass(type(None), x)) - x = int | collections.abc.Mapping - self.assertIsInstance({}, x) - self.assertTrue(issubclass(dict, x)) + def test_instancecheck_and_subclasscheck(self): + for x in (int | str, typing.Union[int, str]): + with self.subTest(x=x): + self.assertIsInstance(1, x) + self.assertIsInstance(True, x) + self.assertIsInstance('a', x) + self.assertNotIsInstance(None, x) + self.assertTrue(issubclass(int, x)) + self.assertTrue(issubclass(bool, x)) + self.assertTrue(issubclass(str, x)) + self.assertFalse(issubclass(type(None), x)) + + for x in (int | None, typing.Union[int, None]): + with self.subTest(x=x): + self.assertIsInstance(None, x) + self.assertTrue(issubclass(type(None), x)) + + for x in ( + int | collections.abc.Mapping, + typing.Union[int, collections.abc.Mapping], + ): + with self.subTest(x=x): + self.assertIsInstance({}, x) + self.assertNotIsInstance((), x) + self.assertTrue(issubclass(dict, x)) + self.assertFalse(issubclass(list, x)) + + def test_instancecheck_and_subclasscheck_order(self): + T = typing.TypeVar('T') + + will_resolve = ( + int | T, + typing.Union[int, T], + ) + for x in will_resolve: + with self.subTest(x=x): + self.assertIsInstance(1, x) + self.assertTrue(issubclass(int, x)) + + wont_resolve = ( + T | int, + typing.Union[T, int], + ) + for x in wont_resolve: + with self.subTest(x=x): + with self.assertRaises(TypeError): + issubclass(int, x) + with self.assertRaises(TypeError): + isinstance(1, x) + + for x in (*will_resolve, *wont_resolve): + with self.subTest(x=x): + with self.assertRaises(TypeError): + issubclass(object, x) + with self.assertRaises(TypeError): + isinstance(object(), x) def test_bad_instancecheck(self): class BadMeta(type): diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 2aee5c33b6674..9eab4618e2b01 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -439,6 +439,8 @@ def test_tuple_subclass(self): class MyTuple(tuple): pass self.assertIsSubclass(MyTuple, Tuple) + self.assertIsSubclass(Tuple, Tuple) + self.assertIsSubclass(tuple, Tuple) def test_tuple_instance_type_error(self): with self.assertRaises(TypeError): @@ -466,6 +468,7 @@ def test_self_subclass(self): with self.assertRaises(TypeError): issubclass(types.FunctionType, Callable[[int], int]) self.assertIsSubclass(types.FunctionType, Callable) + self.assertIsSubclass(Callable, Callable) def test_eq_hash(self): Callable = self.Callable From webhook-mailer at python.org Sun Feb 6 20:59:40 2022 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 07 Feb 2022 01:59:40 -0000 Subject: [Python-checkins] bpo-46589: Improve documentation for typing._GenericAlias (GH-31026) Message-ID: https://github.com/python/cpython/commit/a335d98f19b7fa66159d12932cc9a05eb2d8395f commit: a335d98f19b7fa66159d12932cc9a05eb2d8395f branch: main author: Matthew Rahtz committer: gvanrossum date: 2022-02-06T17:59:22-08:00 summary: bpo-46589: Improve documentation for typing._GenericAlias (GH-31026) (These should arguably be docstrings per convention in the rest of the file, but it doesn't really matter.) files: M Lib/typing.py diff --git a/Lib/typing.py b/Lib/typing.py index 44c239c43ee4e..f733efea09cb3 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -1000,16 +1000,41 @@ def __dir__(self): class _GenericAlias(_BaseGenericAlias, _root=True): - def __init__(self, origin, params, *, inst=True, name=None, + # The type of parameterized generics. + # + # That is, for example, `type(List[int])` is `_GenericAlias`. + # + # Objects which are instances of this class include: + # * Parameterized container types, e.g. `Tuple[int]`, `List[int]`. + # * Note that native container types, e.g. `tuple`, `list`, use + # `types.GenericAlias` instead. + # * Parameterized classes: + # T = TypeVar('T') + # class C(Generic[T]): pass + # # C[int] is a _GenericAlias + # * `Callable` aliases, generic `Callable` aliases, and + # parameterized `Callable` aliases: + # T = TypeVar('T') + # # _CallableGenericAlias inherits from _GenericAlias. + # A = Callable[[], None] # _CallableGenericAlias + # B = Callable[[T], None] # _CallableGenericAlias + # C = B[int] # _CallableGenericAlias + # * Parameterized `Final`, `ClassVar` and `TypeGuard`: + # # All _GenericAlias + # Final[int] + # ClassVar[float] + # TypeVar[bool] + + def __init__(self, origin, args, *, inst=True, name=None, _typevar_types=TypeVar, _paramspec_tvars=False): super().__init__(origin, inst=inst, name=name) - if not isinstance(params, tuple): - params = (params,) + if not isinstance(args, tuple): + args = (args,) self.__args__ = tuple(... if a is _TypingEllipsis else () if a is _TypingEmpty else - a for a in params) - self.__parameters__ = _collect_type_vars(params, typevar_types=_typevar_types) + a for a in args) + self.__parameters__ = _collect_type_vars(args, typevar_types=_typevar_types) self._typevar_types = _typevar_types self._paramspec_tvars = _paramspec_tvars if not name: @@ -1031,44 +1056,97 @@ def __ror__(self, left): return Union[left, self] @_tp_cache - def __getitem__(self, params): + def __getitem__(self, args): + # Parameterizes an already-parameterized object. + # + # For example, we arrive here doing something like: + # T1 = TypeVar('T1') + # T2 = TypeVar('T2') + # T3 = TypeVar('T3') + # class A(Generic[T1]): pass + # B = A[T2] # B is a _GenericAlias + # C = B[T3] # Invokes _GenericAlias.__getitem__ + # + # We also arrive here when parameterizing a generic `Callable` alias: + # T = TypeVar('T') + # C = Callable[[T], None] + # C[int] # Invokes _GenericAlias.__getitem__ + if self.__origin__ in (Generic, Protocol): # Can't subscript Generic[...] or Protocol[...]. raise TypeError(f"Cannot subscript already-subscripted {self}") - if not isinstance(params, tuple): - params = (params,) - params = tuple(_type_convert(p) for p in params) + + # Preprocess `args`. + if not isinstance(args, tuple): + args = (args,) + args = tuple(_type_convert(p) for p in args) if (self._paramspec_tvars and any(isinstance(t, ParamSpec) for t in self.__parameters__)): - params = _prepare_paramspec_params(self, params) + args = _prepare_paramspec_params(self, args) else: - _check_generic(self, params, len(self.__parameters__)) + _check_generic(self, args, len(self.__parameters__)) + + new_args = self._determine_new_args(args) + r = self.copy_with(new_args) + return r + + def _determine_new_args(self, args): + # Determines new __args__ for __getitem__. + # + # For example, suppose we had: + # T1 = TypeVar('T1') + # T2 = TypeVar('T2') + # class A(Generic[T1, T2]): pass + # T3 = TypeVar('T3') + # B = A[int, T3] + # C = B[str] + # `B.__args__` is `(int, T3)`, so `C.__args__` should be `(int, str)`. + # Unfortunately, this is harder than it looks, because if `T3` is + # anything more exotic than a plain `TypeVar`, we need to consider + # edge cases. + + # In the example above, this would be {T3: str} + new_arg_by_param = dict(zip(self.__parameters__, args)) - subst = dict(zip(self.__parameters__, params)) new_args = [] - for arg in self.__args__: - if isinstance(arg, self._typevar_types): - if isinstance(arg, ParamSpec): - arg = subst[arg] - if not _is_param_expr(arg): - raise TypeError(f"Expected a list of types, an ellipsis, " - f"ParamSpec, or Concatenate. Got {arg}") + for old_arg in self.__args__: + + if isinstance(old_arg, ParamSpec): + new_arg = new_arg_by_param[old_arg] + if not _is_param_expr(new_arg): + raise TypeError(f"Expected a list of types, an ellipsis, " + f"ParamSpec, or Concatenate. Got {new_arg}") + elif isinstance(old_arg, self._typevar_types): + new_arg = new_arg_by_param[old_arg] + elif isinstance(old_arg, (_GenericAlias, GenericAlias, types.UnionType)): + subparams = old_arg.__parameters__ + if not subparams: + new_arg = old_arg else: - arg = subst[arg] - elif isinstance(arg, (_GenericAlias, GenericAlias, types.UnionType)): - subparams = arg.__parameters__ - if subparams: - subargs = tuple(subst[x] for x in subparams) - arg = arg[subargs] - # Required to flatten out the args for CallableGenericAlias - if self.__origin__ == collections.abc.Callable and isinstance(arg, tuple): - new_args.extend(arg) + subargs = tuple(new_arg_by_param[x] for x in subparams) + new_arg = old_arg[subargs] + else: + new_arg = old_arg + + if self.__origin__ == collections.abc.Callable and isinstance(new_arg, tuple): + # Consider the following `Callable`. + # C = Callable[[int], str] + # Here, `C.__args__` should be (int, str) - NOT ([int], str). + # That means that if we had something like... + # P = ParamSpec('P') + # T = TypeVar('T') + # C = Callable[P, T] + # D = C[[int, str], float] + # ...we need to be careful; `new_args` should end up as + # `(int, str, float)` rather than `([int, str], float)`. + new_args.extend(new_arg) else: - new_args.append(arg) - return self.copy_with(tuple(new_args)) + new_args.append(new_arg) - def copy_with(self, params): - return self.__class__(self.__origin__, params, name=self._name, inst=self._inst) + return tuple(new_args) + + def copy_with(self, args): + return self.__class__(self.__origin__, args, name=self._name, inst=self._inst) def __repr__(self): if self._name: From webhook-mailer at python.org Mon Feb 7 02:40:59 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 07 Feb 2022 07:40:59 -0000 Subject: [Python-checkins] bpo-40479: Fix undefined behavior in Modules/_hashopenssl.c (GH-31153) Message-ID: https://github.com/python/cpython/commit/59e004af63742361b67d1e1ae70229ff0db1059d commit: 59e004af63742361b67d1e1ae70229ff0db1059d branch: main author: Zackery Spytz committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-06T23:40:54-08:00 summary: bpo-40479: Fix undefined behavior in Modules/_hashopenssl.c (GH-31153) va_end() must be called before returning. files: A Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst M Modules/_hashopenssl.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst new file mode 100644 index 0000000000000..52701d53d8fe2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst @@ -0,0 +1 @@ +Add a missing call to ``va_end()`` in ``Modules/_hashopenssl.c``. diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c index fb155b2e62253..bb9487204e741 100644 --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -266,6 +266,7 @@ _setException(PyObject *exc, const char* altmsg, ...) } else { PyErr_FormatV(exc, altmsg, vargs); } + va_end(vargs); return NULL; } va_end(vargs); From webhook-mailer at python.org Mon Feb 7 03:05:40 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 07 Feb 2022 08:05:40 -0000 Subject: [Python-checkins] bpo-40479: Fix undefined behavior in Modules/_hashopenssl.c (GH-31153) Message-ID: https://github.com/python/cpython/commit/3ceff9968be059209f0d7271365f38895207b24d commit: 3ceff9968be059209f0d7271365f38895207b24d branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-07T00:05:29-08:00 summary: bpo-40479: Fix undefined behavior in Modules/_hashopenssl.c (GH-31153) va_end() must be called before returning. (cherry picked from commit 59e004af63742361b67d1e1ae70229ff0db1059d) Co-authored-by: Zackery Spytz files: A Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst M Modules/_hashopenssl.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst new file mode 100644 index 0000000000000..52701d53d8fe2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst @@ -0,0 +1 @@ +Add a missing call to ``va_end()`` in ``Modules/_hashopenssl.c``. diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c index 2eaa5f7d85d80..65538f63f7bf4 100644 --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -266,6 +266,7 @@ _setException(PyObject *exc, const char* altmsg, ...) } else { PyErr_FormatV(exc, altmsg, vargs); } + va_end(vargs); return NULL; } va_end(vargs); From webhook-mailer at python.org Mon Feb 7 03:08:14 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 07 Feb 2022 08:08:14 -0000 Subject: [Python-checkins] bpo-40479: Fix undefined behavior in Modules/_hashopenssl.c (GH-31153) Message-ID: https://github.com/python/cpython/commit/0892a0e3ec925ac959bc959b985293b4b96ef863 commit: 0892a0e3ec925ac959bc959b985293b4b96ef863 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-07T00:08:10-08:00 summary: bpo-40479: Fix undefined behavior in Modules/_hashopenssl.c (GH-31153) va_end() must be called before returning. (cherry picked from commit 59e004af63742361b67d1e1ae70229ff0db1059d) Co-authored-by: Zackery Spytz files: A Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst M Modules/_hashopenssl.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst new file mode 100644 index 0000000000000..52701d53d8fe2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-06-23-08-30.bpo-40479.zED3Zu.rst @@ -0,0 +1 @@ +Add a missing call to ``va_end()`` in ``Modules/_hashopenssl.c``. diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c index a488945082187..4873bb11aa0cd 100644 --- a/Modules/_hashopenssl.c +++ b/Modules/_hashopenssl.c @@ -311,6 +311,7 @@ _setException(PyObject *exc, const char* altmsg, ...) } else { PyErr_FormatV(exc, altmsg, vargs); } + va_end(vargs); return NULL; } va_end(vargs); From webhook-mailer at python.org Mon Feb 7 03:48:43 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Mon, 07 Feb 2022 08:48:43 -0000 Subject: [Python-checkins] [3.10] bpo-46611: add coverage to instance and class checks in `typing.py` (GH-31078) (GH-31182) Message-ID: https://github.com/python/cpython/commit/c1ff4cb98b11c00aee765019364544c71e7dd2df commit: c1ff4cb98b11c00aee765019364544c71e7dd2df branch: 3.10 author: Nikita Sobolev committer: serhiy-storchaka date: 2022-02-07T10:48:20+02:00 summary: [3.10] bpo-46611: add coverage to instance and class checks in `typing.py` (GH-31078) (GH-31182) (cherry picked from commit 067c03bf40d13393209f0138fa9c4d5980c4ff8a) Co-authored-by: Nikita Sobolev files: M Lib/test/test_types.py M Lib/test/test_typing.py diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py index 3dfda5cb95663..725d80d9ffdcf 100644 --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -695,22 +695,62 @@ def test_hash(self): self.assertEqual(hash(int | str), hash(str | int)) self.assertEqual(hash(int | str), hash(typing.Union[int, str])) - def test_instancecheck(self): - x = int | str - self.assertIsInstance(1, x) - self.assertIsInstance(True, x) - self.assertIsInstance('a', x) - self.assertNotIsInstance(None, x) - self.assertTrue(issubclass(int, x)) - self.assertTrue(issubclass(bool, x)) - self.assertTrue(issubclass(str, x)) - self.assertFalse(issubclass(type(None), x)) - x = int | None - self.assertIsInstance(None, x) - self.assertTrue(issubclass(type(None), x)) - x = int | collections.abc.Mapping - self.assertIsInstance({}, x) - self.assertTrue(issubclass(dict, x)) + def test_instancecheck_and_subclasscheck(self): + for x in (int | str, typing.Union[int, str]): + with self.subTest(x=x): + self.assertIsInstance(1, x) + self.assertIsInstance(True, x) + self.assertIsInstance('a', x) + self.assertNotIsInstance(None, x) + self.assertTrue(issubclass(int, x)) + self.assertTrue(issubclass(bool, x)) + self.assertTrue(issubclass(str, x)) + self.assertFalse(issubclass(type(None), x)) + + for x in (int | None, typing.Union[int, None]): + with self.subTest(x=x): + self.assertIsInstance(None, x) + self.assertTrue(issubclass(type(None), x)) + + for x in ( + int | collections.abc.Mapping, + typing.Union[int, collections.abc.Mapping], + ): + with self.subTest(x=x): + self.assertIsInstance({}, x) + self.assertNotIsInstance((), x) + self.assertTrue(issubclass(dict, x)) + self.assertFalse(issubclass(list, x)) + + def test_instancecheck_and_subclasscheck_order(self): + T = typing.TypeVar('T') + + will_resolve = ( + int | T, + typing.Union[int, T], + ) + for x in will_resolve: + with self.subTest(x=x): + self.assertIsInstance(1, x) + self.assertTrue(issubclass(int, x)) + + wont_resolve = ( + T | int, + typing.Union[T, int], + ) + for x in wont_resolve: + with self.subTest(x=x): + with self.assertRaises(TypeError): + issubclass(int, x) + with self.assertRaises(TypeError): + isinstance(1, x) + + for x in (*will_resolve, *wont_resolve): + with self.subTest(x=x): + with self.assertRaises(TypeError): + issubclass(object, x) + with self.assertRaises(TypeError): + isinstance(object(), x) def test_bad_instancecheck(self): class BadMeta(type): diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 895805fb2a260..f13541e10d58c 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -431,6 +431,8 @@ def test_tuple_subclass(self): class MyTuple(tuple): pass self.assertIsSubclass(MyTuple, Tuple) + self.assertIsSubclass(Tuple, Tuple) + self.assertIsSubclass(tuple, Tuple) def test_tuple_instance_type_error(self): with self.assertRaises(TypeError): @@ -458,6 +460,7 @@ def test_self_subclass(self): with self.assertRaises(TypeError): issubclass(types.FunctionType, Callable[[int], int]) self.assertIsSubclass(types.FunctionType, Callable) + self.assertIsSubclass(Callable, Callable) def test_eq_hash(self): Callable = self.Callable From webhook-mailer at python.org Mon Feb 7 07:48:53 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 12:48:53 -0000 Subject: [Python-checkins] bpo-46648: Rewrite test_urllib2.test_issue16464() with a local HTTP server (GH-31186) Message-ID: https://github.com/python/cpython/commit/8e98175a03fe03d62822d96007a74e5273013764 commit: 8e98175a03fe03d62822d96007a74e5273013764 branch: main author: Nikita Sobolev committer: vstinner date: 2022-02-07T13:48:43+01:00 summary: bpo-46648: Rewrite test_urllib2.test_issue16464() with a local HTTP server (GH-31186) Re-enable test_issue16464() of test_urllib2, move it to urllib2_localnet and use the local HTTP server rather than an external HTTP server. files: M Lib/test/test_urllib2.py M Lib/test/test_urllib2_localnet.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 0936e435a4870..2c93fcada4387 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1788,24 +1788,6 @@ class MyOtherHTTPHandler(urllib.request.HTTPHandler): self.opener_has_handler(o, MyHTTPHandler) self.opener_has_handler(o, MyOtherHTTPHandler) - @unittest.skipUnless(support.is_resource_enabled('network'), - 'test requires network access') - # bpo-46648: test fails randomly with "http://www.example.com/" URL - @unittest.skipIf(True, "POST request to http://www.example.com/ fail randomly") - def test_issue16464(self): - with socket_helper.transient_internet("http://www.example.com/"): - opener = urllib.request.build_opener() - request = urllib.request.Request("http://www.example.com/") - self.assertEqual(None, request.data) - - opener.open(request, "1".encode("us-ascii")) - self.assertEqual(b"1", request.data) - self.assertEqual("1", request.get_header("Content-length")) - - opener.open(request, "1234567890".encode("us-ascii")) - self.assertEqual(b"1234567890", request.data) - self.assertEqual("10", request.get_header("Content-length")) - def test_HTTPError_interface(self): """ Issue 13211 reveals that HTTPError didn't implement the URLError diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py index 0b2d07ce61d5c..1b2baf2f366b5 100644 --- a/Lib/test/test_urllib2_localnet.py +++ b/Lib/test/test_urllib2_localnet.py @@ -660,6 +660,24 @@ def test_line_iteration(self): (index, len(lines[index]), len(line))) self.assertEqual(index + 1, len(lines)) + def test_issue16464(self): + # See https://bugs.python.org/issue16464 + # and https://bugs.python.org/issue46648 + handler = self.start_server([ + (200, [], b'any'), + (200, [], b'any'), + ]) + opener = urllib.request.build_opener() + request = urllib.request.Request("http://localhost:%s" % handler.port) + self.assertEqual(None, request.data) + + opener.open(request, "1".encode("us-ascii")) + self.assertEqual(b"1", request.data) + self.assertEqual("1", request.get_header("Content-length")) + + opener.open(request, "1234567890".encode("us-ascii")) + self.assertEqual(b"1234567890", request.data) + self.assertEqual("10", request.get_header("Content-length")) def setUpModule(): thread_info = threading_helper.threading_setup() From webhook-mailer at python.org Mon Feb 7 08:14:19 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 07 Feb 2022 13:14:19 -0000 Subject: [Python-checkins] bpo-46648: Rewrite test_urllib2.test_issue16464() with a local HTTP server (GH-31186) Message-ID: https://github.com/python/cpython/commit/0d74efc54fa811def386d2cde00986204ba18569 commit: 0d74efc54fa811def386d2cde00986204ba18569 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-07T05:14:06-08:00 summary: bpo-46648: Rewrite test_urllib2.test_issue16464() with a local HTTP server (GH-31186) Re-enable test_issue16464() of test_urllib2, move it to urllib2_localnet and use the local HTTP server rather than an external HTTP server. (cherry picked from commit 8e98175a03fe03d62822d96007a74e5273013764) Co-authored-by: Nikita Sobolev files: M Lib/test/test_urllib2.py M Lib/test/test_urllib2_localnet.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index d8aab53089bc4..648f65d6257d7 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1786,24 +1786,6 @@ class MyOtherHTTPHandler(urllib.request.HTTPHandler): self.opener_has_handler(o, MyHTTPHandler) self.opener_has_handler(o, MyOtherHTTPHandler) - @unittest.skipUnless(support.is_resource_enabled('network'), - 'test requires network access') - # bpo-46648: test fails randomly with "http://www.example.com/" URL - @unittest.skipIf(True, "POST request to http://www.example.com/ fail randomly") - def test_issue16464(self): - with socket_helper.transient_internet("http://www.example.com/"): - opener = urllib.request.build_opener() - request = urllib.request.Request("http://www.example.com/") - self.assertEqual(None, request.data) - - opener.open(request, "1".encode("us-ascii")) - self.assertEqual(b"1", request.data) - self.assertEqual("1", request.get_header("Content-length")) - - opener.open(request, "1234567890".encode("us-ascii")) - self.assertEqual(b"1234567890", request.data) - self.assertEqual("10", request.get_header("Content-length")) - def test_HTTPError_interface(self): """ Issue 13211 reveals that HTTPError didn't implement the URLError diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py index 7c716341c7023..74374c08684da 100644 --- a/Lib/test/test_urllib2_localnet.py +++ b/Lib/test/test_urllib2_localnet.py @@ -659,6 +659,24 @@ def test_line_iteration(self): (index, len(lines[index]), len(line))) self.assertEqual(index + 1, len(lines)) + def test_issue16464(self): + # See https://bugs.python.org/issue16464 + # and https://bugs.python.org/issue46648 + handler = self.start_server([ + (200, [], b'any'), + (200, [], b'any'), + ]) + opener = urllib.request.build_opener() + request = urllib.request.Request("http://localhost:%s" % handler.port) + self.assertEqual(None, request.data) + + opener.open(request, "1".encode("us-ascii")) + self.assertEqual(b"1", request.data) + self.assertEqual("1", request.get_header("Content-length")) + + opener.open(request, "1234567890".encode("us-ascii")) + self.assertEqual(b"1234567890", request.data) + self.assertEqual("10", request.get_header("Content-length")) def setUpModule(): thread_info = support.threading_setup() From webhook-mailer at python.org Mon Feb 7 08:53:24 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 13:53:24 -0000 Subject: [Python-checkins] bpo-46323: _ctypes.CFuncPtr fails if _argtypes_ is too long (GH-31188) Message-ID: https://github.com/python/cpython/commit/4cce1352bb47babaeefb68fcfcc48ffa073745c3 commit: 4cce1352bb47babaeefb68fcfcc48ffa073745c3 branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T14:53:15+01:00 summary: bpo-46323: _ctypes.CFuncPtr fails if _argtypes_ is too long (GH-31188) ctypes.CFUNCTYPE() and ctypes.WINFUNCTYPE() now fail to create the type if its "_argtypes_" member contains too many arguments. Previously, the error was only raised when calling a function. Change also how CFUNCTYPE() and WINFUNCTYPE() handle KeyError to prevent creating a chain of exceptions if ctypes.CFuncPtr raises an error. files: A Misc/NEWS.d/next/Library/2022-02-07-13-27-59.bpo-46323.7UENAj.rst M Lib/ctypes/__init__.py M Lib/ctypes/test/test_callbacks.py M Modules/_ctypes/_ctypes.c M Modules/_ctypes/callproc.c M Modules/_ctypes/ctypes.h diff --git a/Lib/ctypes/__init__.py b/Lib/ctypes/__init__.py index b08629e8df4df..ab4d31b0acb00 100644 --- a/Lib/ctypes/__init__.py +++ b/Lib/ctypes/__init__.py @@ -92,15 +92,18 @@ def CFUNCTYPE(restype, *argtypes, **kw): flags |= _FUNCFLAG_USE_LASTERROR if kw: raise ValueError("unexpected keyword argument(s) %s" % kw.keys()) + try: return _c_functype_cache[(restype, argtypes, flags)] except KeyError: - class CFunctionType(_CFuncPtr): - _argtypes_ = argtypes - _restype_ = restype - _flags_ = flags - _c_functype_cache[(restype, argtypes, flags)] = CFunctionType - return CFunctionType + pass + + class CFunctionType(_CFuncPtr): + _argtypes_ = argtypes + _restype_ = restype + _flags_ = flags + _c_functype_cache[(restype, argtypes, flags)] = CFunctionType + return CFunctionType if _os.name == "nt": from _ctypes import LoadLibrary as _dlopen @@ -116,15 +119,18 @@ def WINFUNCTYPE(restype, *argtypes, **kw): flags |= _FUNCFLAG_USE_LASTERROR if kw: raise ValueError("unexpected keyword argument(s) %s" % kw.keys()) + try: return _win_functype_cache[(restype, argtypes, flags)] except KeyError: - class WinFunctionType(_CFuncPtr): - _argtypes_ = argtypes - _restype_ = restype - _flags_ = flags - _win_functype_cache[(restype, argtypes, flags)] = WinFunctionType - return WinFunctionType + pass + + class WinFunctionType(_CFuncPtr): + _argtypes_ = argtypes + _restype_ = restype + _flags_ = flags + _win_functype_cache[(restype, argtypes, flags)] = WinFunctionType + return WinFunctionType if WINFUNCTYPE.__doc__: WINFUNCTYPE.__doc__ = CFUNCTYPE.__doc__.replace("CFUNCTYPE", "WINFUNCTYPE") diff --git a/Lib/ctypes/test/test_callbacks.py b/Lib/ctypes/test/test_callbacks.py index d8e9c5a760e2c..5561ffefe12f7 100644 --- a/Lib/ctypes/test/test_callbacks.py +++ b/Lib/ctypes/test/test_callbacks.py @@ -294,15 +294,22 @@ def func(*args): return len(args) CTYPES_MAX_ARGCOUNT = 1024 + + # valid call with nargs <= CTYPES_MAX_ARGCOUNT proto = CFUNCTYPE(c_int, *(c_int,) * CTYPES_MAX_ARGCOUNT) cb = proto(func) args1 = (1,) * CTYPES_MAX_ARGCOUNT self.assertEqual(cb(*args1), CTYPES_MAX_ARGCOUNT) + # invalid call with nargs > CTYPES_MAX_ARGCOUNT args2 = (1,) * (CTYPES_MAX_ARGCOUNT + 1) with self.assertRaises(ArgumentError): cb(*args2) + # error when creating the type with too many arguments + with self.assertRaises(ArgumentError): + CFUNCTYPE(c_int, *(c_int,) * (CTYPES_MAX_ARGCOUNT + 1)) + def test_convert_result_error(self): def func(): return ("tuple",) diff --git a/Misc/NEWS.d/next/Library/2022-02-07-13-27-59.bpo-46323.7UENAj.rst b/Misc/NEWS.d/next/Library/2022-02-07-13-27-59.bpo-46323.7UENAj.rst new file mode 100644 index 0000000000000..e144450f2527a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-07-13-27-59.bpo-46323.7UENAj.rst @@ -0,0 +1,3 @@ +``ctypes.CFUNCTYPE()`` and ``ctypes.WINFUNCTYPE()`` now fail to create the type +if its ``_argtypes_`` member contains too many arguments. Previously, the error +was only raised when calling a function. Patch by Victor Stinner. diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index 96078c7726d59..da9dd096c73d8 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -2382,7 +2382,6 @@ converters_from_argtypes(PyObject *ob) _Py_IDENTIFIER(from_param); PyObject *converters; Py_ssize_t i; - Py_ssize_t nArgs; ob = PySequence_Tuple(ob); /* new reference */ if (!ob) { @@ -2391,7 +2390,14 @@ converters_from_argtypes(PyObject *ob) return NULL; } - nArgs = PyTuple_GET_SIZE(ob); + Py_ssize_t nArgs = PyTuple_GET_SIZE(ob); + if (nArgs > CTYPES_MAX_ARGCOUNT) { + PyErr_Format(PyExc_ArgError, + "_argtypes_ has too many arguments (%zi), maximum is %i", + nArgs, CTYPES_MAX_ARGCOUNT); + return NULL; + } + converters = PyTuple_New(nArgs); if (!converters) { Py_DECREF(ob); diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index 928737edb09b1..da2956788a1dd 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -1118,14 +1118,6 @@ GetComError(HRESULT errcode, GUID *riid, IUnknown *pIunk) #define IS_PASS_BY_REF(x) (x > 8 || !POW2(x)) #endif -/* - * bpo-13097: Max number of arguments _ctypes_callproc will accept. - * - * This limit is enforced for the `alloca()` call in `_ctypes_callproc`, - * to avoid allocating a massive buffer on the stack. - */ -#define CTYPES_MAX_ARGCOUNT 1024 - /* * Requirements, must be ensured by the caller: * - argtuple is tuple of arguments diff --git a/Modules/_ctypes/ctypes.h b/Modules/_ctypes/ctypes.h index 9e82ce8574714..0badb48b2a410 100644 --- a/Modules/_ctypes/ctypes.h +++ b/Modules/_ctypes/ctypes.h @@ -11,6 +11,15 @@ #define PARAMFLAG_FLCID 0x4 #endif +/* + * bpo-13097: Max number of arguments CFuncPtr._argtypes_ and + * _ctypes_callproc() will accept. + * + * This limit is enforced for the `alloca()` call in `_ctypes_callproc`, + * to avoid allocating a massive buffer on the stack. + */ +#define CTYPES_MAX_ARGCOUNT 1024 + typedef struct tagPyCArgObject PyCArgObject; typedef struct tagCDataObject CDataObject; typedef PyObject *(* GETFUNC)(void *, Py_ssize_t size); From webhook-mailer at python.org Mon Feb 7 09:31:01 2022 From: webhook-mailer at python.org (markshannon) Date: Mon, 07 Feb 2022 14:31:01 -0000 Subject: [Python-checkins] bpo-46072: Improve LOAD_METHOD stats (GH-31104) Message-ID: https://github.com/python/cpython/commit/062460e8fd54e53c9a1a6f175ef49c9d730851b8 commit: 062460e8fd54e53c9a1a6f175ef49c9d730851b8 branch: main author: Mark Shannon committer: markshannon date: 2022-02-07T14:30:34Z summary: bpo-46072: Improve LOAD_METHOD stats (GH-31104) files: M Python/specialize.c diff --git a/Python/specialize.c b/Python/specialize.c index b7ef478ee5590..b95bdf73e8bf6 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -503,6 +503,10 @@ initial_counter_value(void) { #define SPEC_FAIL_BUILTIN_CLASS_METHOD 17 #define SPEC_FAIL_CLASS_METHOD_OBJ 18 #define SPEC_FAIL_OBJECT_SLOT 19 +#define SPEC_FAIL_HAS_DICT 20 +#define SPEC_FAIL_HAS_MANAGED_DICT 21 +#define SPEC_FAIL_INSTANCE_ATTRIBUTE 22 +#define SPEC_FAIL_METACLASS_ATTRIBUTE 23 /* Binary subscr */ @@ -973,7 +977,7 @@ load_method_fail_kind(DescriptorClassification kind) case NON_DESCRIPTOR: return SPEC_FAIL_NOT_DESCRIPTOR; case ABSENT: - return SPEC_FAIL_EXPECTED_ERROR; + return SPEC_FAIL_INSTANCE_ATTRIBUTE; } Py_UNREACHABLE(); } @@ -994,6 +998,16 @@ specialize_class_load_method(PyObject *owner, _Py_CODEUNIT *instr, PyObject *nam cache2->obj = descr; *instr = _Py_MAKECODEUNIT(LOAD_METHOD_CLASS, _Py_OPARG(*instr)); return 0; +#ifdef Py_STATS + case ABSENT: + if (_PyType_Lookup(Py_TYPE(owner), name) != NULL) { + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_METACLASS_ATTRIBUTE); + } + else { + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_EXPECTED_ERROR); + } + return -1; +#endif default: SPECIALIZATION_FAIL(LOAD_METHOD, load_method_fail_kind(kind)); return -1; @@ -1043,7 +1057,7 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, if (owner_cls->tp_flags & Py_TPFLAGS_MANAGED_DICT) { PyObject **owner_dictptr = _PyObject_ManagedDictPointer(owner); if (*owner_dictptr) { - SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_IS_ATTR); + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_HAS_MANAGED_DICT); goto fail; } PyDictKeysObject *keys = ((PyHeapTypeObject *)owner_cls)->ht_cached_keys; @@ -1065,7 +1079,7 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, *instr = _Py_MAKECODEUNIT(LOAD_METHOD_NO_DICT, _Py_OPARG(*instr)); } else { - SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_IS_ATTR); + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_HAS_DICT); goto fail; } } From webhook-mailer at python.org Mon Feb 7 10:06:42 2022 From: webhook-mailer at python.org (markshannon) Date: Mon, 07 Feb 2022 15:06:42 -0000 Subject: [Python-checkins] bpo-46670: Remove unused macros in ceval.c (GH-31196) Message-ID: https://github.com/python/cpython/commit/4b603f628207b380a8a2f22d7ff5d2dbb0853e2e commit: 4b603f628207b380a8a2f22d7ff5d2dbb0853e2e branch: main author: Victor Stinner committer: markshannon date: 2022-02-07T15:06:27Z summary: bpo-46670: Remove unused macros in ceval.c (GH-31196) Remove CHECKEXC() and EXT_POP() macros. files: M Python/ceval.c diff --git a/Python/ceval.c b/Python/ceval.c index e6b5d3ae24237..31b41b848486a 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -40,9 +40,8 @@ #include #ifdef Py_DEBUG -/* For debugging the interpreter: */ -#define LLTRACE 1 /* Low-level trace feature */ -#define CHECKEXC 1 /* Double-check exception checking */ + /* For debugging the interpreter: */ +# define LLTRACE 1 /* Low-level trace feature */ #endif #if !defined(Py_BUILD_CORE) @@ -1456,15 +1455,11 @@ eval_frame_handle_pending(PyThreadState *tstate) (void)(BASIC_STACKADJ(-(n))); \ assert(STACK_LEVEL() <= frame->f_code->co_stacksize); \ } while (0) -#define EXT_POP(STACK_POINTER) ((void)(lltrace && \ - prtrace(tstate, (STACK_POINTER)[-1], "ext_pop")), \ - *--(STACK_POINTER)) #else #define PUSH(v) BASIC_PUSH(v) #define POP() BASIC_POP() #define STACK_GROW(n) BASIC_STACKADJ(n) #define STACK_SHRINK(n) BASIC_STACKADJ(-(n)) -#define EXT_POP(STACK_POINTER) (*--(STACK_POINTER)) #endif /* Local variable macros */ From webhook-mailer at python.org Mon Feb 7 10:21:14 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 15:21:14 -0000 Subject: [Python-checkins] bpo-39277: Fix PY_TIMEOUT_MAX cast in _threadmodule.c (GH-31195) Message-ID: https://github.com/python/cpython/commit/d3e53bc5321a9f08c7ed5b9383eefb2e03dfa6e2 commit: d3e53bc5321a9f08c7ed5b9383eefb2e03dfa6e2 branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T16:21:09+01:00 summary: bpo-39277: Fix PY_TIMEOUT_MAX cast in _threadmodule.c (GH-31195) Cast PY_TIMEOUT_MAX to double, not to _PyTime_t. Fix the clang warning: Modules/_threadmodule.c:1648:26: warning: implicit conversion from '_PyTime_t' (aka 'long') to 'double' changes value from 9223372036854775 to 9223372036854776 [-Wimplicit-const-int-float-conversion] double timeout_max = (_PyTime_t)PY_TIMEOUT_MAX * 1e-6; ^~~~~~~~~~~~~~~~~~~~~~~~~ ~ files: M Modules/_threadmodule.c diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index 9e6e462b59e06..7052d4c887a66 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -1645,7 +1645,7 @@ thread_module_exec(PyObject *module) } // TIMEOUT_MAX - double timeout_max = (_PyTime_t)PY_TIMEOUT_MAX * 1e-6; + double timeout_max = (double)PY_TIMEOUT_MAX * 1e-6; double time_max = _PyTime_AsSecondsDouble(_PyTime_MAX); timeout_max = Py_MIN(timeout_max, time_max); // Round towards minus infinity From webhook-mailer at python.org Mon Feb 7 10:21:36 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 15:21:36 -0000 Subject: [Python-checkins] bpo-46670: Remove unused macros in the Modules directory (GH-31194) Message-ID: https://github.com/python/cpython/commit/4f1d3f33dd8b37a151b73533f18b08b0500223e5 commit: 4f1d3f33dd8b37a151b73533f18b08b0500223e5 branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T16:21:31+01:00 summary: bpo-46670: Remove unused macros in the Modules directory (GH-31194) * bpo-46670: Remove unused macros in the Modules directory * Add again LINKAT_DIR_FD_CONVERTER: generated by Argument Clinic files: M Modules/_ctypes/stgdict.c M Modules/_ssl.c M Modules/_struct.c M Modules/_testcapimodule.c M Modules/_tkinter.c M Modules/pwdmodule.c diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c index 6c1917b18d450..3470cf114db4f 100644 --- a/Modules/_ctypes/stgdict.c +++ b/Modules/_ctypes/stgdict.c @@ -504,7 +504,6 @@ PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct if (stgdict->format == NULL) return -1; -#define realdict ((PyObject *)&stgdict->dict) for (i = 0; i < len; ++i) { PyObject *name = NULL, *desc = NULL; PyObject *pair = PySequence_GetItem(fields, i); @@ -635,7 +634,6 @@ PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct Py_DECREF(pair); Py_DECREF(prop); } -#undef realdict if (isStruct && !isPacked) { char *ptr = stgdict->format; diff --git a/Modules/_ssl.c b/Modules/_ssl.c index b2e241a0a338e..e17876e49a8be 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -40,8 +40,6 @@ #define PySSL_BEGIN_ALLOW_THREADS { \ PyThreadState *_save = NULL; \ PySSL_BEGIN_ALLOW_THREADS_S(_save); -#define PySSL_BLOCK_THREADS PySSL_END_ALLOW_THREADS_S(_save); -#define PySSL_UNBLOCK_THREADS PySSL_BEGIN_ALLOW_THREADS_S(_save); #define PySSL_END_ALLOW_THREADS PySSL_END_ALLOW_THREADS_S(_save); } diff --git a/Modules/_struct.c b/Modules/_struct.c index 210dbdc752f4c..7d05ec5ac7c50 100644 --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -2065,8 +2065,6 @@ static PyMemberDef s_members[] = { {NULL} /* sentinel */ }; -#define OFF(x) offsetof(PyStructObject, x) - static PyGetSetDef s_getsetlist[] = { {"format", (getter)s_get_format, (setter)NULL, "struct format string", NULL}, {"size", (getter)s_get_size, (setter)NULL, "struct size in bytes", NULL}, diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 7369f094faedd..498a7c2fd686b 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -6060,8 +6060,6 @@ static PyMethodDef TestMethods[] = { {NULL, NULL} /* sentinel */ }; -#define AddSym(d, n, f, v) {PyObject *o = f(v); PyDict_SetItemString(d, n, o); Py_DECREF(o);} - typedef struct { char bool_member; char byte_member; diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c index f4d2716fe302d..cbef9ce0066db 100644 --- a/Modules/_tkinter.c +++ b/Modules/_tkinter.c @@ -318,10 +318,6 @@ typedef struct { #define Tkapp_Interp(v) (((TkappObject *) (v))->interp) -#define DEBUG_REFCNT(v) (printf("DEBUG: id=%p, refcnt=%i\n", \ -(void *) v, Py_REFCNT(v))) - - /**** Error Handling ****/ diff --git a/Modules/pwdmodule.c b/Modules/pwdmodule.c index 14d3f9dcb1c60..a757380bd09f7 100644 --- a/Modules/pwdmodule.c +++ b/Modules/pwdmodule.c @@ -84,7 +84,6 @@ mkpwent(PyObject *module, struct passwd *p) if (v == NULL) return NULL; -#define SETI(i,val) PyStructSequence_SET_ITEM(v, i, PyLong_FromLong((long) val)) #define SETS(i,val) sets(v, i, val) SETS(setIndex++, p->pw_name); @@ -104,7 +103,6 @@ mkpwent(PyObject *module, struct passwd *p) SETS(setIndex++, p->pw_shell); #undef SETS -#undef SETI if (PyErr_Occurred()) { Py_XDECREF(v); From webhook-mailer at python.org Mon Feb 7 10:21:48 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 15:21:48 -0000 Subject: [Python-checkins] bpo-46670: Remove unused macros in the Objects directory (GH-31193) Message-ID: https://github.com/python/cpython/commit/760349198dbd8771629753e096a885c1aa28a1ca commit: 760349198dbd8771629753e096a885c1aa28a1ca branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T16:21:41+01:00 summary: bpo-46670: Remove unused macros in the Objects directory (GH-31193) files: M Objects/dictobject.c M Objects/memoryobject.c M Objects/obmalloc.c M Objects/unicodectype.c diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 0ad0f0b59c87e..2c8ec79ded72c 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -292,7 +292,6 @@ _PyDict_DebugMallocStats(FILE *out) } #define DK_MASK(dk) (DK_SIZE(dk)-1) -#define IS_POWER_OF_2(x) (((x) & (x-1)) == 0) static void free_keys_object(PyDictKeysObject *keys); diff --git a/Objects/memoryobject.c b/Objects/memoryobject.c index dfcb31e5b3307..45fe8985c2adb 100644 --- a/Objects/memoryobject.c +++ b/Objects/memoryobject.c @@ -67,14 +67,6 @@ class memoryview "PyMemoryViewObject *" "&PyMemoryView_Type" */ -#define CHECK_MBUF_RELEASED(mbuf) \ - if (((_PyManagedBufferObject *)mbuf)->flags&_Py_MANAGED_BUFFER_RELEASED) { \ - PyErr_SetString(PyExc_ValueError, \ - "operation forbidden on released memoryview object"); \ - return NULL; \ - } - - static inline _PyManagedBufferObject * mbuf_alloc(void) { diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index bad4dc0963921..560e1c59a9c58 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -900,7 +900,6 @@ static int running_on_valgrind = -1; * currently targets. */ #define SYSTEM_PAGE_SIZE (4 * 1024) -#define SYSTEM_PAGE_SIZE_MASK (SYSTEM_PAGE_SIZE - 1) /* * Maximum amount of memory managed by the allocator for small requests. diff --git a/Objects/unicodectype.c b/Objects/unicodectype.c index d8c95c8b44cea..aa5c5b2a4ad2e 100644 --- a/Objects/unicodectype.c +++ b/Objects/unicodectype.c @@ -14,8 +14,6 @@ #define DECIMAL_MASK 0x02 #define DIGIT_MASK 0x04 #define LOWER_MASK 0x08 -#define LINEBREAK_MASK 0x10 -#define SPACE_MASK 0x20 #define TITLE_MASK 0x40 #define UPPER_MASK 0x80 #define XID_START_MASK 0x100 From webhook-mailer at python.org Mon Feb 7 10:21:57 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 15:21:57 -0000 Subject: [Python-checkins] bpo-46670: Remove unused macros in the Python directory (GH-31192) Message-ID: https://github.com/python/cpython/commit/7d8b69e1d1f125454d8cec81ff0dee72f2bef957 commit: 7d8b69e1d1f125454d8cec81ff0dee72f2bef957 branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T16:21:52+01:00 summary: bpo-46670: Remove unused macros in the Python directory (GH-31192) files: M Python/ast_opt.c M Python/compile.c M Python/import.c M Python/thread.c diff --git a/Python/ast_opt.c b/Python/ast_opt.c index 2821e69a35521..967e75d5771c5 100644 --- a/Python/ast_opt.c +++ b/Python/ast_opt.c @@ -660,15 +660,6 @@ static int astfold_pattern(pattern_ty node_, PyArena *ctx_, _PyASTOptimizeState } \ } -#define CALL_INT_SEQ(FUNC, TYPE, ARG) { \ - int i; \ - asdl_int_seq *seq = (ARG); /* avoid variable capture */ \ - for (i = 0; i < asdl_seq_LEN(seq); i++) { \ - TYPE elt = (TYPE)asdl_seq_GET(seq, i); \ - if (!FUNC(elt, ctx_, state)) \ - return 0; \ - } \ -} static int astfold_body(asdl_stmt_seq *stmts, PyArena *ctx_, _PyASTOptimizeState *state) @@ -1085,7 +1076,6 @@ astfold_match_case(match_case_ty node_, PyArena *ctx_, _PyASTOptimizeState *stat #undef CALL #undef CALL_OPT #undef CALL_SEQ -#undef CALL_INT_SEQ /* See comments in symtable.c. */ #define COMPILER_STACK_FRAME_SCALE 3 diff --git a/Python/compile.c b/Python/compile.c index cecd6a5c42660..3f33f306bc571 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -37,7 +37,6 @@ #define DEFAULT_BLOCK_SIZE 16 -#define DEFAULT_BLOCKS 8 #define DEFAULT_CODE_SIZE 128 #define DEFAULT_LNOTAB_SIZE 16 #define DEFAULT_CNOTAB_SIZE 32 @@ -1472,13 +1471,6 @@ compiler_addop_j_noline(struct compiler *c, int opcode, basicblock *b) Py_DECREF(__new_const); \ } -#define ADDOP_O(C, OP, O, TYPE) { \ - assert(!HAS_CONST(OP)); /* use ADDOP_LOAD_CONST */ \ - if (!compiler_addop_o((C), (OP), (C)->u->u_ ## TYPE, (O))) \ - return 0; \ -} - -/* Same as ADDOP_O, but steals a reference. */ #define ADDOP_N(C, OP, O, TYPE) { \ assert(!HAS_CONST(OP)); /* use ADDOP_LOAD_CONST_NEW */ \ if (!compiler_addop_o((C), (OP), (C)->u->u_ ## TYPE, (O))) { \ @@ -1552,11 +1544,6 @@ compiler_addop_j_noline(struct compiler *c, int opcode, basicblock *b) } \ } -#define VISIT_SLICE(C, V, CTX) {\ - if (!compiler_visit_slice((C), (V), (CTX))) \ - return 0; \ -} - #define VISIT_SEQ(C, TYPE, SEQ) { \ int _i; \ asdl_ ## TYPE ## _seq *seq = (SEQ); /* avoid variable capture */ \ diff --git a/Python/import.c b/Python/import.c index be60c431f7ef8..332db6919f1c2 100644 --- a/Python/import.c +++ b/Python/import.c @@ -24,8 +24,6 @@ extern "C" { #endif -#define CACHEDIR "__pycache__" - /* Forward references */ static PyObject *import_add_module(PyThreadState *tstate, PyObject *name); diff --git a/Python/thread.c b/Python/thread.c index c6b16251a05b6..e80e8a906bc8e 100644 --- a/Python/thread.c +++ b/Python/thread.c @@ -45,11 +45,9 @@ #ifdef Py_DEBUG static int thread_debug = 0; -#define dprintf(args) (void)((thread_debug & 1) && printf args) -#define d2printf(args) ((thread_debug & 8) && printf args) +# define dprintf(args) (void)((thread_debug & 1) && printf args) #else -#define dprintf(args) -#define d2printf(args) +# define dprintf(args) #endif static int initialized; From webhook-mailer at python.org Mon Feb 7 10:22:33 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 15:22:33 -0000 Subject: [Python-checkins] bpo-46670: Fix #ifdef in sha3module.c (GH-31180) Message-ID: https://github.com/python/cpython/commit/f20ca766fe404a20daea29230f161a0eb71bb489 commit: f20ca766fe404a20daea29230f161a0eb71bb489 branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T16:22:24+01:00 summary: bpo-46670: Fix #ifdef in sha3module.c (GH-31180) * Test if HAVE_ALIGNED_REQUIRED is defined, not its value. * Define explicitly NOT_PYTHON macro to 0. Fix "gcc -Wundef" warnings. files: M Modules/_sha3/sha3module.c diff --git a/Modules/_sha3/sha3module.c b/Modules/_sha3/sha3module.c index bfa96616df0c9..bffd177c0e753 100644 --- a/Modules/_sha3/sha3module.c +++ b/Modules/_sha3/sha3module.c @@ -57,6 +57,8 @@ typedef uint64_t UINT64; typedef unsigned char UINT8; #endif +// kcp/KeccakP-1600-opt64.c doesn't need to define UINT8 +#define NOT_PYTHON 0 /* replacement for brg_endian.h */ #define IS_LITTLE_ENDIAN 1234 @@ -69,7 +71,7 @@ #endif /* Prevent bus errors on platforms requiring aligned accesses such ARM. */ -#if HAVE_ALIGNED_REQUIRED && !defined(NO_MISALIGNED_ACCESSES) +#if defined(HAVE_ALIGNED_REQUIRED) && !defined(NO_MISALIGNED_ACCESSES) #define NO_MISALIGNED_ACCESSES #endif From webhook-mailer at python.org Mon Feb 7 11:22:05 2022 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 07 Feb 2022 16:22:05 -0000 Subject: [Python-checkins] [3.10] bpo-46655: allow stringized TypeAlias with get_type_hints (GH-31156). (#31175) Message-ID: https://github.com/python/cpython/commit/e2eeffefed32bb8c47c09bdd94e27a4e949894ef commit: e2eeffefed32bb8c47c09bdd94e27a4e949894ef branch: 3.10 author: Gregory Beauregard committer: gvanrossum date: 2022-02-07T08:21:56-08:00 summary: [3.10] bpo-46655: allow stringized TypeAlias with get_type_hints (GH-31156). (#31175) (cherry picked from commit 77b025be4a4cd5a3bfc1b1af560cc57e8e956c98) Co-authored-by: Gregory Beauregard files: A Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index f13541e10d58c..341be1dd031ab 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4768,6 +4768,11 @@ def test_no_isinstance(self): with self.assertRaises(TypeError): isinstance(42, TypeAlias) + def test_stringized_usage(self): + class A: + a: "TypeAlias" + self.assertEqual(get_type_hints(A), {'a': TypeAlias}) + def test_no_issubclass(self): with self.assertRaises(TypeError): issubclass(Employee, TypeAlias) diff --git a/Lib/typing.py b/Lib/typing.py index abd5899806f29..7743c7f76bde8 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -165,7 +165,7 @@ def _type_check(arg, msg, is_argument=True, module=None, *, allow_special_forms= if (isinstance(arg, _GenericAlias) and arg.__origin__ in invalid_generic_forms): raise TypeError(f"{arg} is not valid as type argument") - if arg in (Any, NoReturn, Final): + if arg in (Any, NoReturn, Final, TypeAlias): return arg if isinstance(arg, _SpecialForm) or arg in (Generic, Protocol): raise TypeError(f"Plain {arg} is not valid as type argument") diff --git a/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst b/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst new file mode 100644 index 0000000000000..4f0de9519a00e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst @@ -0,0 +1 @@ +In :func:`typing.get_type_hints`, support evaluating bare stringified ``TypeAlias`` annotations. Patch by Gregory Beauregard. \ No newline at end of file From webhook-mailer at python.org Mon Feb 7 11:46:32 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 16:46:32 -0000 Subject: [Python-checkins] bpo-46166: Fix compiler warnings in What's New in Python 3.11 (GH-31198) Message-ID: https://github.com/python/cpython/commit/a89772c79183e3e62bf61b92077a04f6ebcc4a2b commit: a89772c79183e3e62bf61b92077a04f6ebcc4a2b branch: main author: Victor Stinner committer: vstinner date: 2022-02-07T17:46:22+01:00 summary: bpo-46166: Fix compiler warnings in What's New in Python 3.11 (GH-31198) Fix compiler warnings on PyObject_GetAttrString() calls in the What's New in Python 3.11 doc of PyFrameObject changes. files: M Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 7b5e7da8f77aa..c3c2c67deaf33 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -816,13 +816,13 @@ Porting to Python 3.11 :c:func:`Py_DECREF`. * ``f_back``: changed (see below), use :c:func:`PyFrame_GetBack`. * ``f_builtins``: removed, - use ``PyObject_GetAttrString(frame, "f_builtins")``. + use ``PyObject_GetAttrString((PyObject*)frame, "f_builtins")``. * ``f_globals``: removed, - use ``PyObject_GetAttrString(frame, "f_globals")``. + use ``PyObject_GetAttrString((PyObject*)frame, "f_globals")``. * ``f_locals``: removed, - use ``PyObject_GetAttrString(frame, "f_locals")``. + use ``PyObject_GetAttrString((PyObject*)frame, "f_locals")``. * ``f_lasti``: removed, - use ``PyObject_GetAttrString(frame, "f_lasti")``. + use ``PyObject_GetAttrString((PyObject*)frame, "f_lasti")``. The following fields were removed entirely, as they were details of the old implementation: From webhook-mailer at python.org Mon Feb 7 11:51:53 2022 From: webhook-mailer at python.org (markshannon) Date: Mon, 07 Feb 2022 16:51:53 -0000 Subject: [Python-checkins] bpo-46072: Merge dxpairs into py_stats. (GH-31197) Message-ID: https://github.com/python/cpython/commit/9c979d7afd839abbb080028bdfeb73727e5cf633 commit: 9c979d7afd839abbb080028bdfeb73727e5cf633 branch: main author: Mark Shannon committer: markshannon date: 2022-02-07T16:51:43Z summary: bpo-46072: Merge dxpairs into py_stats. (GH-31197) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-07-14-38-54.bpo-46072.6ebLyN.rst M Python/ceval.c M Python/sysmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-07-14-38-54.bpo-46072.6ebLyN.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-07-14-38-54.bpo-46072.6ebLyN.rst new file mode 100644 index 0000000000000..288cb56cc205f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-07-14-38-54.bpo-46072.6ebLyN.rst @@ -0,0 +1,2 @@ +Opcode pair stats are now gathered with ``--enable-pystats``. Defining +``DYNAMIC_EXECUTION_PROFILE`` or ``DXPAIRS`` no longer has any effect. diff --git a/Python/ceval.c b/Python/ceval.c index 31b41b848486a..52dbd6e7b0720 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -114,16 +114,6 @@ _PyEvalFrameClearAndPop(PyThreadState *tstate, InterpreterFrame *frame); "cannot access free variable '%s' where it is not associated with a" \ " value in enclosing scope" -/* Dynamic execution profile */ -#ifdef DYNAMIC_EXECUTION_PROFILE -#ifdef DXPAIRS -static long dxpairs[257][256]; -#define dxp dxpairs[256] -#else -static long dxp[256]; -#endif -#endif - #ifndef NDEBUG /* Ensure that tstate is valid: sanity check for PyEval_AcquireThread() and PyEval_RestoreThread(). Detect if tstate memory was freed. It can happen @@ -1238,10 +1228,6 @@ eval_frame_handle_pending(PyThreadState *tstate) faster than the normal "switch" version, depending on the compiler and the CPU architecture. - We disable the optimization if DYNAMIC_EXECUTION_PROFILE is defined, - because it would render the measurements invalid. - - NOTE: care must be taken that the compiler doesn't try to "optimize" the indirect jumps by sharing them between all opcodes. Such optimizations can be disabled on gcc by using the -fno-gcse flag (or possibly @@ -1253,11 +1239,6 @@ eval_frame_handle_pending(PyThreadState *tstate) * We want to be sure that the compiler knows this before it generates * the CFG. */ -#ifdef LLTRACE -#define LLTRACE_INSTR() if (lltrace) { lltrace_instruction(frame, opcode, oparg); } -#else -#define LLTRACE_INSTR() ((void)0) -#endif #ifdef WITH_DTRACE #define OR_DTRACE_LINE | (PyDTrace_LINE_ENABLED() ? 255 : 0) @@ -1265,11 +1246,6 @@ eval_frame_handle_pending(PyThreadState *tstate) #define OR_DTRACE_LINE #endif -#ifdef DYNAMIC_EXECUTION_PROFILE -#undef USE_COMPUTED_GOTOS -#define USE_COMPUTED_GOTOS 0 -#endif - #ifdef HAVE_COMPUTED_GOTOS #ifndef USE_COMPUTED_GOTOS #define USE_COMPUTED_GOTOS 1 @@ -1283,7 +1259,14 @@ eval_frame_handle_pending(PyThreadState *tstate) #endif #ifdef Py_STATS -#define INSTRUCTION_START(op) frame->f_lasti = INSTR_OFFSET(); next_instr++; OPCODE_EXE_INC(op); +#define INSTRUCTION_START(op) \ + do { \ + frame->f_lasti = INSTR_OFFSET(); \ + next_instr++; \ + OPCODE_EXE_INC(op); \ + _py_stats.opcode_stats[lastopcode].pair_count[op]++; \ + lastopcode = op; \ + } while (0) #else #define INSTRUCTION_START(op) frame->f_lasti = INSTR_OFFSET(); next_instr++ #endif @@ -1296,34 +1279,12 @@ eval_frame_handle_pending(PyThreadState *tstate) #define DISPATCH_GOTO() goto dispatch_opcode #endif -/* RECORD_DXPROFILE() records the dxprofile information, if enabled. Normally a no-op */ -#ifdef DYNAMIC_EXECUTION_PROFILE -#ifdef DXPAIRS -#define RECORD_DXPROFILE() \ - do { \ - dxpairs[lastopcode][opcode]++; \ - lastopcode = opcode; \ - dxp[opcode]++; \ - } while (0) -#else - #define RECORD_DXPROFILE() \ - do { \ - dxp[opcode]++; \ - } while (0) -#endif +/* PRE_DISPATCH_GOTO() does lltrace if enabled. Normally a no-op */ +#ifdef LLTRACE +#define PRE_DISPATCH_GOTO() if (lltrace) { lltrace_instruction(frame, opcode, oparg); } #else -#define RECORD_DXPROFILE() ((void)0) -#endif - -/* PRE_DISPATCH_GOTO() does lltrace and dxprofile if either is enabled. Normally a no-op */ -#ifndef LLTRACE -#ifndef DYNAMIC_EXECUTION_PROFILE #define PRE_DISPATCH_GOTO() ((void)0) #endif -#endif -#ifndef PRE_DISPATCH_GOTO -#define PRE_DISPATCH_GOTO() do { LLTRACE_INSTR(); RECORD_DXPROFILE(); } while (0) -#endif #define NOTRACE_DISPATCH() \ { \ @@ -1403,7 +1364,7 @@ eval_frame_handle_pending(PyThreadState *tstate) #define PREDICT_ID(op) PRED_##op -#if defined(DYNAMIC_EXECUTION_PROFILE) || USE_COMPUTED_GOTOS +#if USE_COMPUTED_GOTOS #define PREDICT(op) if (0) goto PREDICT_ID(op) #else #define PREDICT(op) \ @@ -1653,7 +1614,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr #include "opcode_targets.h" #endif -#ifdef DXPAIRS +#ifdef Py_STATS int lastopcode = 0; #endif int opcode; /* Current opcode */ @@ -7489,16 +7450,16 @@ format_awaitable_error(PyThreadState *tstate, PyTypeObject *type, int prevprevpr } } -#ifdef DYNAMIC_EXECUTION_PROFILE +#ifdef Py_STATS static PyObject * -getarray(long a[256]) +getarray(uint64_t a[256]) { int i; PyObject *l = PyList_New(256); if (l == NULL) return NULL; for (i = 0; i < 256; i++) { - PyObject *x = PyLong_FromLong(a[i]); + PyObject *x = PyLong_FromUnsignedLongLong(a[i]); if (x == NULL) { Py_DECREF(l); return NULL; @@ -7513,14 +7474,11 @@ getarray(long a[256]) PyObject * _Py_GetDXProfile(PyObject *self, PyObject *args) { -#ifndef DXPAIRS - return getarray(dxp); -#else int i; PyObject *l = PyList_New(257); if (l == NULL) return NULL; for (i = 0; i < 257; i++) { - PyObject *x = getarray(dxpairs[i]); + PyObject *x = getarray(_py_stats.opcode_stats[i].pair_count); if (x == NULL) { Py_DECREF(l); return NULL; @@ -7528,7 +7486,6 @@ _Py_GetDXProfile(PyObject *self, PyObject *args) PyList_SET_ITEM(l, i, x); } return l; -#endif } #endif diff --git a/Python/sysmodule.c b/Python/sysmodule.c index 7597ea2ea9e49..acb03781a4f14 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -1923,7 +1923,7 @@ sys__debugmallocstats_impl(PyObject *module) extern PyObject *_Py_GetObjects(PyObject *, PyObject *); #endif -#ifdef DYNAMIC_EXECUTION_PROFILE +#ifdef Py_STATS /* Defined in ceval.c because it uses static globals in that file */ extern PyObject *_Py_GetDXProfile(PyObject *, PyObject *); #endif @@ -1992,7 +1992,7 @@ static PyMethodDef sys_methods[] = { SYS_GETDEFAULTENCODING_METHODDEF SYS_GETDLOPENFLAGS_METHODDEF SYS_GETALLOCATEDBLOCKS_METHODDEF -#ifdef DYNAMIC_EXECUTION_PROFILE +#ifdef Py_STATS {"getdxp", _Py_GetDXProfile, METH_VARARGS}, #endif SYS_GETFILESYSTEMENCODING_METHODDEF From webhook-mailer at python.org Mon Feb 7 12:00:04 2022 From: webhook-mailer at python.org (zooba) Date: Mon, 07 Feb 2022 17:00:04 -0000 Subject: [Python-checkins] bpo-46638: Makes registry virtualisation setting stable when building MSIX packages (GH-31130) Message-ID: https://github.com/python/cpython/commit/3a5afc14e16370c1f4f72d43cb553298ad9a1fa4 commit: 3a5afc14e16370c1f4f72d43cb553298ad9a1fa4 branch: main author: Steve Dower committer: zooba date: 2022-02-07T16:59:40Z summary: bpo-46638: Makes registry virtualisation setting stable when building MSIX packages (GH-31130) files: A Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst M PC/layout/support/appxmanifest.py M PC/layout/support/constants.py diff --git a/Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst b/Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst new file mode 100644 index 0000000000000..536aae68f8329 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst @@ -0,0 +1,4 @@ +Ensures registry virtualization is consistently disabled. For 3.10 and +earlier, it remains enabled (some registry writes are protected), while for +3.11 and later it is disabled (registry modifications affect all +applications). diff --git a/PC/layout/support/appxmanifest.py b/PC/layout/support/appxmanifest.py index 15119b02a802a..427a36f31c8f9 100644 --- a/PC/layout/support/appxmanifest.py +++ b/PC/layout/support/appxmanifest.py @@ -412,14 +412,22 @@ def get_appxmanifest(ns): if value: node.text = value - winver = sys.getwindowsversion()[:3] + try: + winver = tuple(int(i) for i in os.getenv("APPX_DATA_WINVER", "").split(".", maxsplit=3)) + except (TypeError, ValueError): + winver = () + + # Default "known good" version is 10.0.22000, first Windows 11 release + winver = winver or (10, 0, 22000) + if winver < (10, 0, 17763): winver = 10, 0, 17763 find_or_add(xml, "m:Dependencies/m:TargetDeviceFamily").set( - "MaxVersionTested", "{}.{}.{}.0".format(*winver) + "MaxVersionTested", "{}.{}.{}.{}".format(*(winver + (0, 0, 0, 0)[:4])) ) - if winver > (10, 0, 17763): + # Only for Python 3.11 and later. Older versions do not disable virtualization + if (VER_MAJOR, VER_MINOR) >= (3, 11) and winver > (10, 0, 17763): disable_registry_virtualization(xml) app = add_application( diff --git a/PC/layout/support/constants.py b/PC/layout/support/constants.py index 6cf0fe1d34c4a..6efd8bcd5cbb5 100644 --- a/PC/layout/support/constants.py +++ b/PC/layout/support/constants.py @@ -16,7 +16,7 @@ def _unpack_hexversion(): hexversion = int(os.getenv("PYTHON_HEXVERSION"), 16) except (TypeError, ValueError): hexversion = sys.hexversion - return struct.pack(">i", sys.hexversion) + return struct.pack(">i", hexversion) def _get_suffix(field4): From webhook-mailer at python.org Mon Feb 7 12:31:47 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 07 Feb 2022 17:31:47 -0000 Subject: [Python-checkins] bpo-46638: Makes registry virtualisation setting stable when building MSIX packages (GH-31130) Message-ID: https://github.com/python/cpython/commit/9c45390208df712126c59f7c2b6f8d2b4e19ccf7 commit: 9c45390208df712126c59f7c2b6f8d2b4e19ccf7 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-07T09:31:32-08:00 summary: bpo-46638: Makes registry virtualisation setting stable when building MSIX packages (GH-31130) (cherry picked from commit 3a5afc14e16370c1f4f72d43cb553298ad9a1fa4) Co-authored-by: Steve Dower files: A Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst M PC/layout/support/appxmanifest.py M PC/layout/support/constants.py diff --git a/Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst b/Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst new file mode 100644 index 0000000000000..536aae68f8329 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst @@ -0,0 +1,4 @@ +Ensures registry virtualization is consistently disabled. For 3.10 and +earlier, it remains enabled (some registry writes are protected), while for +3.11 and later it is disabled (registry modifications affect all +applications). diff --git a/PC/layout/support/appxmanifest.py b/PC/layout/support/appxmanifest.py index 15119b02a802a..427a36f31c8f9 100644 --- a/PC/layout/support/appxmanifest.py +++ b/PC/layout/support/appxmanifest.py @@ -412,14 +412,22 @@ def get_appxmanifest(ns): if value: node.text = value - winver = sys.getwindowsversion()[:3] + try: + winver = tuple(int(i) for i in os.getenv("APPX_DATA_WINVER", "").split(".", maxsplit=3)) + except (TypeError, ValueError): + winver = () + + # Default "known good" version is 10.0.22000, first Windows 11 release + winver = winver or (10, 0, 22000) + if winver < (10, 0, 17763): winver = 10, 0, 17763 find_or_add(xml, "m:Dependencies/m:TargetDeviceFamily").set( - "MaxVersionTested", "{}.{}.{}.0".format(*winver) + "MaxVersionTested", "{}.{}.{}.{}".format(*(winver + (0, 0, 0, 0)[:4])) ) - if winver > (10, 0, 17763): + # Only for Python 3.11 and later. Older versions do not disable virtualization + if (VER_MAJOR, VER_MINOR) >= (3, 11) and winver > (10, 0, 17763): disable_registry_virtualization(xml) app = add_application( diff --git a/PC/layout/support/constants.py b/PC/layout/support/constants.py index 6cf0fe1d34c4a..6efd8bcd5cbb5 100644 --- a/PC/layout/support/constants.py +++ b/PC/layout/support/constants.py @@ -16,7 +16,7 @@ def _unpack_hexversion(): hexversion = int(os.getenv("PYTHON_HEXVERSION"), 16) except (TypeError, ValueError): hexversion = sys.hexversion - return struct.pack(">i", sys.hexversion) + return struct.pack(">i", hexversion) def _get_suffix(field4): From webhook-mailer at python.org Mon Feb 7 12:37:15 2022 From: webhook-mailer at python.org (zooba) Date: Mon, 07 Feb 2022 17:37:15 -0000 Subject: [Python-checkins] bpo-46638: Makes registry virtualisation setting stable when building MSIX packages (GH-31130) Message-ID: https://github.com/python/cpython/commit/76b072717a160c44cb8d54be3d5e878bc31f2c38 commit: 76b072717a160c44cb8d54be3d5e878bc31f2c38 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: zooba date: 2022-02-07T17:37:01Z summary: bpo-46638: Makes registry virtualisation setting stable when building MSIX packages (GH-31130) (cherry picked from commit 3a5afc14e16370c1f4f72d43cb553298ad9a1fa4) files: A Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst M PC/layout/support/appxmanifest.py M PC/layout/support/constants.py diff --git a/Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst b/Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst new file mode 100644 index 0000000000000..536aae68f8329 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2022-02-04-18-02-33.bpo-46638.mSJOSX.rst @@ -0,0 +1,4 @@ +Ensures registry virtualization is consistently disabled. For 3.10 and +earlier, it remains enabled (some registry writes are protected), while for +3.11 and later it is disabled (registry modifications affect all +applications). diff --git a/PC/layout/support/appxmanifest.py b/PC/layout/support/appxmanifest.py index 15119b02a802a..427a36f31c8f9 100644 --- a/PC/layout/support/appxmanifest.py +++ b/PC/layout/support/appxmanifest.py @@ -412,14 +412,22 @@ def get_appxmanifest(ns): if value: node.text = value - winver = sys.getwindowsversion()[:3] + try: + winver = tuple(int(i) for i in os.getenv("APPX_DATA_WINVER", "").split(".", maxsplit=3)) + except (TypeError, ValueError): + winver = () + + # Default "known good" version is 10.0.22000, first Windows 11 release + winver = winver or (10, 0, 22000) + if winver < (10, 0, 17763): winver = 10, 0, 17763 find_or_add(xml, "m:Dependencies/m:TargetDeviceFamily").set( - "MaxVersionTested", "{}.{}.{}.0".format(*winver) + "MaxVersionTested", "{}.{}.{}.{}".format(*(winver + (0, 0, 0, 0)[:4])) ) - if winver > (10, 0, 17763): + # Only for Python 3.11 and later. Older versions do not disable virtualization + if (VER_MAJOR, VER_MINOR) >= (3, 11) and winver > (10, 0, 17763): disable_registry_virtualization(xml) app = add_application( diff --git a/PC/layout/support/constants.py b/PC/layout/support/constants.py index 6cf0fe1d34c4a..6efd8bcd5cbb5 100644 --- a/PC/layout/support/constants.py +++ b/PC/layout/support/constants.py @@ -16,7 +16,7 @@ def _unpack_hexversion(): hexversion = int(os.getenv("PYTHON_HEXVERSION"), 16) except (TypeError, ValueError): hexversion = sys.hexversion - return struct.pack(">i", sys.hexversion) + return struct.pack(">i", hexversion) def _get_suffix(field4): From webhook-mailer at python.org Mon Feb 7 14:40:55 2022 From: webhook-mailer at python.org (zware) Date: Mon, 07 Feb 2022 19:40:55 -0000 Subject: [Python-checkins] Skip cron workflow for forks (GH-30748) Message-ID: https://github.com/python/cpython/commit/39dec1c09c9f5ddf951bed5b875f837735a06733 commit: 39dec1c09c9f5ddf951bed5b875f837735a06733 branch: main author: Hugo van Kemenade committer: zware date: 2022-02-07T13:40:40-06:00 summary: Skip cron workflow for forks (GH-30748) files: M .github/workflows/stale.yml diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml index e3b8b9f942d18..1532af26f59ff 100644 --- a/.github/workflows/stale.yml +++ b/.github/workflows/stale.yml @@ -9,6 +9,7 @@ permissions: jobs: stale: + if: github.repository_owner == 'python' runs-on: ubuntu-latest From webhook-mailer at python.org Mon Feb 7 15:15:46 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 07 Feb 2022 20:15:46 -0000 Subject: [Python-checkins] [3.10] bpo-46648: Rewrite test_urllib2.test_issue16464() with a local HTTP server (GH-31186) (GH-31189) Message-ID: https://github.com/python/cpython/commit/9539400390494f4930c245b3f98453592f4a1a8c commit: 9539400390494f4930c245b3f98453592f4a1a8c branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-07T12:15:22-08:00 summary: [3.10] bpo-46648: Rewrite test_urllib2.test_issue16464() with a local HTTP server (GH-31186) (GH-31189) Re-enable test_issue16464() of test_urllib2, move it to urllib2_localnet and use the local HTTP server rather than an external HTTP server. (cherry picked from commit 8e98175a03fe03d62822d96007a74e5273013764) Co-authored-by: Nikita Sobolev files: M Lib/test/test_urllib2.py M Lib/test/test_urllib2_localnet.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 4d341e357063a..135741e64bb91 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1788,24 +1788,6 @@ class MyOtherHTTPHandler(urllib.request.HTTPHandler): self.opener_has_handler(o, MyHTTPHandler) self.opener_has_handler(o, MyOtherHTTPHandler) - @unittest.skipUnless(support.is_resource_enabled('network'), - 'test requires network access') - # bpo-46648: test fails randomly with "http://www.example.com/" URL - @unittest.skipIf(True, "POST request to http://www.example.com/ fail randomly") - def test_issue16464(self): - with socket_helper.transient_internet("http://www.example.com/"): - opener = urllib.request.build_opener() - request = urllib.request.Request("http://www.example.com/") - self.assertEqual(None, request.data) - - opener.open(request, "1".encode("us-ascii")) - self.assertEqual(b"1", request.data) - self.assertEqual("1", request.get_header("Content-length")) - - opener.open(request, "1234567890".encode("us-ascii")) - self.assertEqual(b"1234567890", request.data) - self.assertEqual("10", request.get_header("Content-length")) - def test_HTTPError_interface(self): """ Issue 13211 reveals that HTTPError didn't implement the URLError diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py index 0b2d07ce61d5c..1b2baf2f366b5 100644 --- a/Lib/test/test_urllib2_localnet.py +++ b/Lib/test/test_urllib2_localnet.py @@ -660,6 +660,24 @@ def test_line_iteration(self): (index, len(lines[index]), len(line))) self.assertEqual(index + 1, len(lines)) + def test_issue16464(self): + # See https://bugs.python.org/issue16464 + # and https://bugs.python.org/issue46648 + handler = self.start_server([ + (200, [], b'any'), + (200, [], b'any'), + ]) + opener = urllib.request.build_opener() + request = urllib.request.Request("http://localhost:%s" % handler.port) + self.assertEqual(None, request.data) + + opener.open(request, "1".encode("us-ascii")) + self.assertEqual(b"1", request.data) + self.assertEqual("1", request.get_header("Content-length")) + + opener.open(request, "1234567890".encode("us-ascii")) + self.assertEqual(b"1234567890", request.data) + self.assertEqual("10", request.get_header("Content-length")) def setUpModule(): thread_info = threading_helper.threading_setup() From webhook-mailer at python.org Mon Feb 7 15:48:00 2022 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 07 Feb 2022 20:48:00 -0000 Subject: [Python-checkins] bpo-46534: Implement PEP 673 Self in typing.py (GH-30924) Message-ID: https://github.com/python/cpython/commit/7ba1cc8049fbcb94ac039ab02522f78177130588 commit: 7ba1cc8049fbcb94ac039ab02522f78177130588 branch: main author: James Hilton-Balfe committer: gvanrossum date: 2022-02-07T12:47:48-08:00 summary: bpo-46534: Implement PEP 673 Self in typing.py (GH-30924) Co-authored-by: Pradeep Kumar Srinivasan Co-authored-by: Jelle Zijlstra files: A Misc/NEWS.d/next/Library/2022-01-26-18-06-08.bpo-46534.vhzUM4.rst M Doc/library/typing.rst M Doc/whatsnew/3.11.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 9007c0daf59a4..8c1c34e90507f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -68,6 +68,8 @@ annotations. These include: *Introducing* :data:`TypeAlias` * :pep:`647`: User-Defined Type Guards *Introducing* :data:`TypeGuard` +* :pep:`673`: Self type + *Introducing* :data:`Self` .. _type-aliases: @@ -585,6 +587,51 @@ These can be used as types in annotations and do not support ``[]``. .. versionadded:: 3.5.4 .. versionadded:: 3.6.2 +.. data:: Self + + Special type to represent the current enclosed class. + For example:: + + from typing import Self + + class Foo: + def returns_self(self) -> Self: + ... + return self + + + This annotation is semantically equivalent to the following, + albeit in a more succinct fashion:: + + from typing import TypeVar + + Self = TypeVar("Self", bound="Foo") + + class Foo: + def returns_self(self: Self) -> Self: + ... + return self + + In general if something currently follows the pattern of:: + + class Foo: + def return_self(self) -> "Foo": + ... + return self + + You should use use :data:`Self` as calls to ``SubclassOfFoo.returns_self`` would have + ``Foo`` as the return type and not ``SubclassOfFoo``. + + Other common use cases include: + + - :class:`classmethod`\s that are used as alternative constructors and return instances + of the ``cls`` parameter. + - Annotating an :meth:`object.__enter__` method which returns self. + + For more information, see :pep:`673`. + + .. versionadded:: 3.11 + .. data:: TypeAlias Special annotation for explicitly declaring a :ref:`type alias `. diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index c3c2c67deaf33..c1f267a4126a6 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -67,6 +67,8 @@ Summary -- Release highlights PEP-654: Exception Groups and ``except*``. (Contributed by Irit Katriel in :issue:`45292`.) +PEP-673: ``Self`` Type. +(Contributed by James Hilton-Balfe and Pradeep Kumar in :issue:`30924`.) New Features ============ diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 9eab4618e2b01..a37bb43296929 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -27,6 +27,7 @@ from typing import IO, TextIO, BinaryIO from typing import Pattern, Match from typing import Annotated, ForwardRef +from typing import Self from typing import TypeAlias from typing import ParamSpec, Concatenate, ParamSpecArgs, ParamSpecKwargs from typing import TypeGuard @@ -157,8 +158,48 @@ def test_cannot_instantiate(self): type(NoReturn)() -class TypeVarTests(BaseTestCase): +class SelfTests(BaseTestCase): + def test_basics(self): + class Foo: + def bar(self) -> Self: ... + + self.assertEqual(gth(Foo.bar), {'return': Self}) + + def test_repr(self): + self.assertEqual(repr(Self), 'typing.Self') + + def test_cannot_subscript(self): + with self.assertRaises(TypeError): + Self[int] + + def test_cannot_subclass(self): + with self.assertRaises(TypeError): + class C(type(Self)): + pass + + def test_cannot_init(self): + with self.assertRaises(TypeError): + Self() + with self.assertRaises(TypeError): + type(Self)() + + def test_no_isinstance(self): + with self.assertRaises(TypeError): + isinstance(1, Self) + with self.assertRaises(TypeError): + issubclass(int, Self) + def test_alias(self): + # TypeAliases are not actually part of the spec + alias_1 = Tuple[Self, Self] + alias_2 = List[Self] + alias_3 = ClassVar[Self] + self.assertEqual(get_args(alias_1), (Self, Self)) + self.assertEqual(get_args(alias_2), (Self,)) + self.assertEqual(get_args(alias_3), (Self,)) + + +class TypeVarTests(BaseTestCase): def test_basic_plain(self): T = TypeVar('T') # T equals itself. diff --git a/Lib/typing.py b/Lib/typing.py index f733efea09cb3..d1d513062394c 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -132,6 +132,7 @@ def _idfunc(_, x): 'ParamSpecKwargs', 'reveal_type', 'runtime_checkable', + 'Self', 'Text', 'TYPE_CHECKING', 'TypeAlias', @@ -174,7 +175,7 @@ def _type_check(arg, msg, is_argument=True, module=None, *, allow_special_forms= if (isinstance(arg, _GenericAlias) and arg.__origin__ in invalid_generic_forms): raise TypeError(f"{arg} is not valid as type argument") - if arg in (Any, NoReturn, ClassVar, Final, TypeAlias): + if arg in (Any, NoReturn, Self, ClassVar, Final, TypeAlias): return arg if isinstance(arg, _SpecialForm) or arg in (Generic, Protocol): raise TypeError(f"Plain {arg} is not valid as type argument") @@ -445,6 +446,27 @@ def stop() -> NoReturn: """ raise TypeError(f"{self} is not subscriptable") + + at _SpecialForm +def Self(self, parameters): + """Used to spell the type of "self" in classes. + + Example:: + + from typing import Self + + class Foo: + def returns_self(self) -> Self: + ... + return self + + This is especially useful for: + - classmethods that are used as alternative constructors + - annotating an `__enter__` method which returns self + """ + raise TypeError(f"{self} is not subscriptable") + + @_SpecialForm def ClassVar(self, parameters): """Special type construct to mark class variables. diff --git a/Misc/NEWS.d/next/Library/2022-01-26-18-06-08.bpo-46534.vhzUM4.rst b/Misc/NEWS.d/next/Library/2022-01-26-18-06-08.bpo-46534.vhzUM4.rst new file mode 100644 index 0000000000000..35a70aae17004 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-26-18-06-08.bpo-46534.vhzUM4.rst @@ -0,0 +1,2 @@ +Implement :pep:`673` :class:`typing.Self`. +Patch by James Hilton-Balfe. From webhook-mailer at python.org Mon Feb 7 18:24:44 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 07 Feb 2022 23:24:44 -0000 Subject: [Python-checkins] bpo-46659: calendar uses locale.getlocale() (GH-31166) Message-ID: https://github.com/python/cpython/commit/7a0486eaa98083e0407ff491872db6d7a0da2635 commit: 7a0486eaa98083e0407ff491872db6d7a0da2635 branch: main author: Victor Stinner committer: vstinner date: 2022-02-08T00:24:09+01:00 summary: bpo-46659: calendar uses locale.getlocale() (GH-31166) The calendar.LocaleTextCalendar and calendar.LocaleHTMLCalendar classes module now use locale.getlocale(), instead of using locale.getdefaultlocale(), if no locale is specified. files: A Misc/NEWS.d/next/Library/2022-02-06-19-13-02.bpo-46659.q-vNL9.rst M Doc/library/calendar.rst M Doc/whatsnew/3.11.rst M Lib/calendar.py M Lib/test/test_calendar.py diff --git a/Doc/library/calendar.rst b/Doc/library/calendar.rst index b667c42e708fa..86f5b142a6c34 100644 --- a/Doc/library/calendar.rst +++ b/Doc/library/calendar.rst @@ -290,7 +290,7 @@ interpreted as prescribed by the ISO 8601 standard. Year 0 is 1 BC, year -1 is .. note:: The :meth:`formatweekday` and :meth:`formatmonthname` methods of these two - classes temporarily change the current locale to the given *locale*. Because + classes temporarily change the ``LC_TIME`` locale to the given *locale*. Because the current locale is a process-wide setting, they are not thread-safe. diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index c1f267a4126a6..5738745ba1323 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -591,6 +591,12 @@ Changes in the Python API of sorting simply isn't well-defined in the absence of a total ordering on list elements. +* :mod:`calendar`: The :class:`calendar.LocaleTextCalendar` and + :class:`calendar.LocaleHTMLCalendar` classes now use + :func:`locale.getlocale`, instead of using :func:`locale.getdefaultlocale`, + if no locale is specified. + (Contributed by Victor Stinner in :issue:`46659`.) + Build Changes ============= diff --git a/Lib/calendar.py b/Lib/calendar.py index 06c65a80cd80f..361898dc8161e 100644 --- a/Lib/calendar.py +++ b/Lib/calendar.py @@ -566,7 +566,7 @@ class LocaleTextCalendar(TextCalendar): def __init__(self, firstweekday=0, locale=None): TextCalendar.__init__(self, firstweekday) if locale is None: - locale = _locale.getdefaultlocale() + locale = _locale.getlocale(_locale.LC_TIME) self.locale = locale def formatweekday(self, day, width): @@ -586,7 +586,7 @@ class LocaleHTMLCalendar(HTMLCalendar): def __init__(self, firstweekday=0, locale=None): HTMLCalendar.__init__(self, firstweekday) if locale is None: - locale = _locale.getdefaultlocale() + locale = _locale.getlocale(_locale.LC_TIME) self.locale = locale def formatweekday(self, day): diff --git a/Lib/test/test_calendar.py b/Lib/test/test_calendar.py index 39094ad6fd9ab..e6bd4d03e0f63 100644 --- a/Lib/test/test_calendar.py +++ b/Lib/test/test_calendar.py @@ -859,7 +859,8 @@ def test_option_locale(self): self.assertFailure('-L') self.assertFailure('--locale') self.assertFailure('-L', 'en') - lang, enc = locale.getdefaultlocale() + + lang, enc = locale.getlocale() lang = lang or 'C' enc = enc or 'UTF-8' try: diff --git a/Misc/NEWS.d/next/Library/2022-02-06-19-13-02.bpo-46659.q-vNL9.rst b/Misc/NEWS.d/next/Library/2022-02-06-19-13-02.bpo-46659.q-vNL9.rst new file mode 100644 index 0000000000000..2e30de17626b2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-06-19-13-02.bpo-46659.q-vNL9.rst @@ -0,0 +1,4 @@ +The :class:`calendar.LocaleTextCalendar` and +:class:`calendar.LocaleHTMLCalendar` classes now use :func:`locale.getlocale`, +instead of using :func:`locale.getdefaultlocale`, if no locale is specified. +Patch by Victor Stinner. From webhook-mailer at python.org Mon Feb 7 20:09:28 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 08 Feb 2022 01:09:28 -0000 Subject: [Python-checkins] bpo-46678: Fix Invalid cross device link in Lib/test/support/import_helper.py (GH-31204) Message-ID: https://github.com/python/cpython/commit/da576e08296490e94924421af71001bcfbccb317 commit: da576e08296490e94924421af71001bcfbccb317 branch: main author: Jason Wilkes committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-07T17:09:07-08:00 summary: bpo-46678: Fix Invalid cross device link in Lib/test/support/import_helper.py (GH-31204) In [Lib/test/support/import_helper.py](https://github.com/python/cpython/blob/master/Lib/test/support/import_helper.py), the function `make_legacy_pyc` makes a call to `os.rename` which can fail when the source and target live on different devices. This happens (for example) when `PYTHONPYCACHEPREFIX` is set to a directory anywhere on disk, while a ramdisk is mounted on `/tmp` (the latter of which is the default on various Linux distros). Replacing `os.rename` with `shutil.move` fixes this. Automerge-Triggered-By: GH:brettcannon files: A Misc/NEWS.d/next/Tests/2022-02-07-12-40-45.bpo-46678.zfOrgL.rst M Lib/test/support/import_helper.py diff --git a/Lib/test/support/import_helper.py b/Lib/test/support/import_helper.py index 9bce29895249a..5201dc84cf6df 100644 --- a/Lib/test/support/import_helper.py +++ b/Lib/test/support/import_helper.py @@ -3,6 +3,7 @@ import importlib import importlib.util import os +import shutil import sys import unittest import warnings @@ -59,7 +60,7 @@ def make_legacy_pyc(source): pyc_file = importlib.util.cache_from_source(source) up_one = os.path.dirname(os.path.abspath(source)) legacy_pyc = os.path.join(up_one, source + 'c') - os.rename(pyc_file, legacy_pyc) + shutil.move(pyc_file, legacy_pyc) return legacy_pyc diff --git a/Misc/NEWS.d/next/Tests/2022-02-07-12-40-45.bpo-46678.zfOrgL.rst b/Misc/NEWS.d/next/Tests/2022-02-07-12-40-45.bpo-46678.zfOrgL.rst new file mode 100644 index 0000000000000..e369cb1f67baf --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-07-12-40-45.bpo-46678.zfOrgL.rst @@ -0,0 +1,3 @@ +The function ``make_legacy_pyc`` in ``Lib/test/support/import_helper.py`` no +longer fails when ``PYTHONPYCACHEPREFIX`` is set to a directory on a +different device from where tempfiles are stored. From webhook-mailer at python.org Tue Feb 8 00:22:38 2022 From: webhook-mailer at python.org (corona10) Date: Tue, 08 Feb 2022 05:22:38 -0000 Subject: [Python-checkins] bpo-46323 Fix ref leak if ctypes.CFuncPtr raises an error. (GH-31209) Message-ID: https://github.com/python/cpython/commit/e959dd9f5c1d8865d4e10c49eb30ee0a4fe2735d commit: e959dd9f5c1d8865d4e10c49eb30ee0a4fe2735d branch: main author: Dong-hee Na committer: corona10 date: 2022-02-08T14:22:13+09:00 summary: bpo-46323 Fix ref leak if ctypes.CFuncPtr raises an error. (GH-31209) files: M Modules/_ctypes/_ctypes.c diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index da9dd096c73d8..15eb7521baa1c 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -2392,6 +2392,7 @@ converters_from_argtypes(PyObject *ob) Py_ssize_t nArgs = PyTuple_GET_SIZE(ob); if (nArgs > CTYPES_MAX_ARGCOUNT) { + Py_DECREF(ob); PyErr_Format(PyExc_ArgError, "_argtypes_ has too many arguments (%zi), maximum is %i", nArgs, CTYPES_MAX_ARGCOUNT); From webhook-mailer at python.org Tue Feb 8 02:47:06 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 08 Feb 2022 07:47:06 -0000 Subject: [Python-checkins] bpo-46676: Make ParamSpec args and kwargs equal to themselves (GH-31203) Message-ID: https://github.com/python/cpython/commit/c8b62bbe46e20d4b6dd556f2fa85960d1269aa45 commit: c8b62bbe46e20d4b6dd556f2fa85960d1269aa45 branch: main author: Gregory Beauregard committer: serhiy-storchaka date: 2022-02-08T09:46:58+02:00 summary: bpo-46676: Make ParamSpec args and kwargs equal to themselves (GH-31203) files: A Misc/NEWS.d/next/Library/2022-02-07-19-20-42.bpo-46676.3Aws1o.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index a37bb43296929..4d1c50ce73549 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4965,12 +4965,20 @@ def test_valid_uses(self): def test_args_kwargs(self): P = ParamSpec('P') + P_2 = ParamSpec('P_2') self.assertIn('args', dir(P)) self.assertIn('kwargs', dir(P)) self.assertIsInstance(P.args, ParamSpecArgs) self.assertIsInstance(P.kwargs, ParamSpecKwargs) self.assertIs(P.args.__origin__, P) self.assertIs(P.kwargs.__origin__, P) + self.assertEqual(P.args, P.args) + self.assertEqual(P.kwargs, P.kwargs) + self.assertNotEqual(P.args, P_2.args) + self.assertNotEqual(P.kwargs, P_2.kwargs) + self.assertNotEqual(P.args, P.kwargs) + self.assertNotEqual(P.kwargs, P.args) + self.assertNotEqual(P.args, P_2.kwargs) self.assertEqual(repr(P.args), "P.args") self.assertEqual(repr(P.kwargs), "P.kwargs") diff --git a/Lib/typing.py b/Lib/typing.py index d1d513062394c..0ee5c8596a021 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -859,6 +859,11 @@ def __init__(self, origin): def __repr__(self): return f"{self.__origin__.__name__}.args" + def __eq__(self, other): + if not isinstance(other, ParamSpecArgs): + return NotImplemented + return self.__origin__ == other.__origin__ + class ParamSpecKwargs(_Final, _Immutable, _root=True): """The kwargs for a ParamSpec object. @@ -878,6 +883,11 @@ def __init__(self, origin): def __repr__(self): return f"{self.__origin__.__name__}.kwargs" + def __eq__(self, other): + if not isinstance(other, ParamSpecKwargs): + return NotImplemented + return self.__origin__ == other.__origin__ + class ParamSpec(_Final, _Immutable, _TypeVarLike, _root=True): """Parameter specification variable. diff --git a/Misc/NEWS.d/next/Library/2022-02-07-19-20-42.bpo-46676.3Aws1o.rst b/Misc/NEWS.d/next/Library/2022-02-07-19-20-42.bpo-46676.3Aws1o.rst new file mode 100644 index 0000000000000..408412e6ff15d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-07-19-20-42.bpo-46676.3Aws1o.rst @@ -0,0 +1 @@ +Make :data:`typing.ParamSpec` args and kwargs equal to themselves. Patch by Gregory Beauregard. From webhook-mailer at python.org Tue Feb 8 03:41:36 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 08 Feb 2022 08:41:36 -0000 Subject: [Python-checkins] [3.10] bpo-46676: Make ParamSpec args and kwargs equal to themselves (GH-31203) (GH-31210) Message-ID: https://github.com/python/cpython/commit/cbdcae5ab90710e8d82c213f3798af1154670ff9 commit: cbdcae5ab90710e8d82c213f3798af1154670ff9 branch: 3.10 author: Gregory Beauregard committer: serhiy-storchaka date: 2022-02-08T10:41:13+02:00 summary: [3.10] bpo-46676: Make ParamSpec args and kwargs equal to themselves (GH-31203) (GH-31210) (cherry picked from commit c8b62bbe46e20d4b6dd556f2fa85960d1269aa45) Co-authored-by: Gregory Beauregard files: A Misc/NEWS.d/next/Library/2022-02-07-19-20-42.bpo-46676.3Aws1o.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 341be1dd031ab..62762418b3f4a 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4823,12 +4823,20 @@ def test_valid_uses(self): def test_args_kwargs(self): P = ParamSpec('P') + P_2 = ParamSpec('P_2') self.assertIn('args', dir(P)) self.assertIn('kwargs', dir(P)) self.assertIsInstance(P.args, ParamSpecArgs) self.assertIsInstance(P.kwargs, ParamSpecKwargs) self.assertIs(P.args.__origin__, P) self.assertIs(P.kwargs.__origin__, P) + self.assertEqual(P.args, P.args) + self.assertEqual(P.kwargs, P.kwargs) + self.assertNotEqual(P.args, P_2.args) + self.assertNotEqual(P.kwargs, P_2.kwargs) + self.assertNotEqual(P.args, P.kwargs) + self.assertNotEqual(P.kwargs, P.args) + self.assertNotEqual(P.args, P_2.kwargs) self.assertEqual(repr(P.args), "P.args") self.assertEqual(repr(P.kwargs), "P.kwargs") diff --git a/Lib/typing.py b/Lib/typing.py index 7743c7f76bde8..135ca582177b9 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -830,6 +830,11 @@ def __init__(self, origin): def __repr__(self): return f"{self.__origin__.__name__}.args" + def __eq__(self, other): + if not isinstance(other, ParamSpecArgs): + return NotImplemented + return self.__origin__ == other.__origin__ + class ParamSpecKwargs(_Final, _Immutable, _root=True): """The kwargs for a ParamSpec object. @@ -849,6 +854,11 @@ def __init__(self, origin): def __repr__(self): return f"{self.__origin__.__name__}.kwargs" + def __eq__(self, other): + if not isinstance(other, ParamSpecKwargs): + return NotImplemented + return self.__origin__ == other.__origin__ + class ParamSpec(_Final, _Immutable, _TypeVarLike, _root=True): """Parameter specification variable. diff --git a/Misc/NEWS.d/next/Library/2022-02-07-19-20-42.bpo-46676.3Aws1o.rst b/Misc/NEWS.d/next/Library/2022-02-07-19-20-42.bpo-46676.3Aws1o.rst new file mode 100644 index 0000000000000..408412e6ff15d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-07-19-20-42.bpo-46676.3Aws1o.rst @@ -0,0 +1 @@ +Make :data:`typing.ParamSpec` args and kwargs equal to themselves. Patch by Gregory Beauregard. From webhook-mailer at python.org Tue Feb 8 04:52:53 2022 From: webhook-mailer at python.org (ambv) Date: Tue, 08 Feb 2022 09:52:53 -0000 Subject: [Python-checkins] bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) (GH-31173) Message-ID: https://github.com/python/cpython/commit/f87e616af038ee8963185e11b96841c81e8ef15a commit: f87e616af038ee8963185e11b96841c81e8ef15a branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2022-02-08T10:52:39+01:00 summary: bpo-46648: Skip test_urllib2.test_issue16464() (GH-31161) (GH-31173) POST requests to http://www.example.com/ fail randomly. (cherry picked from commit 1578de2fcd685c71f9c84e09bac32901dea192c1) Co-authored-by: Victor Stinner files: M Lib/test/test_urllib2.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 160315394094f..b77f90a6ccbf1 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1785,6 +1785,8 @@ class MyOtherHTTPHandler(urllib.request.HTTPHandler): @unittest.skipUnless(support.is_resource_enabled('network'), 'test requires network access') + # bpo-46648: test fails randomly with "http://www.example.com/" URL + @unittest.skipIf(True, "POST request to http://www.example.com/ fail randomly") def test_issue16464(self): with support.transient_internet("http://www.example.com/"): opener = urllib.request.build_opener() From webhook-mailer at python.org Tue Feb 8 06:50:09 2022 From: webhook-mailer at python.org (markshannon) Date: Tue, 08 Feb 2022 11:50:09 -0000 Subject: [Python-checkins] Print summary stats for overall success of specialization. (GH-31211) Message-ID: https://github.com/python/cpython/commit/328fe3fd2034a91a15b77a24a307e218d02d7bf1 commit: 328fe3fd2034a91a15b77a24a307e218d02d7bf1 branch: main author: Mark Shannon committer: markshannon date: 2022-02-08T11:50:02Z summary: Print summary stats for overall success of specialization. (GH-31211) files: M Tools/scripts/summarize_stats.py diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index 6d0020739a31f..1271c194e96a1 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -77,6 +77,33 @@ def extract_opcode_stats(stats): return opcode_stats +def categorized_counts(opcode_stats): + basic = 0 + specialized = 0 + not_specialized = 0 + specialized_instructions = { + op for op in opcode._specialized_instructions + if "__" not in op and "ADAPTIVE" not in op} + adaptive_instructions = { + op for op in opcode._specialized_instructions + if "ADAPTIVE" in op} + for i, opcode_stat in enumerate(opcode_stats): + if "execution_count" not in opcode_stat: + continue + count = opcode_stat['execution_count'] + name = opname[i] + if "specializable" in opcode_stat: + not_specialized += count + elif name in adaptive_instructions: + not_specialized += count + elif name in specialized_instructions: + miss = opcode_stat.get("specialization.miss", 0) + not_specialized += miss + specialized += count - miss + else: + basic += count + return basic, not_specialized, specialized + def main(): stats = gather_stats() opcode_stats = extract_opcode_stats(stats) @@ -102,6 +129,11 @@ def main(): for i, opcode_stat in enumerate(opcode_stats): name = opname[i] print_specialization_stats(name, opcode_stat) + basic, not_specialized, specialized = categorized_counts(opcode_stats) + print("Specialization effectiveness:") + print(f" Base instructions {basic} {basic*100/total:0.1f}%") + print(f" Not specialized {not_specialized} {not_specialized*100/total:0.1f}%") + print(f" Specialized {specialized} {specialized*100/total:0.1f}%") print("Call stats:") total = 0 for key, value in stats.items(): From webhook-mailer at python.org Tue Feb 8 06:50:44 2022 From: webhook-mailer at python.org (markshannon) Date: Tue, 08 Feb 2022 11:50:44 -0000 Subject: [Python-checkins] bpo-46675: Allow object value arrays and split key dictionaries larger than 16 (GH-31191) Message-ID: https://github.com/python/cpython/commit/25db2b361beb865192a3424830ddcb0ae4b17318 commit: 25db2b361beb865192a3424830ddcb0ae4b17318 branch: main author: Mark Shannon committer: markshannon date: 2022-02-08T11:50:38Z summary: bpo-46675: Allow object value arrays and split key dictionaries larger than 16 (GH-31191) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-07-14-33-45.bpo-46675.ZPbdMp.rst M Include/internal/pycore_dict.h M Lib/test/test_descr.py M Lib/test/test_sys.py M Objects/dictobject.c M Python/ceval.c diff --git a/Include/internal/pycore_dict.h b/Include/internal/pycore_dict.h index faa8bb49bb7a4..64d70d187df4e 100644 --- a/Include/internal/pycore_dict.h +++ b/Include/internal/pycore_dict.h @@ -99,11 +99,17 @@ struct _dictkeysobject { see the DK_ENTRIES() macro */ }; -/* This must be no more than 16, for the order vector to fit in 64 bits */ -#define SHARED_KEYS_MAX_SIZE 16 - +/* This must be no more than 250, for the prefix size to fit in one byte. */ +#define SHARED_KEYS_MAX_SIZE 30 +#define NEXT_LOG2_SHARED_KEYS_MAX_SIZE 6 + +/* Layout of dict values: + * + * The PyObject *values are preceded by an array of bytes holding + * the insertion order and size. + * [-1] = prefix size. [-2] = used size. size[-2-n...] = insertion order. + */ struct _dictvalues { - uint64_t mv_order; PyObject *values[1]; }; @@ -131,6 +137,18 @@ extern uint64_t _pydict_global_version; PyObject *_PyObject_MakeDictFromInstanceAttributes(PyObject *obj, PyDictValues *values); +static inline void +_PyDictValues_AddToInsertionOrder(PyDictValues *values, Py_ssize_t ix) +{ + assert(ix < SHARED_KEYS_MAX_SIZE); + uint8_t *size_ptr = ((uint8_t *)values)-2; + int size = *size_ptr; + assert(size+2 < ((uint8_t *)values)[-1]); + size++; + size_ptr[-size] = (uint8_t)ix; + *size_ptr = size; +} + #ifdef __cplusplus } #endif diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py index a0942d6a85b81..5d36cb98792ac 100644 --- a/Lib/test/test_descr.py +++ b/Lib/test/test_descr.py @@ -5505,7 +5505,7 @@ class B(A): pass #Shrink keys by repeatedly creating instances - [(A(), B()) for _ in range(20)] + [(A(), B()) for _ in range(30)] a, b = A(), B() self.assertEqual(sys.getsizeof(vars(a)), sys.getsizeof(vars(b))) diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index 41c4618ad10d4..21d7ccbf70966 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -1504,15 +1504,16 @@ def delx(self): del self.__x '6P') class newstyleclass(object): pass # Separate block for PyDictKeysObject with 8 keys and 5 entries - check(newstyleclass, s + calcsize(DICT_KEY_STRUCT_FORMAT) + 32 + 21*calcsize("n2P")) + check(newstyleclass, s + calcsize(DICT_KEY_STRUCT_FORMAT) + 64 + 42*calcsize("n2P")) # dict with shared keys - check(newstyleclass().__dict__, size('nQ2P') + 15*self.P) + [newstyleclass() for _ in range(100)] + check(newstyleclass().__dict__, size('nQ2P') + self.P) o = newstyleclass() o.a = o.b = o.c = o.d = o.e = o.f = o.g = o.h = 1 # Separate block for PyDictKeysObject with 16 keys and 10 entries - check(newstyleclass, s + calcsize(DICT_KEY_STRUCT_FORMAT) + 32 + 21*calcsize("n2P")) + check(newstyleclass, s + calcsize(DICT_KEY_STRUCT_FORMAT) + 64 + 42*calcsize("n2P")) # dict with shared keys - check(newstyleclass().__dict__, size('nQ2P') + 13*self.P) + check(newstyleclass().__dict__, size('nQ2P') + self.P) # unicode # each tuple contains a string and its expected character size # don't put any static strings here, as they may contain diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-07-14-33-45.bpo-46675.ZPbdMp.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-07-14-33-45.bpo-46675.ZPbdMp.rst new file mode 100644 index 0000000000000..c3fd3fba947f8 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-07-14-33-45.bpo-46675.ZPbdMp.rst @@ -0,0 +1,2 @@ +Allow more than 16 items in a split dict before it is combined. The limit is +now 254. diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 2c8ec79ded72c..c41bdb86dbfe9 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -453,8 +453,14 @@ static PyDictKeysObject empty_keys_struct = { }; -static PyDictValues empty_values_struct = { 0, { NULL }}; -#define empty_values (&empty_values_struct) +struct { + uint8_t prefix[sizeof(PyObject *)]; + PyDictValues values; +} empty_values_struct = { + { [sizeof(PyObject *)-1] = sizeof(PyObject *) }, + {{NULL}} +}; +#define empty_values (&empty_values_struct.values) #define Py_EMPTY_KEYS &empty_keys_struct @@ -470,9 +476,9 @@ static PyDictValues empty_values_struct = { 0, { NULL }}; static inline int get_index_from_order(PyDictObject *mp, Py_ssize_t i) { - assert(mp->ma_used <= 16); - int shift = (int)(mp->ma_used-1-i)*4; - return (int)(mp->ma_values->mv_order >> shift) & 15; + assert(mp->ma_used <= SHARED_KEYS_MAX_SIZE); + assert(i < (((char *)mp->ma_values)[-2])); + return ((char *)mp->ma_values)[-3-i]; } int @@ -636,11 +642,25 @@ free_keys_object(PyDictKeysObject *keys) static inline PyDictValues* new_values(Py_ssize_t size) { - Py_ssize_t n = sizeof(PyDictValues) + sizeof(PyObject *) * (size-1); - return (PyDictValues*)PyMem_Malloc(n); + assert(size > 0); + size_t prefix_size = _Py_SIZE_ROUND_UP(size+2, sizeof(PyObject *)); + assert(prefix_size < 256); + size_t n = prefix_size + size * sizeof(PyObject *); + uint8_t *mem = PyMem_Malloc(n); + if (mem == NULL) { + return NULL; + } + assert(prefix_size % sizeof(PyObject *) == 0); + mem[prefix_size-1] = (uint8_t)prefix_size; + return (PyDictValues*)(mem + prefix_size); } -#define free_values(values) PyMem_Free(values) +static inline void +free_values(PyDictValues *values) +{ + int prefix_size = ((uint8_t *)values)[-1]; + PyMem_Free(((char *)values)-prefix_size); +} /* Consumes a reference to the keys object */ static PyObject * @@ -699,7 +719,7 @@ new_dict_with_shared_keys(PyDictKeysObject *keys) dictkeys_decref(keys); return PyErr_NoMemory(); } - values->mv_order = 0; + ((char *)values)[-2] = 0; for (i = 0; i < size; i++) { values->values[i] = NULL; } @@ -1017,7 +1037,7 @@ insertion_resize(PyDictObject *mp) return dictresize(mp, calculate_log2_keysize(GROWTH_RATE(mp))); } -static int +static Py_ssize_t insert_into_dictkeys(PyDictKeysObject *keys, PyObject *name) { assert(PyUnicode_CheckExact(name)); @@ -1048,7 +1068,7 @@ insert_into_dictkeys(PyDictKeysObject *keys, PyObject *name) keys->dk_nentries++; } assert (ix < SHARED_KEYS_MAX_SIZE); - return (int)ix; + return ix; } /* @@ -1093,9 +1113,7 @@ insertdict(PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject *value) ep->me_hash = hash; if (mp->ma_values) { Py_ssize_t index = mp->ma_keys->dk_nentries; - assert(index < SHARED_KEYS_MAX_SIZE); - assert((mp->ma_values->mv_order >> 60) == 0); - mp->ma_values->mv_order = ((mp->ma_values->mv_order)<<4) | index; + _PyDictValues_AddToInsertionOrder(mp->ma_values, index); assert (mp->ma_values->values[index] == NULL); mp->ma_values->values[index] = value; } @@ -1115,7 +1133,7 @@ insertdict(PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject *value) if (_PyDict_HasSplitTable(mp)) { mp->ma_values->values[ix] = value; if (old_value == NULL) { - mp->ma_values->mv_order = (mp->ma_values->mv_order << 4) | ix; + _PyDictValues_AddToInsertionOrder(mp->ma_values, ix); mp->ma_used++; } } @@ -1598,19 +1616,20 @@ _PyDict_SetItem_KnownHash(PyObject *op, PyObject *key, PyObject *value, return insertdict(mp, key, hash, value); } -static uint64_t -delete_index_from_order(uint64_t order, Py_ssize_t ix) -{ /* Update order */ - for (int i = 0;; i+= 4) { - assert (i < 64); - if (((order >> i) & 15) == (uint64_t)ix) { - /* Remove 4 bits at ith position */ - uint64_t high = ((order>>i)>>4)<ma_values->values[ix] = NULL; assert(ix < SHARED_KEYS_MAX_SIZE); /* Update order */ - mp->ma_values->mv_order = - delete_index_from_order(mp->ma_values->mv_order, ix); + delete_index_from_values(mp->ma_values, ix); ASSERT_CONSISTENT(mp); } else { @@ -2729,7 +2747,8 @@ PyDict_Copy(PyObject *o) free_values(newvalues); return NULL; } - newvalues->mv_order = mp->ma_values->mv_order; + size_t prefix_size = ((uint8_t *)newvalues)[-1]; + memcpy(((char *)newvalues)-prefix_size, ((char *)mp->ma_values)-prefix_size, prefix_size-1); split_copy->ma_values = newvalues; split_copy->ma_keys = mp->ma_keys; split_copy->ma_used = mp->ma_used; @@ -3031,11 +3050,11 @@ PyDict_SetDefault(PyObject *d, PyObject *key, PyObject *defaultobj) ep->me_key = key; ep->me_hash = hash; if (_PyDict_HasSplitTable(mp)) { - int index = (int)mp->ma_keys->dk_nentries; + Py_ssize_t index = (int)mp->ma_keys->dk_nentries; assert(index < SHARED_KEYS_MAX_SIZE); assert(mp->ma_values->values[index] == NULL); mp->ma_values->values[index] = value; - mp->ma_values->mv_order = (mp->ma_values->mv_order << 4) | index; + _PyDictValues_AddToInsertionOrder(mp->ma_values, index); } else { ep->me_value = value; @@ -3053,7 +3072,7 @@ PyDict_SetDefault(PyObject *d, PyObject *key, PyObject *defaultobj) Py_INCREF(value); MAINTAIN_TRACKING(mp, key, value); mp->ma_values->values[ix] = value; - mp->ma_values->mv_order = (mp->ma_values->mv_order << 4) | ix; + _PyDictValues_AddToInsertionOrder(mp->ma_values, ix); mp->ma_used++; mp->ma_version_tag = DICT_NEXT_VERSION(); } @@ -4941,7 +4960,7 @@ dictvalues_reversed(_PyDictViewObject *dv, PyObject *Py_UNUSED(ignored)) PyDictKeysObject * _PyDict_NewKeysForClass(void) { - PyDictKeysObject *keys = new_keys_object(5); /* log2(32) */ + PyDictKeysObject *keys = new_keys_object(NEXT_LOG2_SHARED_KEYS_MAX_SIZE); if (keys == NULL) { PyErr_Clear(); } @@ -4974,7 +4993,8 @@ init_inline_values(PyObject *obj, PyTypeObject *tp) PyErr_NoMemory(); return -1; } - values->mv_order = 0; + assert(((uint8_t *)values)[-1] >= size+2); + ((uint8_t *)values)[-2] = 0; for (int i = 0; i < size; i++) { values->values[i] = NULL; } @@ -5047,14 +5067,14 @@ _PyObject_StoreInstanceAttribute(PyObject *obj, PyDictValues *values, assert(keys != NULL); assert(values != NULL); assert(Py_TYPE(obj)->tp_flags & Py_TPFLAGS_MANAGED_DICT); - int ix = insert_into_dictkeys(keys, name); + Py_ssize_t ix = insert_into_dictkeys(keys, name); if (ix == DKIX_EMPTY) { if (value == NULL) { PyErr_SetObject(PyExc_AttributeError, name); return -1; } #ifdef Py_STATS - if (shared_keys_usable_size(keys) > 14) { + if (shared_keys_usable_size(keys) == SHARED_KEYS_MAX_SIZE) { OBJECT_STAT_INC(dict_materialized_too_big); } else { @@ -5077,11 +5097,11 @@ _PyObject_StoreInstanceAttribute(PyObject *obj, PyDictValues *values, PyErr_SetObject(PyExc_AttributeError, name); return -1; } - values->mv_order = (values->mv_order << 4) | ix; + _PyDictValues_AddToInsertionOrder(values, ix); } else { if (value == NULL) { - values->mv_order = delete_index_from_order(values->mv_order, ix); + delete_index_from_values(values, ix); } Py_DECREF(old_value); } diff --git a/Python/ceval.c b/Python/ceval.c index 52dbd6e7b0720..dcceee582a4e6 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -3536,14 +3536,13 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyDictValues *values = *_PyObject_ValuesPointer(owner); DEOPT_IF(values == NULL, STORE_ATTR); STAT_INC(STORE_ATTR, hit); - int index = cache0->index; + Py_ssize_t index = cache0->index; STACK_SHRINK(1); PyObject *value = POP(); PyObject *old_value = values->values[index]; values->values[index] = value; if (old_value == NULL) { - assert(index < 16); - values->mv_order = (values->mv_order << 4) | index; + _PyDictValues_AddToInsertionOrder(values, index); } else { Py_DECREF(old_value); From webhook-mailer at python.org Tue Feb 8 06:54:43 2022 From: webhook-mailer at python.org (pablogsal) Date: Tue, 08 Feb 2022 11:54:43 -0000 Subject: [Python-checkins] bpo-46521: Fix codeop to use a new partial-input mode of the parser (GH-31010) Message-ID: https://github.com/python/cpython/commit/69e10976b2e7682c6d57f4272932ebc19f8e8859 commit: 69e10976b2e7682c6d57f4272932ebc19f8e8859 branch: main author: Pablo Galindo Salgado committer: pablogsal date: 2022-02-08T11:54:37Z summary: bpo-46521: Fix codeop to use a new partial-input mode of the parser (GH-31010) files: A Misc/NEWS.d/next/Library/2022-02-01-19-34-28.bpo-46521.IMUIrs.rst M Include/cpython/compile.h M Include/errcode.h M Lib/codeop.py M Parser/pegen.c M Parser/pegen.h M Parser/tokenizer.c diff --git a/Include/cpython/compile.h b/Include/cpython/compile.h index a202c0b0e6550..518a376499295 100644 --- a/Include/cpython/compile.h +++ b/Include/cpython/compile.h @@ -18,8 +18,10 @@ #define PyCF_IGNORE_COOKIE 0x0800 #define PyCF_TYPE_COMMENTS 0x1000 #define PyCF_ALLOW_TOP_LEVEL_AWAIT 0x2000 +#define PyCF_ALLOW_INCOMPLETE_INPUT 0x4000 #define PyCF_COMPILE_MASK (PyCF_ONLY_AST | PyCF_ALLOW_TOP_LEVEL_AWAIT | \ - PyCF_TYPE_COMMENTS | PyCF_DONT_IMPLY_DEDENT) + PyCF_TYPE_COMMENTS | PyCF_DONT_IMPLY_DEDENT | \ + PyCF_ALLOW_INCOMPLETE_INPUT) typedef struct { int cf_flags; /* bitmask of CO_xxx flags relevant to future */ diff --git a/Include/errcode.h b/Include/errcode.h index 2e07fc2c96328..54ae929bf2587 100644 --- a/Include/errcode.h +++ b/Include/errcode.h @@ -26,6 +26,8 @@ extern "C" { #define E_TOODEEP 20 /* Too many indentation levels */ #define E_DEDENT 21 /* No matching outer block for dedent */ #define E_DECODE 22 /* Error in decoding into Unicode */ +#define E_EOFS 23 /* EOF in triple-quoted string */ +#define E_EOLS 24 /* EOL in single-quoted string */ #define E_LINECONT 25 /* Unexpected characters after a line continuation */ #define E_BADSINGLE 27 /* Ill-formed single statement input */ #define E_INTERACT_STOP 28 /* Interactive mode stopped tokenization */ diff --git a/Lib/codeop.py b/Lib/codeop.py index 6b56be488eeb0..568e9bbc11805 100644 --- a/Lib/codeop.py +++ b/Lib/codeop.py @@ -10,30 +10,6 @@ syntax error (OverflowError and ValueError can be produced by malformed literals). -Approach: - -First, check if the source consists entirely of blank lines and -comments; if so, replace it with 'pass', because the built-in -parser doesn't always do the right thing for these. - -Compile three times: as is, with \n, and with \n\n appended. If it -compiles as is, it's complete. If it compiles with one \n appended, -we expect more. If it doesn't compile either way, we compare the -error we get when compiling with \n or \n\n appended. If the errors -are the same, the code is broken. But if the errors are different, we -expect more. Not intuitive; not even guaranteed to hold in future -releases; but this matches the compiler's behavior from Python 1.4 -through 2.2, at least. - -Caveat: - -It is possible (but not likely) that the parser stops parsing with a -successful outcome before reaching the end of the source; in this -case, trailing symbols may be ignored instead of causing an error. -For example, a backslash followed by two newlines may be followed by -arbitrary garbage. This will be fixed once the API for the parser is -better. - The two interfaces are: compile_command(source, filename, symbol): @@ -64,7 +40,11 @@ __all__ = ["compile_command", "Compile", "CommandCompiler"] -PyCF_DONT_IMPLY_DEDENT = 0x200 # Matches pythonrun.h. +# The following flags match the values from Include/cpython/compile.h +# Caveat emptor: These flags are undocumented on purpose and depending +# on their effect outside the standard library is **unsupported**. +PyCF_DONT_IMPLY_DEDENT = 0x200 +PyCF_ALLOW_INCOMPLETE_INPUT = 0x4000 def _maybe_compile(compiler, source, filename, symbol): # Check for source consisting of only blank lines and comments. @@ -86,24 +66,12 @@ def _maybe_compile(compiler, source, filename, symbol): with warnings.catch_warnings(): warnings.simplefilter("error") - code1 = err1 = err2 = None - try: - code1 = compiler(source + "\n", filename, symbol) - except SyntaxError as e: - err1 = e - try: - code2 = compiler(source + "\n\n", filename, symbol) + compiler(source + "\n", filename, symbol) except SyntaxError as e: - err2 = e - - try: - if not code1 and _is_syntax_error(err1, err2): - raise err1 - else: - return None - finally: - err1 = err2 = None + if "incomplete input" in str(e): + return None + raise def _is_syntax_error(err1, err2): rep1 = repr(err1) @@ -115,7 +83,7 @@ def _is_syntax_error(err1, err2): return False def _compile(source, filename, symbol): - return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT) + return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT | PyCF_ALLOW_INCOMPLETE_INPUT) def compile_command(source, filename="", symbol="single"): r"""Compile a command and determine whether it is incomplete. @@ -144,7 +112,7 @@ class Compile: statement, it "remembers" and compiles all subsequent program texts with the statement in force.""" def __init__(self): - self.flags = PyCF_DONT_IMPLY_DEDENT + self.flags = PyCF_DONT_IMPLY_DEDENT | PyCF_ALLOW_INCOMPLETE_INPUT def __call__(self, source, filename, symbol): codeob = compile(source, filename, symbol, self.flags, True) diff --git a/Misc/NEWS.d/next/Library/2022-02-01-19-34-28.bpo-46521.IMUIrs.rst b/Misc/NEWS.d/next/Library/2022-02-01-19-34-28.bpo-46521.IMUIrs.rst new file mode 100644 index 0000000000000..4e9fa08d4dfbc --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-01-19-34-28.bpo-46521.IMUIrs.rst @@ -0,0 +1,2 @@ +Fix a bug in the :mod:`codeop` module that was incorrectly identifying +invalid code involving string quotes as valid code. diff --git a/Parser/pegen.c b/Parser/pegen.c index 470c2cbd7438b..6adde8432307d 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -726,6 +726,9 @@ compute_parser_flags(PyCompilerFlags *flags) if ((flags->cf_flags & PyCF_ONLY_AST) && flags->cf_feature_version < 7) { parser_flags |= PyPARSE_ASYNC_HACKS; } + if (flags->cf_flags & PyCF_ALLOW_INCOMPLETE_INPUT) { + parser_flags |= PyPARSE_ALLOW_INCOMPLETE_INPUT; + } return parser_flags; } @@ -811,16 +814,26 @@ reset_parser_state_for_error_pass(Parser *p) p->tok->interactive_underflow = IUNDERFLOW_STOP; } +static inline int +_is_end_of_source(Parser *p) { + int err = p->tok->done; + return err == E_EOF || err == E_EOFS || err == E_EOLS; +} + void * _PyPegen_run_parser(Parser *p) { void *res = _PyPegen_parse(p); assert(p->level == 0); if (res == NULL) { + if ((p->flags & PyPARSE_ALLOW_INCOMPLETE_INPUT) && _is_end_of_source(p)) { + PyErr_Clear(); + return RAISE_SYNTAX_ERROR("incomplete input"); + } if (PyErr_Occurred() && !PyErr_ExceptionMatches(PyExc_SyntaxError)) { return NULL; } - // Make a second parser pass. In this pass we activate heavier and slower checks + // Make a second parser pass. In this pass we activate heavier and slower checks // to produce better error messages and more complete diagnostics. Extra "invalid_*" // rules will be active during parsing. Token *last_token = p->tokens[p->fill - 1]; diff --git a/Parser/pegen.h b/Parser/pegen.h index caba34e535b6a..061ca3a2013cb 100644 --- a/Parser/pegen.h +++ b/Parser/pegen.h @@ -22,6 +22,7 @@ #define PyPARSE_BARRY_AS_BDFL 0x0020 #define PyPARSE_TYPE_COMMENTS 0x0040 #define PyPARSE_ASYNC_HACKS 0x0080 +#define PyPARSE_ALLOW_INCOMPLETE_INPUT 0x0100 #define CURRENT_POS (-5) diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 5b5cbdb809ebe..d38df66c69ed0 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -40,7 +40,7 @@ static struct tok_state *tok_new(void); static int tok_nextc(struct tok_state *tok); static void tok_backup(struct tok_state *tok, int c); - +static int syntaxerror(struct tok_state *tok, const char *format, ...); /* Spaces in this constant are treated as "zero or more spaces or tabs" when tokenizing. */ @@ -1031,8 +1031,9 @@ tok_nextc(struct tok_state *tok) if (tok->cur != tok->inp) { return Py_CHARMASK(*tok->cur++); /* Fast path */ } - if (tok->done != E_OK) - return EOF; + if (tok->done != E_OK) { + return EOF; + } if (tok->fp == NULL) { rc = tok_underflow_string(tok); } @@ -1964,16 +1965,21 @@ tok_get(struct tok_state *tok, const char **p_start, const char **p_end) tok->line_start = tok->multi_line_start; int start = tok->lineno; tok->lineno = tok->first_lineno; - if (quote_size == 3) { - return syntaxerror(tok, - "unterminated triple-quoted string literal" - " (detected at line %d)", start); + syntaxerror(tok, "unterminated triple-quoted string literal" + " (detected at line %d)", start); + if (c != '\n') { + tok->done = E_EOFS; + } + return ERRORTOKEN; } else { - return syntaxerror(tok, - "unterminated string literal (detected at" - " line %d)", start); + syntaxerror(tok, "unterminated string literal (detected at" + " line %d)", start); + if (c != '\n') { + tok->done = E_EOLS; + } + return ERRORTOKEN; } } if (c == quote) { From webhook-mailer at python.org Tue Feb 8 07:25:25 2022 From: webhook-mailer at python.org (pablogsal) Date: Tue, 08 Feb 2022 12:25:25 -0000 Subject: [Python-checkins] [3.10] bpo-46521: Fix codeop to use a new partial-input mode of the parser (GH-31010). (GH-31213) Message-ID: https://github.com/python/cpython/commit/5b58db75291cfbb9b6785c9845824b3e2da01c1c commit: 5b58db75291cfbb9b6785c9845824b3e2da01c1c branch: 3.10 author: Pablo Galindo Salgado committer: pablogsal date: 2022-02-08T12:25:15Z summary: [3.10] bpo-46521: Fix codeop to use a new partial-input mode of the parser (GH-31010). (GH-31213) (cherry picked from commit 69e10976b2e7682c6d57f4272932ebc19f8e8859) Co-authored-by: Pablo Galindo Salgado files: A Misc/NEWS.d/next/Library/2022-02-01-19-34-28.bpo-46521.IMUIrs.rst M Include/cpython/compile.h M Include/errcode.h M Lib/codeop.py M Parser/pegen.c M Parser/pegen.h M Parser/tokenizer.c diff --git a/Include/cpython/compile.h b/Include/cpython/compile.h index a202c0b0e6550..518a376499295 100644 --- a/Include/cpython/compile.h +++ b/Include/cpython/compile.h @@ -18,8 +18,10 @@ #define PyCF_IGNORE_COOKIE 0x0800 #define PyCF_TYPE_COMMENTS 0x1000 #define PyCF_ALLOW_TOP_LEVEL_AWAIT 0x2000 +#define PyCF_ALLOW_INCOMPLETE_INPUT 0x4000 #define PyCF_COMPILE_MASK (PyCF_ONLY_AST | PyCF_ALLOW_TOP_LEVEL_AWAIT | \ - PyCF_TYPE_COMMENTS | PyCF_DONT_IMPLY_DEDENT) + PyCF_TYPE_COMMENTS | PyCF_DONT_IMPLY_DEDENT | \ + PyCF_ALLOW_INCOMPLETE_INPUT) typedef struct { int cf_flags; /* bitmask of CO_xxx flags relevant to future */ diff --git a/Include/errcode.h b/Include/errcode.h index 2e07fc2c96328..54ae929bf2587 100644 --- a/Include/errcode.h +++ b/Include/errcode.h @@ -26,6 +26,8 @@ extern "C" { #define E_TOODEEP 20 /* Too many indentation levels */ #define E_DEDENT 21 /* No matching outer block for dedent */ #define E_DECODE 22 /* Error in decoding into Unicode */ +#define E_EOFS 23 /* EOF in triple-quoted string */ +#define E_EOLS 24 /* EOL in single-quoted string */ #define E_LINECONT 25 /* Unexpected characters after a line continuation */ #define E_BADSINGLE 27 /* Ill-formed single statement input */ #define E_INTERACT_STOP 28 /* Interactive mode stopped tokenization */ diff --git a/Lib/codeop.py b/Lib/codeop.py index 6b56be488eeb0..568e9bbc11805 100644 --- a/Lib/codeop.py +++ b/Lib/codeop.py @@ -10,30 +10,6 @@ syntax error (OverflowError and ValueError can be produced by malformed literals). -Approach: - -First, check if the source consists entirely of blank lines and -comments; if so, replace it with 'pass', because the built-in -parser doesn't always do the right thing for these. - -Compile three times: as is, with \n, and with \n\n appended. If it -compiles as is, it's complete. If it compiles with one \n appended, -we expect more. If it doesn't compile either way, we compare the -error we get when compiling with \n or \n\n appended. If the errors -are the same, the code is broken. But if the errors are different, we -expect more. Not intuitive; not even guaranteed to hold in future -releases; but this matches the compiler's behavior from Python 1.4 -through 2.2, at least. - -Caveat: - -It is possible (but not likely) that the parser stops parsing with a -successful outcome before reaching the end of the source; in this -case, trailing symbols may be ignored instead of causing an error. -For example, a backslash followed by two newlines may be followed by -arbitrary garbage. This will be fixed once the API for the parser is -better. - The two interfaces are: compile_command(source, filename, symbol): @@ -64,7 +40,11 @@ __all__ = ["compile_command", "Compile", "CommandCompiler"] -PyCF_DONT_IMPLY_DEDENT = 0x200 # Matches pythonrun.h. +# The following flags match the values from Include/cpython/compile.h +# Caveat emptor: These flags are undocumented on purpose and depending +# on their effect outside the standard library is **unsupported**. +PyCF_DONT_IMPLY_DEDENT = 0x200 +PyCF_ALLOW_INCOMPLETE_INPUT = 0x4000 def _maybe_compile(compiler, source, filename, symbol): # Check for source consisting of only blank lines and comments. @@ -86,24 +66,12 @@ def _maybe_compile(compiler, source, filename, symbol): with warnings.catch_warnings(): warnings.simplefilter("error") - code1 = err1 = err2 = None - try: - code1 = compiler(source + "\n", filename, symbol) - except SyntaxError as e: - err1 = e - try: - code2 = compiler(source + "\n\n", filename, symbol) + compiler(source + "\n", filename, symbol) except SyntaxError as e: - err2 = e - - try: - if not code1 and _is_syntax_error(err1, err2): - raise err1 - else: - return None - finally: - err1 = err2 = None + if "incomplete input" in str(e): + return None + raise def _is_syntax_error(err1, err2): rep1 = repr(err1) @@ -115,7 +83,7 @@ def _is_syntax_error(err1, err2): return False def _compile(source, filename, symbol): - return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT) + return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT | PyCF_ALLOW_INCOMPLETE_INPUT) def compile_command(source, filename="", symbol="single"): r"""Compile a command and determine whether it is incomplete. @@ -144,7 +112,7 @@ class Compile: statement, it "remembers" and compiles all subsequent program texts with the statement in force.""" def __init__(self): - self.flags = PyCF_DONT_IMPLY_DEDENT + self.flags = PyCF_DONT_IMPLY_DEDENT | PyCF_ALLOW_INCOMPLETE_INPUT def __call__(self, source, filename, symbol): codeob = compile(source, filename, symbol, self.flags, True) diff --git a/Misc/NEWS.d/next/Library/2022-02-01-19-34-28.bpo-46521.IMUIrs.rst b/Misc/NEWS.d/next/Library/2022-02-01-19-34-28.bpo-46521.IMUIrs.rst new file mode 100644 index 0000000000000..4e9fa08d4dfbc --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-01-19-34-28.bpo-46521.IMUIrs.rst @@ -0,0 +1,2 @@ +Fix a bug in the :mod:`codeop` module that was incorrectly identifying +invalid code involving string quotes as valid code. diff --git a/Parser/pegen.c b/Parser/pegen.c index 26143f57c0924..d7a498393753e 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -1202,6 +1202,9 @@ compute_parser_flags(PyCompilerFlags *flags) if ((flags->cf_flags & PyCF_ONLY_AST) && flags->cf_feature_version < 7) { parser_flags |= PyPARSE_ASYNC_HACKS; } + if (flags->cf_flags & PyCF_ALLOW_INCOMPLETE_INPUT) { + parser_flags |= PyPARSE_ALLOW_INCOMPLETE_INPUT; + } return parser_flags; } @@ -1327,15 +1330,29 @@ _PyPegen_check_tokenizer_errors(Parser *p) { return ret; } + +static inline int +_is_end_of_source(Parser *p) { + int err = p->tok->done; + return err == E_EOF || err == E_EOFS || err == E_EOLS; +} + void * _PyPegen_run_parser(Parser *p) { void *res = _PyPegen_parse(p); assert(p->level == 0); if (res == NULL) { + if ((p->flags & PyPARSE_ALLOW_INCOMPLETE_INPUT) && _is_end_of_source(p)) { + PyErr_Clear(); + return RAISE_SYNTAX_ERROR("incomplete input"); + } if (PyErr_Occurred() && !PyErr_ExceptionMatches(PyExc_SyntaxError)) { return NULL; } + // Make a second parser pass. In this pass we activate heavier and slower checks + // to produce better error messages and more complete diagnostics. Extra "invalid_*" + // rules will be active during parsing. Token *last_token = p->tokens[p->fill - 1]; reset_parser_state(p); _PyPegen_parse(p); diff --git a/Parser/pegen.h b/Parser/pegen.h index 29d48052e4bc9..bf58f1e1f2b7e 100644 --- a/Parser/pegen.h +++ b/Parser/pegen.h @@ -22,6 +22,7 @@ #define PyPARSE_BARRY_AS_BDFL 0x0020 #define PyPARSE_TYPE_COMMENTS 0x0040 #define PyPARSE_ASYNC_HACKS 0x0080 +#define PyPARSE_ALLOW_INCOMPLETE_INPUT 0x0100 typedef struct _memo { int type; diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 3738a9021fcd7..eb15ef7a5d5f7 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -39,7 +39,7 @@ static struct tok_state *tok_new(void); static int tok_nextc(struct tok_state *tok); static void tok_backup(struct tok_state *tok, int c); - +static int syntaxerror(struct tok_state *tok, const char *format, ...); /* Spaces in this constant are treated as "zero or more spaces or tabs" when tokenizing. */ @@ -1030,8 +1030,9 @@ tok_nextc(struct tok_state *tok) if (tok->cur != tok->inp) { return Py_CHARMASK(*tok->cur++); /* Fast path */ } - if (tok->done != E_OK) - return EOF; + if (tok->done != E_OK) { + return EOF; + } if (tok->fp == NULL) { rc = tok_underflow_string(tok); } @@ -1963,16 +1964,21 @@ tok_get(struct tok_state *tok, const char **p_start, const char **p_end) tok->line_start = tok->multi_line_start; int start = tok->lineno; tok->lineno = tok->first_lineno; - if (quote_size == 3) { - return syntaxerror(tok, - "unterminated triple-quoted string literal" - " (detected at line %d)", start); + syntaxerror(tok, "unterminated triple-quoted string literal" + " (detected at line %d)", start); + if (c != '\n') { + tok->done = E_EOFS; + } + return ERRORTOKEN; } else { - return syntaxerror(tok, - "unterminated string literal (detected at" - " line %d)", start); + syntaxerror(tok, "unterminated string literal (detected at" + " line %d)", start); + if (c != '\n') { + tok->done = E_EOLS; + } + return ERRORTOKEN; } } if (c == quote) { From webhook-mailer at python.org Tue Feb 8 08:09:30 2022 From: webhook-mailer at python.org (corona10) Date: Tue, 08 Feb 2022 13:09:30 -0000 Subject: [Python-checkins] bpo-46323: Use PyObject_Vectorcall while calling ctypes callback function (GH-31138) Message-ID: https://github.com/python/cpython/commit/b5527688aae11d0b5af58176267a9943576e71e5 commit: b5527688aae11d0b5af58176267a9943576e71e5 branch: main author: Dong-hee Na committer: corona10 date: 2022-02-08T22:09:17+09:00 summary: bpo-46323: Use PyObject_Vectorcall while calling ctypes callback function (GH-31138) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-05-14-46-21.bpo-46323.FC1OJg.rst M Modules/_ctypes/callbacks.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-05-14-46-21.bpo-46323.FC1OJg.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-05-14-46-21.bpo-46323.FC1OJg.rst new file mode 100644 index 0000000000000..893c9589eb54b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-05-14-46-21.bpo-46323.FC1OJg.rst @@ -0,0 +1,2 @@ +Use :c:func:`PyObject_Vectorcall` while calling ctypes callback function. +Patch by Dong-hee Na. diff --git a/Modules/_ctypes/callbacks.c b/Modules/_ctypes/callbacks.c index b4079eee44667..a8fee0de71aa8 100644 --- a/Modules/_ctypes/callbacks.c +++ b/Modules/_ctypes/callbacks.c @@ -146,47 +146,48 @@ static void _CallPythonObject(void *mem, int flags, void **pArgs) { - Py_ssize_t i; - PyObject *result; - PyObject *arglist = NULL; - Py_ssize_t nArgs; + PyObject *result = NULL; + PyObject **args = NULL; + Py_ssize_t i = 0, j = 0, nargs = 0; PyObject *error_object = NULL; int *space; PyGILState_STATE state = PyGILState_Ensure(); - nArgs = PySequence_Length(converters); + assert(PyTuple_Check(converters)); + nargs = PyTuple_GET_SIZE(converters); /* Hm. What to return in case of error? For COM, 0xFFFFFFFF seems better than 0. */ - if (nArgs < 0) { + if (nargs < 0) { PrintError("BUG: PySequence_Length"); goto Done; } - arglist = PyTuple_New(nArgs); - if (!arglist) { - PrintError("PyTuple_New()"); - goto Done; + PyObject *args_stack[CTYPES_MAX_ARGCOUNT]; + if (nargs <= CTYPES_MAX_ARGCOUNT) { + args = args_stack; } - for (i = 0; i < nArgs; ++i) { - /* Note: new reference! */ - PyObject *cnv = PySequence_GetItem(converters, i); - StgDictObject *dict; - if (cnv) - dict = PyType_stgdict(cnv); - else { - PrintError("Getting argument converter %zd\n", i); + else { + args = PyMem_Malloc(nargs * sizeof(PyObject *)); + if (args == NULL) { + PyErr_NoMemory(); goto Done; } + } + + PyObject **cnvs = PySequence_Fast_ITEMS(converters); + for (i = 0; i < nargs; i++) { + PyObject *cnv = cnvs[i]; // borrowed ref + StgDictObject *dict; + dict = PyType_stgdict(cnv); if (dict && dict->getfunc && !_ctypes_simple_instance(cnv)) { PyObject *v = dict->getfunc(*pArgs, dict->size); if (!v) { PrintError("create argument %zd:\n", i); - Py_DECREF(cnv); goto Done; } - PyTuple_SET_ITEM(arglist, i, v); + args[i] = v; /* XXX XXX XX We have the problem that c_byte or c_short have dict->size of 1 resp. 4, but these parameters are pushed as sizeof(int) bytes. @@ -202,12 +203,11 @@ static void _CallPythonObject(void *mem, } if (!CDataObject_Check(obj)) { Py_DECREF(obj); - Py_DECREF(cnv); PrintError("unexpected result of create argument %zd:\n", i); goto Done; } memcpy(obj->b_ptr, *pArgs, dict->size); - PyTuple_SET_ITEM(arglist, i, (PyObject *)obj); + args[i] = (PyObject *)obj; #ifdef MS_WIN32 TryAddRef(dict, obj); #endif @@ -215,10 +215,8 @@ static void _CallPythonObject(void *mem, PyErr_SetString(PyExc_TypeError, "cannot build parameter"); PrintError("Parsing argument %zd\n", i); - Py_DECREF(cnv); goto Done; } - Py_DECREF(cnv); /* XXX error handling! */ pArgs++; } @@ -241,7 +239,7 @@ static void _CallPythonObject(void *mem, #endif } - result = PyObject_CallObject(callable, arglist); + result = PyObject_Vectorcall(callable, args, nargs, NULL); if (result == NULL) { _PyErr_WriteUnraisableMsg("on calling ctypes callback function", callable); @@ -308,7 +306,12 @@ static void _CallPythonObject(void *mem, Py_XDECREF(result); Done: - Py_XDECREF(arglist); + for (j = 0; j < i; j++) { + Py_DECREF(args[j]); + } + if (args != args_stack) { + PyMem_Free(args); + } PyGILState_Release(state); } @@ -328,12 +331,12 @@ static void closure_fcn(ffi_cif *cif, args); } -static CThunkObject* CThunkObject_new(Py_ssize_t nArgs) +static CThunkObject* CThunkObject_new(Py_ssize_t nargs) { CThunkObject *p; Py_ssize_t i; - p = PyObject_GC_NewVar(CThunkObject, &PyCThunk_Type, nArgs); + p = PyObject_GC_NewVar(CThunkObject, &PyCThunk_Type, nargs); if (p == NULL) { return NULL; } @@ -348,7 +351,7 @@ static CThunkObject* CThunkObject_new(Py_ssize_t nArgs) p->setfunc = NULL; p->ffi_restype = NULL; - for (i = 0; i < nArgs + 1; ++i) + for (i = 0; i < nargs + 1; ++i) p->atypes[i] = NULL; PyObject_GC_Track((PyObject *)p); return p; @@ -361,11 +364,12 @@ CThunkObject *_ctypes_alloc_callback(PyObject *callable, { int result; CThunkObject *p; - Py_ssize_t nArgs, i; + Py_ssize_t nargs, i; ffi_abi cc; - nArgs = PySequence_Size(converters); - p = CThunkObject_new(nArgs); + assert(PyTuple_Check(converters)); + nargs = PyTuple_GET_SIZE(converters); + p = CThunkObject_new(nargs); if (p == NULL) return NULL; @@ -378,12 +382,10 @@ CThunkObject *_ctypes_alloc_callback(PyObject *callable, } p->flags = flags; - for (i = 0; i < nArgs; ++i) { - PyObject *cnv = PySequence_GetItem(converters, i); - if (cnv == NULL) - goto error; + PyObject **cnvs = PySequence_Fast_ITEMS(converters); + for (i = 0; i < nargs; ++i) { + PyObject *cnv = cnvs[i]; // borrowed ref p->atypes[i] = _ctypes_get_ffi_type(cnv); - Py_DECREF(cnv); } p->atypes[i] = NULL; @@ -409,7 +411,7 @@ CThunkObject *_ctypes_alloc_callback(PyObject *callable, cc = FFI_STDCALL; #endif result = ffi_prep_cif(&p->cif, cc, - Py_SAFE_DOWNCAST(nArgs, Py_ssize_t, int), + Py_SAFE_DOWNCAST(nargs, Py_ssize_t, int), _ctypes_get_ffi_type(restype), &p->atypes[0]); if (result != FFI_OK) { From webhook-mailer at python.org Tue Feb 8 13:42:27 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Tue, 08 Feb 2022 18:42:27 -0000 Subject: [Python-checkins] bpo-45952: Get the C analyzer tool working again. (gh-31219) Message-ID: https://github.com/python/cpython/commit/1e6214dbd6a980b47123229aefd60bb2c9341b53 commit: 1e6214dbd6a980b47123229aefd60bb2c9341b53 branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-08T11:42:17-07:00 summary: bpo-45952: Get the C analyzer tool working again. (gh-31219) https://bugs.python.org/issue45952 files: M Tools/c-analyzer/cpython/_parser.py diff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.py index 90b470c8196c0..752f1afe9feb0 100644 --- a/Tools/c-analyzer/cpython/_parser.py +++ b/Tools/c-analyzer/cpython/_parser.py @@ -171,6 +171,7 @@ def clean_lines(text): Objects/stringlib/unicode_format.h Py_BUILD_CORE 1 Python/ceval_gil.h Py_BUILD_CORE 1 Python/condvar.h Py_BUILD_CORE 1 +Modules/getpath_noop.c Py_BUILD_CORE 1 Modules/_json.c Py_BUILD_CORE_BUILTIN 1 Modules/_pickle.c Py_BUILD_CORE_BUILTIN 1 @@ -232,6 +233,7 @@ def clean_lines(text): # implied include of Include/**/*.h _POSIX_THREADS 1 +Include/**/*.h HAVE_PTHREAD_H 1 # from Makefile Modules/getpath.c PYTHONPATH 1 @@ -239,6 +241,7 @@ def clean_lines(text): Modules/getpath.c EXEC_PREFIX ... Modules/getpath.c VERSION ... Modules/getpath.c VPATH ... +Modules/getpath.c PLATLIBDIR ... # from Modules/_sha3/sha3module.c Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c PLATFORM_BYTE_ORDER 4321 # force big-endian From webhook-mailer at python.org Tue Feb 8 13:50:33 2022 From: webhook-mailer at python.org (gvanrossum) Date: Tue, 08 Feb 2022 18:50:33 -0000 Subject: [Python-checkins] bpo-46475: Add typing.Never and typing.assert_never (GH-30842) Message-ID: https://github.com/python/cpython/commit/243436f3779c1e7bed1fd4b23d5a8ec5eff40699 commit: 243436f3779c1e7bed1fd4b23d5a8ec5eff40699 branch: main author: Jelle Zijlstra committer: gvanrossum date: 2022-02-08T10:50:26-08:00 summary: bpo-46475: Add typing.Never and typing.assert_never (GH-30842) files: A Misc/NEWS.d/next/Library/2022-01-23-15-35-07.bpo-46475.UCe18S.rst M Doc/library/typing.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 8c1c34e90507f..94a46b01a1a8c 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -574,6 +574,34 @@ These can be used as types in annotations and do not support ``[]``. * Every type is compatible with :data:`Any`. * :data:`Any` is compatible with every type. +.. data:: Never + + The `bottom type `_, + a type that has no members. + + This can be used to define a function that should never be + called, or a function that never returns:: + + from typing import Never + + def never_call_me(arg: Never) -> None: + pass + + def int_or_str(arg: int | str) -> None: + never_call_me(arg) # type checker error + match arg: + case int(): + print("It's an int") + case str(): + print("It's a str") + case _: + never_call_me(arg) # ok, arg is of type Never + + .. versionadded:: 3.11 + + On older Python versions, :data:`NoReturn` may be used to express the + same concept. ``Never`` was added to make the intended meaning more explicit. + .. data:: NoReturn Special type indicating that a function never returns. @@ -584,6 +612,12 @@ These can be used as types in annotations and do not support ``[]``. def stop() -> NoReturn: raise RuntimeError('no way') + ``NoReturn`` can also be used as a + `bottom type `_, a type that + has no values. Starting in Python 3.11, the :data:`Never` type should + be used for this concept instead. Type checkers should treat the two + equivalently. + .. versionadded:: 3.5.4 .. versionadded:: 3.6.2 @@ -1979,6 +2013,28 @@ Functions and decorators runtime we intentionally don't check anything (we want this to be as fast as possible). +.. function:: assert_never(arg, /) + + Assert to the type checker that a line of code is unreachable. + + Example:: + + def int_or_str(arg: int | str) -> None: + match arg: + case int(): + print("It's an int") + case str(): + print("It's a str") + case _ as unreachable: + assert_never(unreachable) + + If a type checker finds that a call to ``assert_never()`` is + reachable, it will emit an error. + + At runtime, this throws an exception when called. + + .. versionadded:: 3.11 + .. function:: reveal_type(obj) Reveal the inferred static type of an expression. diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 4d1c50ce73549..78e58928332f0 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -9,7 +9,7 @@ from unittest import TestCase, main, skipUnless, skip from copy import copy, deepcopy -from typing import Any, NoReturn +from typing import Any, NoReturn, Never, assert_never from typing import TypeVar, AnyStr from typing import T, KT, VT # Not in __all__. from typing import Union, Optional, Literal @@ -124,38 +124,56 @@ def test_any_works_with_alias(self): typing.IO[Any] -class NoReturnTests(BaseTestCase): +class BottomTypeTestsMixin: + bottom_type: ClassVar[Any] - def test_noreturn_instance_type_error(self): + def test_instance_type_error(self): with self.assertRaises(TypeError): - isinstance(42, NoReturn) + isinstance(42, self.bottom_type) - def test_noreturn_subclass_type_error(self): + def test_subclass_type_error(self): with self.assertRaises(TypeError): - issubclass(Employee, NoReturn) + issubclass(Employee, self.bottom_type) with self.assertRaises(TypeError): - issubclass(NoReturn, Employee) - - def test_repr(self): - self.assertEqual(repr(NoReturn), 'typing.NoReturn') + issubclass(NoReturn, self.bottom_type) def test_not_generic(self): with self.assertRaises(TypeError): - NoReturn[int] + self.bottom_type[int] def test_cannot_subclass(self): with self.assertRaises(TypeError): - class A(NoReturn): + class A(self.bottom_type): pass with self.assertRaises(TypeError): - class A(type(NoReturn)): + class A(type(self.bottom_type)): pass def test_cannot_instantiate(self): with self.assertRaises(TypeError): - NoReturn() + self.bottom_type() with self.assertRaises(TypeError): - type(NoReturn)() + type(self.bottom_type)() + + +class NoReturnTests(BottomTypeTestsMixin, BaseTestCase): + bottom_type = NoReturn + + def test_repr(self): + self.assertEqual(repr(NoReturn), 'typing.NoReturn') + + +class NeverTests(BottomTypeTestsMixin, BaseTestCase): + bottom_type = Never + + def test_repr(self): + self.assertEqual(repr(Never), 'typing.Never') + + +class AssertNeverTests(BaseTestCase): + def test_exception(self): + with self.assertRaises(AssertionError): + assert_never(None) class SelfTests(BaseTestCase): diff --git a/Lib/typing.py b/Lib/typing.py index 0ee5c8596a021..1de48cca00d84 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -5,7 +5,7 @@ * Imports and exports, all public names should be explicitly added to __all__. * Internal helper functions: these should never be used in code outside this module. * _SpecialForm and its instances (special forms): - Any, NoReturn, ClassVar, Union, Optional, Concatenate + Any, NoReturn, Never, ClassVar, Union, Optional, Concatenate * Classes whose instances can be type arguments in addition to types: ForwardRef, TypeVar and ParamSpec * The core of internal generics API: _GenericAlias and _VariadicGenericAlias, the latter is @@ -117,12 +117,14 @@ def _idfunc(_, x): # One-off things. 'AnyStr', + 'assert_never', 'cast', 'final', 'get_args', 'get_origin', 'get_type_hints', 'is_typeddict', + 'Never', 'NewType', 'no_type_check', 'no_type_check_decorator', @@ -175,7 +177,7 @@ def _type_check(arg, msg, is_argument=True, module=None, *, allow_special_forms= if (isinstance(arg, _GenericAlias) and arg.__origin__ in invalid_generic_forms): raise TypeError(f"{arg} is not valid as type argument") - if arg in (Any, NoReturn, Self, ClassVar, Final, TypeAlias): + if arg in (Any, NoReturn, Never, Self, ClassVar, Final, TypeAlias): return arg if isinstance(arg, _SpecialForm) or arg in (Generic, Protocol): raise TypeError(f"Plain {arg} is not valid as type argument") @@ -441,8 +443,39 @@ def NoReturn(self, parameters): def stop() -> NoReturn: raise Exception('no way') - This type is invalid in other positions, e.g., ``List[NoReturn]`` - will fail in static type checkers. + NoReturn can also be used as a bottom type, a type that + has no values. Starting in Python 3.11, the Never type should + be used for this concept instead. Type checkers should treat the two + equivalently. + + """ + raise TypeError(f"{self} is not subscriptable") + +# This is semantically identical to NoReturn, but it is implemented +# separately so that type checkers can distinguish between the two +# if they want. + at _SpecialForm +def Never(self, parameters): + """The bottom type, a type that has no members. + + This can be used to define a function that should never be + called, or a function that never returns:: + + from typing import Never + + def never_call_me(arg: Never) -> None: + pass + + def int_or_str(arg: int | str) -> None: + never_call_me(arg) # type checker error + match arg: + case int(): + print("It's an int") + case str(): + print("It's a str") + case _: + never_call_me(arg) # ok, arg is of type Never + """ raise TypeError(f"{self} is not subscriptable") @@ -2060,6 +2093,29 @@ class Film(TypedDict): return isinstance(tp, _TypedDictMeta) +def assert_never(arg: Never, /) -> Never: + """Statically assert that a line of code is unreachable. + + Example:: + + def int_or_str(arg: int | str) -> None: + match arg: + case int(): + print("It's an int") + case str(): + print("It's a str") + case _: + assert_never(arg) + + If a type checker finds that a call to assert_never() is + reachable, it will emit an error. + + At runtime, this throws an exception when called. + + """ + raise AssertionError("Expected code to be unreachable") + + def no_type_check(arg): """Decorator to indicate that annotations are not type hints. diff --git a/Misc/NEWS.d/next/Library/2022-01-23-15-35-07.bpo-46475.UCe18S.rst b/Misc/NEWS.d/next/Library/2022-01-23-15-35-07.bpo-46475.UCe18S.rst new file mode 100644 index 0000000000000..99d5e2b42c4f6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-23-15-35-07.bpo-46475.UCe18S.rst @@ -0,0 +1,2 @@ +Add :data:`typing.Never` and :func:`typing.assert_never`. Patch by Jelle +Zijlstra. From webhook-mailer at python.org Tue Feb 8 14:38:01 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Tue, 08 Feb 2022 19:38:01 -0000 Subject: [Python-checkins] bpo-45952: Get the C analyzer tool working again. (gh-31220) Message-ID: https://github.com/python/cpython/commit/c018d3037b5b62e6d48d5985d1a37b91762fbffb commit: c018d3037b5b62e6d48d5985d1a37b91762fbffb branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-08T12:37:53-07:00 summary: bpo-45952: Get the C analyzer tool working again. (gh-31220) https://bugs.python.org/issue45952 files: M Tools/c-analyzer/cpython/_parser.py diff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.py index 752f1afe9feb0..6b55c41c2c71e 100644 --- a/Tools/c-analyzer/cpython/_parser.py +++ b/Tools/c-analyzer/cpython/_parser.py @@ -215,6 +215,7 @@ def clean_lines(text): Modules/_testcapimodule.c PyAPI_FUNC(RTYPE) RTYPE Python/getargs.c PyAPI_FUNC(RTYPE) RTYPE Objects/stringlib/unicode_format.h Py_LOCAL_INLINE(type) static inline type +Include/pymath.h _Py__has_builtin(x) 0 # implied include of pymacro.h */clinic/*.c.h PyDoc_VAR(name) static const char name[] From webhook-mailer at python.org Tue Feb 8 15:39:36 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Tue, 08 Feb 2022 20:39:36 -0000 Subject: [Python-checkins] bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928) Message-ID: https://github.com/python/cpython/commit/81c72044a181dbbfbf689d7a977d0d99090f26a8 commit: 81c72044a181dbbfbf689d7a977d0d99090f26a8 branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-08T13:39:07-07:00 summary: bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928) We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules. The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings). https://bugs.python.org/issue46541#msg411799 explains the rationale for this change. The core of the change is in: * (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros * Include/internal/pycore_runtime_init.h - added the static initializers for the global strings * Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState * Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config. The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *. The following are not changed (yet): * stop using _Py_IDENTIFIER() in the stdlib modules * (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API * (maybe) intern the strings during runtime init https://bugs.python.org/issue46541 files: A Include/internal/pycore_global_strings.h M .github/workflows/build.yml M Include/cpython/abstract.h M Include/cpython/ceval.h M Include/cpython/dictobject.h M Include/cpython/object.h M Include/cpython/sysmodule.h M Include/internal/pycore_call.h M Include/internal/pycore_global_objects.h M Include/internal/pycore_object.h M Include/internal/pycore_runtime_init.h M Include/internal/pycore_structseq.h M Include/internal/pycore_sysmodule.h M Include/internal/pycore_unicodeobject.h M Makefile.pre.in M Modules/_abc.c M Modules/_asynciomodule.c M Modules/_bisectmodule.c M Modules/_collectionsmodule.c M Modules/_csv.c M Modules/_ctypes/_ctypes.c M Modules/_ctypes/callbacks.c M Modules/_ctypes/callproc.c M Modules/_ctypes/stgdict.c M Modules/_cursesmodule.c M Modules/_datetimemodule.c M Modules/_dbmmodule.c M Modules/_elementtree.c M Modules/_gdbmmodule.c M Modules/_io/_iomodule.c M Modules/_io/bufferedio.c M Modules/_io/fileio.c M Modules/_io/iobase.c M Modules/_io/textio.c M Modules/_io/winconsoleio.c M Modules/_json.c M Modules/_lzmamodule.c M Modules/_operator.c M Modules/_pickle.c M Modules/_sqlite/connection.c M Modules/_sqlite/cursor.c M Modules/_sqlite/microprotocols.c M Modules/_sqlite/module.c M Modules/_ssl.c M Modules/_testcapimodule.c M Modules/_threadmodule.c M Modules/arraymodule.c M Modules/cjkcodecs/multibytecodec.c M Modules/faulthandler.c M Modules/itertoolsmodule.c M Modules/main.c M Modules/mathmodule.c M Modules/mmapmodule.c M Modules/ossaudiodev.c M Modules/posixmodule.c M Modules/pyexpat.c M Modules/timemodule.c M Modules/unicodedata.c M Objects/abstract.c M Objects/bytearrayobject.c M Objects/bytesobject.c M Objects/call.c M Objects/classobject.c M Objects/complexobject.c M Objects/descrobject.c M Objects/dictobject.c M Objects/enumobject.c M Objects/exceptions.c M Objects/fileobject.c M Objects/frameobject.c M Objects/funcobject.c M Objects/genericaliasobject.c M Objects/genobject.c M Objects/iterobject.c M Objects/listobject.c M Objects/longobject.c M Objects/methodobject.c M Objects/moduleobject.c M Objects/object.c M Objects/odictobject.c M Objects/rangeobject.c M Objects/setobject.c M Objects/structseq.c M Objects/tupleobject.c M Objects/typeobject.c M Objects/unicodeobject.c M Objects/unionobject.c M Objects/weakrefobject.c M Parser/tokenizer.c M Programs/_testembed.c M Programs/test_frozenmain.h M Python/_warnings.c M Python/ast_opt.c M Python/bltinmodule.c M Python/ceval.c M Python/codecs.c M Python/compile.c M Python/errors.c M Python/import.c M Python/importdl.c M Python/marshal.c M Python/pylifecycle.c M Python/pythonrun.c M Python/specialize.c M Python/sysmodule.c M Python/traceback.c M Tools/c-analyzer/TODO M Tools/scripts/generate_global_objects.py diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 5d36dffa80108..376359ccf2e2d 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -100,6 +100,8 @@ jobs: run: make smelly - name: Check limited ABI symbols run: make check-limited-abi + - name: Check global objects + run: make check-global-objects build_win32: name: 'Windows (x86)' diff --git a/Include/cpython/abstract.h b/Include/cpython/abstract.h index b5a31392f2ed0..cf142b8a0bfb2 100644 --- a/Include/cpython/abstract.h +++ b/Include/cpython/abstract.h @@ -116,6 +116,10 @@ PyObject_CallMethodOneArg(PyObject *self, PyObject *name, PyObject *arg) 2 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); } +PyAPI_FUNC(PyObject *) _PyObject_CallMethod(PyObject *obj, + PyObject *name, + const char *format, ...); + /* Like PyObject_CallMethod(), but expect a _Py_Identifier* as the method name. */ PyAPI_FUNC(PyObject *) _PyObject_CallMethodId(PyObject *obj, diff --git a/Include/cpython/ceval.h b/Include/cpython/ceval.h index caf64401307c0..aedc7367a18f6 100644 --- a/Include/cpython/ceval.h +++ b/Include/cpython/ceval.h @@ -15,6 +15,7 @@ PyAPI_FUNC(int) _PyEval_SetAsyncGenFinalizer(PyObject *); PyAPI_FUNC(PyObject *) _PyEval_GetAsyncGenFinalizer(void); /* Helper to look up a builtin object */ +PyAPI_FUNC(PyObject *) _PyEval_GetBuiltin(PyObject *); PyAPI_FUNC(PyObject *) _PyEval_GetBuiltinId(_Py_Identifier *); /* Look at the current frame's (if any) code's co_flags, and turn on the corresponding compiler flags in cf->cf_flags. Return 1 if any diff --git a/Include/cpython/dictobject.h b/Include/cpython/dictobject.h index e97969be4de0c..d5119644c5212 100644 --- a/Include/cpython/dictobject.h +++ b/Include/cpython/dictobject.h @@ -30,6 +30,7 @@ typedef struct { PyAPI_FUNC(PyObject *) _PyDict_GetItem_KnownHash(PyObject *mp, PyObject *key, Py_hash_t hash); +PyAPI_FUNC(PyObject *) _PyDict_GetItemWithError(PyObject *dp, PyObject *key); PyAPI_FUNC(PyObject *) _PyDict_GetItemIdWithError(PyObject *dp, struct _Py_Identifier *key); PyAPI_FUNC(PyObject *) _PyDict_GetItemStringWithError(PyObject *, const char *); diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 1554ac8aef1c4..a3632cf1dcf81 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -43,10 +43,16 @@ typedef struct _Py_Identifier { Py_ssize_t index; } _Py_Identifier; +#if defined(NEEDS_PY_IDENTIFIER) || !defined(Py_BUILD_CORE) +// For now we are keeping _Py_IDENTIFIER for continued use +// in non-builtin extensions (and naughty PyPI modules). + #define _Py_static_string_init(value) { .string = value, .index = -1 } #define _Py_static_string(varname, value) static _Py_Identifier varname = _Py_static_string_init(value) #define _Py_IDENTIFIER(varname) _Py_static_string(PyId_##varname, #varname) +#endif /* NEEDS_PY_IDENTIFIER */ + typedef int (*getbufferproc)(PyObject *, Py_buffer *, int); typedef void (*releasebufferproc)(PyObject *, Py_buffer *); @@ -249,7 +255,12 @@ typedef struct _heaptypeobject { PyAPI_FUNC(const char *) _PyType_Name(PyTypeObject *); PyAPI_FUNC(PyObject *) _PyType_Lookup(PyTypeObject *, PyObject *); PyAPI_FUNC(PyObject *) _PyType_LookupId(PyTypeObject *, _Py_Identifier *); -PyAPI_FUNC(PyObject *) _PyObject_LookupSpecial(PyObject *, _Py_Identifier *); +PyAPI_FUNC(PyObject *) _PyObject_LookupSpecialId(PyObject *, _Py_Identifier *); +#ifndef Py_BUILD_CORE +// Backward compatibility for 3rd-party extensions +// that may be using the old name. +#define _PyObject_LookupSpecial _PyObject_LookupSpecialId +#endif PyAPI_FUNC(PyTypeObject *) _PyType_CalculateMetaclass(PyTypeObject *, PyObject *); PyAPI_FUNC(PyObject *) _PyType_GetDocFromInternalDoc(const char *, const char *); PyAPI_FUNC(PyObject *) _PyType_GetTextSignatureFromInternalDoc(const char *, const char *); diff --git a/Include/cpython/sysmodule.h b/Include/cpython/sysmodule.h index fc4c899b3fe3d..27dff7b2e3d93 100644 --- a/Include/cpython/sysmodule.h +++ b/Include/cpython/sysmodule.h @@ -2,6 +2,8 @@ # error "this header file must not be included directly" #endif +PyAPI_FUNC(PyObject *) _PySys_GetAttr(PyThreadState *tstate, + PyObject *name); PyAPI_FUNC(PyObject *) _PySys_GetObjectId(_Py_Identifier *key); PyAPI_FUNC(int) _PySys_SetObjectId(_Py_Identifier *key, PyObject *); diff --git a/Include/internal/pycore_call.h b/Include/internal/pycore_call.h index f2cfd2fd53ed9..3ccacfa0b8b03 100644 --- a/Include/internal/pycore_call.h +++ b/Include/internal/pycore_call.h @@ -30,6 +30,9 @@ PyAPI_FUNC(PyObject *) _PyObject_Call( PyObject *args, PyObject *kwargs); +extern PyObject * _PyObject_CallMethodFormat( + PyThreadState *tstate, PyObject *callable, const char *format, ...); + // Static inline variant of public PyVectorcall_Function(). static inline vectorcallfunc diff --git a/Include/internal/pycore_global_objects.h b/Include/internal/pycore_global_objects.h index acf0b2aed5ae4..2135fa3b5d79a 100644 --- a/Include/internal/pycore_global_objects.h +++ b/Include/internal/pycore_global_objects.h @@ -8,6 +8,8 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +#include "pycore_global_strings.h" // struct _Py_global_strings + // These would be in pycore_long.h if it weren't for an include cycle. #define _PY_NSMALLPOSINTS 257 @@ -36,6 +38,8 @@ struct _Py_global_objects { PyBytesObject ob; char eos; } bytes_characters[256]; + + struct _Py_global_strings strings; } singletons; }; diff --git a/Include/internal/pycore_global_strings.h b/Include/internal/pycore_global_strings.h new file mode 100644 index 0000000000000..17241b3a3dd16 --- /dev/null +++ b/Include/internal/pycore_global_strings.h @@ -0,0 +1,337 @@ +#ifndef Py_INTERNAL_GLOBAL_STRINGS_H +#define Py_INTERNAL_GLOBAL_STRINGS_H +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef Py_BUILD_CORE +# error "this header requires Py_BUILD_CORE define" +#endif + +// The data structure & init here are inspired by Tools/scripts/deepfreeze.py. + +// All field names generated by ASCII_STR() have a common prefix, +// to help avoid collisions with keywords, etc. + +#define STRUCT_FOR_ASCII_STR(LITERAL) \ + struct { \ + PyASCIIObject _ascii; \ + uint8_t _data[sizeof(LITERAL)]; \ + } +#define STRUCT_FOR_STR(NAME, LITERAL) \ + STRUCT_FOR_ASCII_STR(LITERAL) _ ## NAME; +#define STRUCT_FOR_ID(NAME) \ + STRUCT_FOR_ASCII_STR(#NAME) _ ## NAME; + +// XXX Order by frequency of use? + +/* The following is auto-generated by Tools/scripts/generate_global_objects.py. */ +struct _Py_global_strings { + struct { + STRUCT_FOR_STR(empty, "") + STRUCT_FOR_STR(dot, ".") + STRUCT_FOR_STR(comma_sep, ", ") + STRUCT_FOR_STR(percent, "%") + STRUCT_FOR_STR(dbl_percent, "%%") + + // "anonymous" labels + STRUCT_FOR_STR(anon_dictcomp, "") + STRUCT_FOR_STR(anon_genexpr, "") + STRUCT_FOR_STR(anon_lambda, "") + STRUCT_FOR_STR(anon_listcomp, "") + STRUCT_FOR_STR(anon_module, "") + STRUCT_FOR_STR(anon_setcomp, "") + STRUCT_FOR_STR(anon_string, "") + STRUCT_FOR_STR(dot_locals, ".") + } literals; + + struct { + STRUCT_FOR_ID(Py_Repr) + STRUCT_FOR_ID(TextIOWrapper) + STRUCT_FOR_ID(WarningMessage) + STRUCT_FOR_ID(_) + STRUCT_FOR_ID(__IOBase_closed) + STRUCT_FOR_ID(__abc_tpflags__) + STRUCT_FOR_ID(__abs__) + STRUCT_FOR_ID(__abstractmethods__) + STRUCT_FOR_ID(__add__) + STRUCT_FOR_ID(__aenter__) + STRUCT_FOR_ID(__aexit__) + STRUCT_FOR_ID(__aiter__) + STRUCT_FOR_ID(__all__) + STRUCT_FOR_ID(__and__) + STRUCT_FOR_ID(__anext__) + STRUCT_FOR_ID(__annotations__) + STRUCT_FOR_ID(__args__) + STRUCT_FOR_ID(__await__) + STRUCT_FOR_ID(__bases__) + STRUCT_FOR_ID(__bool__) + STRUCT_FOR_ID(__build_class__) + STRUCT_FOR_ID(__builtins__) + STRUCT_FOR_ID(__bytes__) + STRUCT_FOR_ID(__call__) + STRUCT_FOR_ID(__cantrace__) + STRUCT_FOR_ID(__class__) + STRUCT_FOR_ID(__class_getitem__) + STRUCT_FOR_ID(__classcell__) + STRUCT_FOR_ID(__complex__) + STRUCT_FOR_ID(__contains__) + STRUCT_FOR_ID(__copy__) + STRUCT_FOR_ID(__del__) + STRUCT_FOR_ID(__delattr__) + STRUCT_FOR_ID(__delete__) + STRUCT_FOR_ID(__delitem__) + STRUCT_FOR_ID(__dict__) + STRUCT_FOR_ID(__dir__) + STRUCT_FOR_ID(__divmod__) + STRUCT_FOR_ID(__doc__) + STRUCT_FOR_ID(__enter__) + STRUCT_FOR_ID(__eq__) + STRUCT_FOR_ID(__exit__) + STRUCT_FOR_ID(__file__) + STRUCT_FOR_ID(__float__) + STRUCT_FOR_ID(__floordiv__) + STRUCT_FOR_ID(__format__) + STRUCT_FOR_ID(__fspath__) + STRUCT_FOR_ID(__ge__) + STRUCT_FOR_ID(__get__) + STRUCT_FOR_ID(__getattr__) + STRUCT_FOR_ID(__getattribute__) + STRUCT_FOR_ID(__getinitargs__) + STRUCT_FOR_ID(__getitem__) + STRUCT_FOR_ID(__getnewargs__) + STRUCT_FOR_ID(__getnewargs_ex__) + STRUCT_FOR_ID(__getstate__) + STRUCT_FOR_ID(__gt__) + STRUCT_FOR_ID(__hash__) + STRUCT_FOR_ID(__iadd__) + STRUCT_FOR_ID(__iand__) + STRUCT_FOR_ID(__ifloordiv__) + STRUCT_FOR_ID(__ilshift__) + STRUCT_FOR_ID(__imatmul__) + STRUCT_FOR_ID(__imod__) + STRUCT_FOR_ID(__import__) + STRUCT_FOR_ID(__imul__) + STRUCT_FOR_ID(__index__) + STRUCT_FOR_ID(__init__) + STRUCT_FOR_ID(__init_subclass__) + STRUCT_FOR_ID(__instancecheck__) + STRUCT_FOR_ID(__int__) + STRUCT_FOR_ID(__invert__) + STRUCT_FOR_ID(__ior__) + STRUCT_FOR_ID(__ipow__) + STRUCT_FOR_ID(__irshift__) + STRUCT_FOR_ID(__isabstractmethod__) + STRUCT_FOR_ID(__isub__) + STRUCT_FOR_ID(__iter__) + STRUCT_FOR_ID(__itruediv__) + STRUCT_FOR_ID(__ixor__) + STRUCT_FOR_ID(__le__) + STRUCT_FOR_ID(__len__) + STRUCT_FOR_ID(__length_hint__) + STRUCT_FOR_ID(__loader__) + STRUCT_FOR_ID(__lshift__) + STRUCT_FOR_ID(__lt__) + STRUCT_FOR_ID(__ltrace__) + STRUCT_FOR_ID(__main__) + STRUCT_FOR_ID(__matmul__) + STRUCT_FOR_ID(__missing__) + STRUCT_FOR_ID(__mod__) + STRUCT_FOR_ID(__module__) + STRUCT_FOR_ID(__mro_entries__) + STRUCT_FOR_ID(__mul__) + STRUCT_FOR_ID(__name__) + STRUCT_FOR_ID(__ne__) + STRUCT_FOR_ID(__neg__) + STRUCT_FOR_ID(__new__) + STRUCT_FOR_ID(__newobj__) + STRUCT_FOR_ID(__newobj_ex__) + STRUCT_FOR_ID(__next__) + STRUCT_FOR_ID(__note__) + STRUCT_FOR_ID(__or__) + STRUCT_FOR_ID(__origin__) + STRUCT_FOR_ID(__package__) + STRUCT_FOR_ID(__parameters__) + STRUCT_FOR_ID(__path__) + STRUCT_FOR_ID(__pos__) + STRUCT_FOR_ID(__pow__) + STRUCT_FOR_ID(__prepare__) + STRUCT_FOR_ID(__qualname__) + STRUCT_FOR_ID(__radd__) + STRUCT_FOR_ID(__rand__) + STRUCT_FOR_ID(__rdivmod__) + STRUCT_FOR_ID(__reduce__) + STRUCT_FOR_ID(__reduce_ex__) + STRUCT_FOR_ID(__repr__) + STRUCT_FOR_ID(__reversed__) + STRUCT_FOR_ID(__rfloordiv__) + STRUCT_FOR_ID(__rlshift__) + STRUCT_FOR_ID(__rmatmul__) + STRUCT_FOR_ID(__rmod__) + STRUCT_FOR_ID(__rmul__) + STRUCT_FOR_ID(__ror__) + STRUCT_FOR_ID(__round__) + STRUCT_FOR_ID(__rpow__) + STRUCT_FOR_ID(__rrshift__) + STRUCT_FOR_ID(__rshift__) + STRUCT_FOR_ID(__rsub__) + STRUCT_FOR_ID(__rtruediv__) + STRUCT_FOR_ID(__rxor__) + STRUCT_FOR_ID(__set__) + STRUCT_FOR_ID(__set_name__) + STRUCT_FOR_ID(__setattr__) + STRUCT_FOR_ID(__setitem__) + STRUCT_FOR_ID(__setstate__) + STRUCT_FOR_ID(__sizeof__) + STRUCT_FOR_ID(__slotnames__) + STRUCT_FOR_ID(__slots__) + STRUCT_FOR_ID(__spec__) + STRUCT_FOR_ID(__str__) + STRUCT_FOR_ID(__sub__) + STRUCT_FOR_ID(__subclasscheck__) + STRUCT_FOR_ID(__subclasshook__) + STRUCT_FOR_ID(__truediv__) + STRUCT_FOR_ID(__trunc__) + STRUCT_FOR_ID(__warningregistry__) + STRUCT_FOR_ID(__weakref__) + STRUCT_FOR_ID(__xor__) + STRUCT_FOR_ID(_abc_impl) + STRUCT_FOR_ID(_blksize) + STRUCT_FOR_ID(_dealloc_warn) + STRUCT_FOR_ID(_finalizing) + STRUCT_FOR_ID(_find_and_load) + STRUCT_FOR_ID(_fix_up_module) + STRUCT_FOR_ID(_get_sourcefile) + STRUCT_FOR_ID(_handle_fromlist) + STRUCT_FOR_ID(_initializing) + STRUCT_FOR_ID(_is_text_encoding) + STRUCT_FOR_ID(_lock_unlock_module) + STRUCT_FOR_ID(_showwarnmsg) + STRUCT_FOR_ID(_shutdown) + STRUCT_FOR_ID(_slotnames) + STRUCT_FOR_ID(_strptime_time) + STRUCT_FOR_ID(_uninitialized_submodules) + STRUCT_FOR_ID(_warn_unawaited_coroutine) + STRUCT_FOR_ID(_xoptions) + STRUCT_FOR_ID(add) + STRUCT_FOR_ID(append) + STRUCT_FOR_ID(big) + STRUCT_FOR_ID(buffer) + STRUCT_FOR_ID(builtins) + STRUCT_FOR_ID(clear) + STRUCT_FOR_ID(close) + STRUCT_FOR_ID(code) + STRUCT_FOR_ID(copy) + STRUCT_FOR_ID(copyreg) + STRUCT_FOR_ID(decode) + STRUCT_FOR_ID(default) + STRUCT_FOR_ID(defaultaction) + STRUCT_FOR_ID(difference_update) + STRUCT_FOR_ID(dispatch_table) + STRUCT_FOR_ID(displayhook) + STRUCT_FOR_ID(enable) + STRUCT_FOR_ID(encoding) + STRUCT_FOR_ID(end_lineno) + STRUCT_FOR_ID(end_offset) + STRUCT_FOR_ID(errors) + STRUCT_FOR_ID(excepthook) + STRUCT_FOR_ID(extend) + STRUCT_FOR_ID(filename) + STRUCT_FOR_ID(fileno) + STRUCT_FOR_ID(fillvalue) + STRUCT_FOR_ID(filters) + STRUCT_FOR_ID(find_class) + STRUCT_FOR_ID(flush) + STRUCT_FOR_ID(get) + STRUCT_FOR_ID(get_source) + STRUCT_FOR_ID(getattr) + STRUCT_FOR_ID(ignore) + STRUCT_FOR_ID(importlib) + STRUCT_FOR_ID(intersection) + STRUCT_FOR_ID(isatty) + STRUCT_FOR_ID(items) + STRUCT_FOR_ID(iter) + STRUCT_FOR_ID(keys) + STRUCT_FOR_ID(last_traceback) + STRUCT_FOR_ID(last_type) + STRUCT_FOR_ID(last_value) + STRUCT_FOR_ID(latin1) + STRUCT_FOR_ID(lineno) + STRUCT_FOR_ID(little) + STRUCT_FOR_ID(match) + STRUCT_FOR_ID(metaclass) + STRUCT_FOR_ID(mode) + STRUCT_FOR_ID(modules) + STRUCT_FOR_ID(mro) + STRUCT_FOR_ID(msg) + STRUCT_FOR_ID(n_fields) + STRUCT_FOR_ID(n_sequence_fields) + STRUCT_FOR_ID(n_unnamed_fields) + STRUCT_FOR_ID(name) + STRUCT_FOR_ID(obj) + STRUCT_FOR_ID(offset) + STRUCT_FOR_ID(onceregistry) + STRUCT_FOR_ID(open) + STRUCT_FOR_ID(parent) + STRUCT_FOR_ID(partial) + STRUCT_FOR_ID(path) + STRUCT_FOR_ID(peek) + STRUCT_FOR_ID(persistent_id) + STRUCT_FOR_ID(persistent_load) + STRUCT_FOR_ID(print_file_and_line) + STRUCT_FOR_ID(ps1) + STRUCT_FOR_ID(ps2) + STRUCT_FOR_ID(raw) + STRUCT_FOR_ID(read) + STRUCT_FOR_ID(read1) + STRUCT_FOR_ID(readable) + STRUCT_FOR_ID(readall) + STRUCT_FOR_ID(readinto) + STRUCT_FOR_ID(readinto1) + STRUCT_FOR_ID(readline) + STRUCT_FOR_ID(reducer_override) + STRUCT_FOR_ID(reload) + STRUCT_FOR_ID(replace) + STRUCT_FOR_ID(reset) + STRUCT_FOR_ID(return) + STRUCT_FOR_ID(reversed) + STRUCT_FOR_ID(seek) + STRUCT_FOR_ID(seekable) + STRUCT_FOR_ID(send) + STRUCT_FOR_ID(setstate) + STRUCT_FOR_ID(sort) + STRUCT_FOR_ID(stderr) + STRUCT_FOR_ID(stdin) + STRUCT_FOR_ID(stdout) + STRUCT_FOR_ID(strict) + STRUCT_FOR_ID(symmetric_difference_update) + STRUCT_FOR_ID(tell) + STRUCT_FOR_ID(text) + STRUCT_FOR_ID(threading) + STRUCT_FOR_ID(throw) + STRUCT_FOR_ID(unraisablehook) + STRUCT_FOR_ID(values) + STRUCT_FOR_ID(version) + STRUCT_FOR_ID(warnings) + STRUCT_FOR_ID(warnoptions) + STRUCT_FOR_ID(writable) + STRUCT_FOR_ID(write) + STRUCT_FOR_ID(zipimporter) + } identifiers; +}; +/* End auto-generated code */ + +#undef ID +#undef STR + + +#define _Py_ID(NAME) \ + (_Py_SINGLETON(strings.identifiers._ ## NAME._ascii.ob_base)) +#define _Py_STR(NAME) \ + (_Py_SINGLETON(strings.literals._ ## NAME._ascii.ob_base)) + + +#ifdef __cplusplus +} +#endif +#endif /* !Py_INTERNAL_GLOBAL_STRINGS_H */ diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h index e2da2537c11c1..65abc1884c3bb 100644 --- a/Include/internal/pycore_object.h +++ b/Include/internal/pycore_object.h @@ -8,9 +8,11 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +#include #include "pycore_gc.h" // _PyObject_GC_IS_TRACKED() #include "pycore_interp.h" // PyInterpreterState.gc #include "pycore_pystate.h" // _PyInterpreterState_GET() +#include "pycore_runtime.h" // _PyRuntime #define _PyObject_IMMORTAL_INIT(type) \ @@ -236,6 +238,8 @@ extern PyObject* _PyType_GetSubclasses(PyTypeObject *); #define _PyHeapType_GET_MEMBERS(etype) \ ((PyMemberDef *)(((char *)etype) + Py_TYPE(etype)->tp_basicsize)) +PyAPI_FUNC(PyObject *) _PyObject_LookupSpecial(PyObject *, PyObject *); + #ifdef __cplusplus } #endif diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h index 3b7f2629033bf..045ae5d2835b1 100644 --- a/Include/internal/pycore_runtime_init.h +++ b/Include/internal/pycore_runtime_init.h @@ -82,7 +82,6 @@ extern "C" { .ob_digit = { ((val) >= 0 ? (val) : -(val)) }, \ } - #define _PyBytes_SIMPLE_INIT(CH, LEN) \ { \ _PyVarObject_IMMORTAL_INIT(&PyBytes_Type, LEN), \ @@ -94,6 +93,26 @@ extern "C" { _PyBytes_SIMPLE_INIT(CH, 1) \ } +#define _PyASCIIObject_INIT(LITERAL) \ + { \ + ._ascii = { \ + .ob_base = _PyObject_IMMORTAL_INIT(&PyUnicode_Type), \ + .length = sizeof(LITERAL) - 1, \ + .hash = -1, \ + .state = { \ + .kind = 1, \ + .compact = 1, \ + .ascii = 1, \ + .ready = 1, \ + }, \ + }, \ + ._data = LITERAL, \ + } +#define INIT_STR(NAME, LITERAL) \ + ._ ## NAME = _PyASCIIObject_INIT(LITERAL) +#define INIT_ID(NAME) \ + ._ ## NAME = _PyASCIIObject_INIT(#NAME) + /* The following is auto-generated by Tools/scripts/generate_global_objects.py. */ #define _Py_global_objects_INIT { \ @@ -622,6 +641,298 @@ extern "C" { _PyBytes_CHAR_INIT(254), \ _PyBytes_CHAR_INIT(255), \ }, \ + \ + .strings = { \ + .literals = { \ + INIT_STR(empty, ""), \ + INIT_STR(dot, "."), \ + INIT_STR(comma_sep, ", "), \ + INIT_STR(percent, "%"), \ + INIT_STR(dbl_percent, "%%"), \ + \ + INIT_STR(anon_dictcomp, ""), \ + INIT_STR(anon_genexpr, ""), \ + INIT_STR(anon_lambda, ""), \ + INIT_STR(anon_listcomp, ""), \ + INIT_STR(anon_module, ""), \ + INIT_STR(anon_setcomp, ""), \ + INIT_STR(anon_string, ""), \ + INIT_STR(dot_locals, "."), \ + }, \ + .identifiers = { \ + INIT_ID(Py_Repr), \ + INIT_ID(TextIOWrapper), \ + INIT_ID(WarningMessage), \ + INIT_ID(_), \ + INIT_ID(__IOBase_closed), \ + INIT_ID(__abc_tpflags__), \ + INIT_ID(__abs__), \ + INIT_ID(__abstractmethods__), \ + INIT_ID(__add__), \ + INIT_ID(__aenter__), \ + INIT_ID(__aexit__), \ + INIT_ID(__aiter__), \ + INIT_ID(__all__), \ + INIT_ID(__and__), \ + INIT_ID(__anext__), \ + INIT_ID(__annotations__), \ + INIT_ID(__args__), \ + INIT_ID(__await__), \ + INIT_ID(__bases__), \ + INIT_ID(__bool__), \ + INIT_ID(__build_class__), \ + INIT_ID(__builtins__), \ + INIT_ID(__bytes__), \ + INIT_ID(__call__), \ + INIT_ID(__cantrace__), \ + INIT_ID(__class__), \ + INIT_ID(__class_getitem__), \ + INIT_ID(__classcell__), \ + INIT_ID(__complex__), \ + INIT_ID(__contains__), \ + INIT_ID(__copy__), \ + INIT_ID(__del__), \ + INIT_ID(__delattr__), \ + INIT_ID(__delete__), \ + INIT_ID(__delitem__), \ + INIT_ID(__dict__), \ + INIT_ID(__dir__), \ + INIT_ID(__divmod__), \ + INIT_ID(__doc__), \ + INIT_ID(__enter__), \ + INIT_ID(__eq__), \ + INIT_ID(__exit__), \ + INIT_ID(__file__), \ + INIT_ID(__float__), \ + INIT_ID(__floordiv__), \ + INIT_ID(__format__), \ + INIT_ID(__fspath__), \ + INIT_ID(__ge__), \ + INIT_ID(__get__), \ + INIT_ID(__getattr__), \ + INIT_ID(__getattribute__), \ + INIT_ID(__getinitargs__), \ + INIT_ID(__getitem__), \ + INIT_ID(__getnewargs__), \ + INIT_ID(__getnewargs_ex__), \ + INIT_ID(__getstate__), \ + INIT_ID(__gt__), \ + INIT_ID(__hash__), \ + INIT_ID(__iadd__), \ + INIT_ID(__iand__), \ + INIT_ID(__ifloordiv__), \ + INIT_ID(__ilshift__), \ + INIT_ID(__imatmul__), \ + INIT_ID(__imod__), \ + INIT_ID(__import__), \ + INIT_ID(__imul__), \ + INIT_ID(__index__), \ + INIT_ID(__init__), \ + INIT_ID(__init_subclass__), \ + INIT_ID(__instancecheck__), \ + INIT_ID(__int__), \ + INIT_ID(__invert__), \ + INIT_ID(__ior__), \ + INIT_ID(__ipow__), \ + INIT_ID(__irshift__), \ + INIT_ID(__isabstractmethod__), \ + INIT_ID(__isub__), \ + INIT_ID(__iter__), \ + INIT_ID(__itruediv__), \ + INIT_ID(__ixor__), \ + INIT_ID(__le__), \ + INIT_ID(__len__), \ + INIT_ID(__length_hint__), \ + INIT_ID(__loader__), \ + INIT_ID(__lshift__), \ + INIT_ID(__lt__), \ + INIT_ID(__ltrace__), \ + INIT_ID(__main__), \ + INIT_ID(__matmul__), \ + INIT_ID(__missing__), \ + INIT_ID(__mod__), \ + INIT_ID(__module__), \ + INIT_ID(__mro_entries__), \ + INIT_ID(__mul__), \ + INIT_ID(__name__), \ + INIT_ID(__ne__), \ + INIT_ID(__neg__), \ + INIT_ID(__new__), \ + INIT_ID(__newobj__), \ + INIT_ID(__newobj_ex__), \ + INIT_ID(__next__), \ + INIT_ID(__note__), \ + INIT_ID(__or__), \ + INIT_ID(__origin__), \ + INIT_ID(__package__), \ + INIT_ID(__parameters__), \ + INIT_ID(__path__), \ + INIT_ID(__pos__), \ + INIT_ID(__pow__), \ + INIT_ID(__prepare__), \ + INIT_ID(__qualname__), \ + INIT_ID(__radd__), \ + INIT_ID(__rand__), \ + INIT_ID(__rdivmod__), \ + INIT_ID(__reduce__), \ + INIT_ID(__reduce_ex__), \ + INIT_ID(__repr__), \ + INIT_ID(__reversed__), \ + INIT_ID(__rfloordiv__), \ + INIT_ID(__rlshift__), \ + INIT_ID(__rmatmul__), \ + INIT_ID(__rmod__), \ + INIT_ID(__rmul__), \ + INIT_ID(__ror__), \ + INIT_ID(__round__), \ + INIT_ID(__rpow__), \ + INIT_ID(__rrshift__), \ + INIT_ID(__rshift__), \ + INIT_ID(__rsub__), \ + INIT_ID(__rtruediv__), \ + INIT_ID(__rxor__), \ + INIT_ID(__set__), \ + INIT_ID(__set_name__), \ + INIT_ID(__setattr__), \ + INIT_ID(__setitem__), \ + INIT_ID(__setstate__), \ + INIT_ID(__sizeof__), \ + INIT_ID(__slotnames__), \ + INIT_ID(__slots__), \ + INIT_ID(__spec__), \ + INIT_ID(__str__), \ + INIT_ID(__sub__), \ + INIT_ID(__subclasscheck__), \ + INIT_ID(__subclasshook__), \ + INIT_ID(__truediv__), \ + INIT_ID(__trunc__), \ + INIT_ID(__warningregistry__), \ + INIT_ID(__weakref__), \ + INIT_ID(__xor__), \ + INIT_ID(_abc_impl), \ + INIT_ID(_blksize), \ + INIT_ID(_dealloc_warn), \ + INIT_ID(_finalizing), \ + INIT_ID(_find_and_load), \ + INIT_ID(_fix_up_module), \ + INIT_ID(_get_sourcefile), \ + INIT_ID(_handle_fromlist), \ + INIT_ID(_initializing), \ + INIT_ID(_is_text_encoding), \ + INIT_ID(_lock_unlock_module), \ + INIT_ID(_showwarnmsg), \ + INIT_ID(_shutdown), \ + INIT_ID(_slotnames), \ + INIT_ID(_strptime_time), \ + INIT_ID(_uninitialized_submodules), \ + INIT_ID(_warn_unawaited_coroutine), \ + INIT_ID(_xoptions), \ + INIT_ID(add), \ + INIT_ID(append), \ + INIT_ID(big), \ + INIT_ID(buffer), \ + INIT_ID(builtins), \ + INIT_ID(clear), \ + INIT_ID(close), \ + INIT_ID(code), \ + INIT_ID(copy), \ + INIT_ID(copyreg), \ + INIT_ID(decode), \ + INIT_ID(default), \ + INIT_ID(defaultaction), \ + INIT_ID(difference_update), \ + INIT_ID(dispatch_table), \ + INIT_ID(displayhook), \ + INIT_ID(enable), \ + INIT_ID(encoding), \ + INIT_ID(end_lineno), \ + INIT_ID(end_offset), \ + INIT_ID(errors), \ + INIT_ID(excepthook), \ + INIT_ID(extend), \ + INIT_ID(filename), \ + INIT_ID(fileno), \ + INIT_ID(fillvalue), \ + INIT_ID(filters), \ + INIT_ID(find_class), \ + INIT_ID(flush), \ + INIT_ID(get), \ + INIT_ID(get_source), \ + INIT_ID(getattr), \ + INIT_ID(ignore), \ + INIT_ID(importlib), \ + INIT_ID(intersection), \ + INIT_ID(isatty), \ + INIT_ID(items), \ + INIT_ID(iter), \ + INIT_ID(keys), \ + INIT_ID(last_traceback), \ + INIT_ID(last_type), \ + INIT_ID(last_value), \ + INIT_ID(latin1), \ + INIT_ID(lineno), \ + INIT_ID(little), \ + INIT_ID(match), \ + INIT_ID(metaclass), \ + INIT_ID(mode), \ + INIT_ID(modules), \ + INIT_ID(mro), \ + INIT_ID(msg), \ + INIT_ID(n_fields), \ + INIT_ID(n_sequence_fields), \ + INIT_ID(n_unnamed_fields), \ + INIT_ID(name), \ + INIT_ID(obj), \ + INIT_ID(offset), \ + INIT_ID(onceregistry), \ + INIT_ID(open), \ + INIT_ID(parent), \ + INIT_ID(partial), \ + INIT_ID(path), \ + INIT_ID(peek), \ + INIT_ID(persistent_id), \ + INIT_ID(persistent_load), \ + INIT_ID(print_file_and_line), \ + INIT_ID(ps1), \ + INIT_ID(ps2), \ + INIT_ID(raw), \ + INIT_ID(read), \ + INIT_ID(read1), \ + INIT_ID(readable), \ + INIT_ID(readall), \ + INIT_ID(readinto), \ + INIT_ID(readinto1), \ + INIT_ID(readline), \ + INIT_ID(reducer_override), \ + INIT_ID(reload), \ + INIT_ID(replace), \ + INIT_ID(reset), \ + INIT_ID(return), \ + INIT_ID(reversed), \ + INIT_ID(seek), \ + INIT_ID(seekable), \ + INIT_ID(send), \ + INIT_ID(setstate), \ + INIT_ID(sort), \ + INIT_ID(stderr), \ + INIT_ID(stdin), \ + INIT_ID(stdout), \ + INIT_ID(strict), \ + INIT_ID(symmetric_difference_update), \ + INIT_ID(tell), \ + INIT_ID(text), \ + INIT_ID(threading), \ + INIT_ID(throw), \ + INIT_ID(unraisablehook), \ + INIT_ID(values), \ + INIT_ID(version), \ + INIT_ID(warnings), \ + INIT_ID(warnoptions), \ + INIT_ID(writable), \ + INIT_ID(write), \ + INIT_ID(zipimporter), \ + }, \ + }, \ }, \ } /* End auto-generated code */ diff --git a/Include/internal/pycore_structseq.h b/Include/internal/pycore_structseq.h index c0323bbea8991..0199c790e24ce 100644 --- a/Include/internal/pycore_structseq.h +++ b/Include/internal/pycore_structseq.h @@ -9,11 +9,6 @@ extern "C" { #endif -/* runtime lifecycle */ - -extern PyStatus _PyStructSequence_InitState(PyInterpreterState *); - - /* other API */ PyAPI_FUNC(PyTypeObject *) _PyStructSequence_NewType( diff --git a/Include/internal/pycore_sysmodule.h b/Include/internal/pycore_sysmodule.h index 738a7746a0384..10d092cdc30a2 100644 --- a/Include/internal/pycore_sysmodule.h +++ b/Include/internal/pycore_sysmodule.h @@ -18,6 +18,8 @@ PyAPI_FUNC(int) _PySys_Audit( PyAPI_FUNC() to not export the symbol. */ extern void _PySys_ClearAuditHooks(PyThreadState *tstate); +PyAPI_FUNC(int) _PySys_SetAttr(PyObject *, PyObject *); + #ifdef __cplusplus } #endif diff --git a/Include/internal/pycore_unicodeobject.h b/Include/internal/pycore_unicodeobject.h index fabe522f6fc23..977bbeb191712 100644 --- a/Include/internal/pycore_unicodeobject.h +++ b/Include/internal/pycore_unicodeobject.h @@ -44,8 +44,6 @@ struct _Py_unicode_ids { }; struct _Py_unicode_state { - // The empty Unicode object is a singleton to improve performance. - PyObject *empty_string; /* Single character Unicode strings in the Latin-1 range are being shared as well. */ PyObject *latin1[256]; diff --git a/Makefile.pre.in b/Makefile.pre.in index 4dcedd684aa6d..c279cf38bd926 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -2436,6 +2436,9 @@ patchcheck: @DEF_MAKE_RULE@ check-limited-abi: all $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/stable_abi.py --all $(srcdir)/Misc/stable_abi.txt +check-global-objects: all + $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/generate_global_objects.py --check + .PHONY: update-config update-config: curl -sL -o config.guess 'https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD' diff --git a/Modules/_abc.c b/Modules/_abc.c index a043961812041..641d6198d1fec 100644 --- a/Modules/_abc.c +++ b/Modules/_abc.c @@ -4,8 +4,9 @@ #endif #include "Python.h" -#include "pycore_object.h" // _PyType_GetSubclasses() #include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_object.h" // _PyType_GetSubclasses() +#include "pycore_runtime.h" // _Py_ID() #include "clinic/_abc.c.h" /*[clinic input] @@ -16,15 +17,6 @@ module _abc PyDoc_STRVAR(_abc__doc__, "Module contains faster C implementation of abc.ABCMeta"); -_Py_IDENTIFIER(__abstractmethods__); -_Py_IDENTIFIER(__class__); -_Py_IDENTIFIER(__dict__); -_Py_IDENTIFIER(__abc_tpflags__); -_Py_IDENTIFIER(__bases__); -_Py_IDENTIFIER(_abc_impl); -_Py_IDENTIFIER(__subclasscheck__); -_Py_IDENTIFIER(__subclasshook__); - typedef struct { PyTypeObject *_abc_data_type; unsigned long long abc_invalidation_counter; @@ -122,7 +114,7 @@ static _abc_data * _get_impl(PyObject *module, PyObject *self) { _abcmodule_state *state = get_abc_state(module); - PyObject *impl = _PyObject_GetAttrId(self, &PyId__abc_impl); + PyObject *impl = PyObject_GetAttr(self, &_Py_ID(_abc_impl)); if (impl == NULL) { return NULL; } @@ -311,7 +303,7 @@ compute_abstract_methods(PyObject *self) PyObject *ns = NULL, *items = NULL, *bases = NULL; // Py_XDECREF()ed on error. /* Stage 1: direct abstract methods. */ - ns = _PyObject_GetAttrId(self, &PyId___dict__); + ns = PyObject_GetAttr(self, &_Py_ID(__dict__)); if (!ns) { goto error; } @@ -355,7 +347,7 @@ compute_abstract_methods(PyObject *self) } /* Stage 2: inherited abstract methods. */ - bases = _PyObject_GetAttrId(self, &PyId___bases__); + bases = PyObject_GetAttr(self, &_Py_ID(__bases__)); if (!bases) { goto error; } @@ -368,8 +360,8 @@ compute_abstract_methods(PyObject *self) PyObject *item = PyTuple_GET_ITEM(bases, pos); // borrowed PyObject *base_abstracts, *iter; - if (_PyObject_LookupAttrId(item, &PyId___abstractmethods__, - &base_abstracts) < 0) { + if (_PyObject_LookupAttr(item, &_Py_ID(__abstractmethods__), + &base_abstracts) < 0) { goto error; } if (base_abstracts == NULL) { @@ -409,7 +401,7 @@ compute_abstract_methods(PyObject *self) } } - if (_PyObject_SetAttrId(self, &PyId___abstractmethods__, abstracts) < 0) { + if (PyObject_SetAttr(self, &_Py_ID(__abstractmethods__), abstracts) < 0) { goto error; } @@ -448,7 +440,7 @@ _abc__abc_init(PyObject *module, PyObject *self) if (data == NULL) { return NULL; } - if (_PyObject_SetAttrId(self, &PyId__abc_impl, data) < 0) { + if (PyObject_SetAttr(self, &_Py_ID(_abc_impl), data) < 0) { Py_DECREF(data); return NULL; } @@ -459,7 +451,8 @@ _abc__abc_init(PyObject *module, PyObject *self) * their special status w.r.t. pattern matching. */ if (PyType_Check(self)) { PyTypeObject *cls = (PyTypeObject *)self; - PyObject *flags = _PyDict_GetItemIdWithError(cls->tp_dict, &PyId___abc_tpflags__); + PyObject *flags = PyDict_GetItemWithError(cls->tp_dict, + &_Py_ID(__abc_tpflags__)); if (flags == NULL) { if (PyErr_Occurred()) { return NULL; @@ -477,7 +470,7 @@ _abc__abc_init(PyObject *module, PyObject *self) } ((PyTypeObject *)self)->tp_flags |= (val & COLLECTION_FLAGS); } - if (_PyDict_DelItemId(cls->tp_dict, &PyId___abc_tpflags__) < 0) { + if (PyDict_DelItem(cls->tp_dict, &_Py_ID(__abc_tpflags__)) < 0) { return NULL; } } @@ -593,7 +586,7 @@ _abc__abc_instancecheck_impl(PyObject *module, PyObject *self, return NULL; } - subclass = _PyObject_GetAttrId(instance, &PyId___class__); + subclass = PyObject_GetAttr(instance, &_Py_ID(__class__)); if (subclass == NULL) { Py_DECREF(impl); return NULL; @@ -622,12 +615,12 @@ _abc__abc_instancecheck_impl(PyObject *module, PyObject *self, } } /* Fall back to the subclass check. */ - result = _PyObject_CallMethodIdOneArg(self, &PyId___subclasscheck__, - subclass); + result = PyObject_CallMethodOneArg(self, &_Py_ID(__subclasscheck__), + subclass); goto end; } - result = _PyObject_CallMethodIdOneArg(self, &PyId___subclasscheck__, - subclass); + result = PyObject_CallMethodOneArg(self, &_Py_ID(__subclasscheck__), + subclass); if (result == NULL) { goto end; } @@ -639,8 +632,8 @@ _abc__abc_instancecheck_impl(PyObject *module, PyObject *self, break; case 0: Py_DECREF(result); - result = _PyObject_CallMethodIdOneArg(self, &PyId___subclasscheck__, - subtype); + result = PyObject_CallMethodOneArg(self, &_Py_ID(__subclasscheck__), + subtype); break; case 1: // Nothing to do. break; @@ -723,8 +716,8 @@ _abc__abc_subclasscheck_impl(PyObject *module, PyObject *self, } /* 3. Check the subclass hook. */ - ok = _PyObject_CallMethodIdOneArg((PyObject *)self, &PyId___subclasshook__, - subclass); + ok = PyObject_CallMethodOneArg( + (PyObject *)self, &_Py_ID(__subclasshook__), subclass); if (ok == NULL) { goto end; } diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 2216dd0178173..72dbdb8902f12 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -1,6 +1,7 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #include "Python.h" #include "pycore_pyerrors.h" // _PyErr_ClearExcState() diff --git a/Modules/_bisectmodule.c b/Modules/_bisectmodule.c index 26c4b9bfb26b2..f884d94413c0d 100644 --- a/Modules/_bisectmodule.c +++ b/Modules/_bisectmodule.c @@ -4,6 +4,7 @@ Converted to C by Dmitry Vasiliev (dima at hlabs.spb.ru). */ #define PY_SSIZE_T_CLEAN +#define NEEDS_PY_IDENTIFIER #include "Python.h" /*[clinic input] diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c index 4c4e1f762a4a6..e7bd8bc15122f 100644 --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -1348,9 +1348,8 @@ static PyObject * deque_reduce(dequeobject *deque, PyObject *Py_UNUSED(ignored)) { PyObject *dict, *it; - _Py_IDENTIFIER(__dict__); - if (_PyObject_LookupAttrId((PyObject *)deque, &PyId___dict__, &dict) < 0) { + if (_PyObject_LookupAttr((PyObject *)deque, &_Py_ID(__dict__), &dict) < 0) { return NULL; } if (dict == NULL) { @@ -2064,7 +2063,6 @@ defdict_reduce(defdictobject *dd, PyObject *Py_UNUSED(ignored)) PyObject *items; PyObject *iter; PyObject *result; - _Py_IDENTIFIER(items); if (dd->default_factory == NULL || dd->default_factory == Py_None) args = PyTuple_New(0); @@ -2072,7 +2070,7 @@ defdict_reduce(defdictobject *dd, PyObject *Py_UNUSED(ignored)) args = PyTuple_Pack(1, dd->default_factory); if (args == NULL) return NULL; - items = _PyObject_CallMethodIdNoArgs((PyObject *)dd, &PyId_items); + items = PyObject_CallMethodNoArgs((PyObject *)dd, &_Py_ID(items)); if (items == NULL) { Py_DECREF(args); return NULL; @@ -2310,8 +2308,6 @@ _collections__count_elements_impl(PyObject *module, PyObject *mapping, PyObject *iterable) /*[clinic end generated code: output=7e0c1789636b3d8f input=e79fad04534a0b45]*/ { - _Py_IDENTIFIER(get); - _Py_IDENTIFIER(__setitem__); PyObject *it, *oldval; PyObject *newval = NULL; PyObject *key = NULL; @@ -2329,10 +2325,10 @@ _collections__count_elements_impl(PyObject *module, PyObject *mapping, /* Only take the fast path when get() and __setitem__() * have not been overridden. */ - mapping_get = _PyType_LookupId(Py_TYPE(mapping), &PyId_get); - dict_get = _PyType_LookupId(&PyDict_Type, &PyId_get); - mapping_setitem = _PyType_LookupId(Py_TYPE(mapping), &PyId___setitem__); - dict_setitem = _PyType_LookupId(&PyDict_Type, &PyId___setitem__); + mapping_get = _PyType_Lookup(Py_TYPE(mapping), &_Py_ID(get)); + dict_get = _PyType_Lookup(&PyDict_Type, &_Py_ID(get)); + mapping_setitem = _PyType_Lookup(Py_TYPE(mapping), &_Py_ID(__setitem__)); + dict_setitem = _PyType_Lookup(&PyDict_Type, &_Py_ID(__setitem__)); if (mapping_get != NULL && mapping_get == dict_get && mapping_setitem != NULL && mapping_setitem == dict_setitem && @@ -2381,7 +2377,7 @@ _collections__count_elements_impl(PyObject *module, PyObject *mapping, } } else { - bound_get = _PyObject_GetAttrId(mapping, &PyId_get); + bound_get = PyObject_GetAttr(mapping, &_Py_ID(get)); if (bound_get == NULL) goto done; diff --git a/Modules/_csv.c b/Modules/_csv.c index 1c2f504ea5c09..d2cb77f9f0334 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -10,6 +10,8 @@ module instead. #define MODULE_VERSION "1.0" +#define NEEDS_PY_IDENTIFIER + #include "Python.h" #include "structmember.h" // PyMemberDef #include diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index 15eb7521baa1c..481de85fde3ad 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -101,6 +101,7 @@ bytes(cdata) #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #define PY_SSIZE_T_CLEAN diff --git a/Modules/_ctypes/callbacks.c b/Modules/_ctypes/callbacks.c index a8fee0de71aa8..f2d9a530e6e38 100644 --- a/Modules/_ctypes/callbacks.c +++ b/Modules/_ctypes/callbacks.c @@ -1,6 +1,7 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #include "Python.h" // windows.h must be included before pycore internal headers diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index da2956788a1dd..6dba0ffaa5159 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -54,6 +54,8 @@ */ +#define NEEDS_PY_IDENTIFIER + #include "Python.h" #include "structmember.h" // PyMemberDef diff --git a/Modules/_ctypes/stgdict.c b/Modules/_ctypes/stgdict.c index 3470cf114db4f..a819ce910d4b5 100644 --- a/Modules/_ctypes/stgdict.c +++ b/Modules/_ctypes/stgdict.c @@ -1,6 +1,7 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #include "Python.h" // windows.h must be included before pycore internal headers diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c index 423b042b90755..7e2699e1b5c5b 100644 --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -103,6 +103,7 @@ static const char PyCursesVersion[] = "2.2"; #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #define PY_SSIZE_T_CLEAN diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index fda8401b84cd1..ae97190bccbde 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -10,6 +10,7 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #include "Python.h" #include "pycore_long.h" // _PyLong_GetOne() diff --git a/Modules/_dbmmodule.c b/Modules/_dbmmodule.c index c0371fb3f3b20..65778fa77bfec 100644 --- a/Modules/_dbmmodule.c +++ b/Modules/_dbmmodule.c @@ -3,6 +3,7 @@ #define PY_SSIZE_T_CLEAN +#define NEEDS_PY_IDENTIFIER #include "Python.h" #include diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 9dadeef712938..f155dc72152d4 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -12,6 +12,7 @@ */ #define PY_SSIZE_T_CLEAN +#define NEEDS_PY_IDENTIFIER #include "Python.h" #include "structmember.h" // PyMemberDef diff --git a/Modules/_gdbmmodule.c b/Modules/_gdbmmodule.c index 445500c7ee4c1..2d690c0abfd08 100644 --- a/Modules/_gdbmmodule.c +++ b/Modules/_gdbmmodule.c @@ -4,6 +4,7 @@ /* Doc strings: Mitch Chapman */ #define PY_SSIZE_T_CLEAN +#define NEEDS_PY_IDENTIFIER #include "Python.h" #include "gdbm.h" diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c index 116688da5e7af..9ecc6718cb1fd 100644 --- a/Modules/_io/_iomodule.c +++ b/Modules/_io/_iomodule.c @@ -241,11 +241,6 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode, PyObject *raw, *modeobj = NULL, *buffer, *wrapper, *result = NULL, *path_or_fd = NULL; - _Py_IDENTIFIER(_blksize); - _Py_IDENTIFIER(isatty); - _Py_IDENTIFIER(mode); - _Py_IDENTIFIER(close); - is_number = PyNumber_Check(file); if (is_number) { @@ -381,7 +376,7 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode, /* buffering */ if (buffering < 0) { - PyObject *res = _PyObject_CallMethodIdNoArgs(raw, &PyId_isatty); + PyObject *res = PyObject_CallMethodNoArgs(raw, &_Py_ID(isatty)); if (res == NULL) goto error; isatty = PyLong_AsLong(res); @@ -399,7 +394,7 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode, if (buffering < 0) { PyObject *blksize_obj; - blksize_obj = _PyObject_GetAttrId(raw, &PyId__blksize); + blksize_obj = PyObject_GetAttr(raw, &_Py_ID(_blksize)); if (blksize_obj == NULL) goto error; buffering = PyLong_AsLong(blksize_obj); @@ -466,7 +461,7 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode, result = wrapper; Py_DECREF(buffer); - if (_PyObject_SetAttrId(wrapper, &PyId_mode, modeobj) < 0) + if (PyObject_SetAttr(wrapper, &_Py_ID(mode), modeobj) < 0) goto error; Py_DECREF(modeobj); return result; @@ -475,7 +470,7 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode, if (result != NULL) { PyObject *exc, *val, *tb, *close_result; PyErr_Fetch(&exc, &val, &tb); - close_result = _PyObject_CallMethodIdNoArgs(result, &PyId_close); + close_result = PyObject_CallMethodNoArgs(result, &_Py_ID(close)); _PyErr_ChainExceptions(exc, val, tb); Py_XDECREF(close_result); Py_DECREF(result); diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c index 28995051abd4b..bb93ae361f892 100644 --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -25,21 +25,6 @@ class _io.BufferedRandom "buffered *" "&PyBufferedRandom_Type" [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=59460b9c5639984d]*/ -_Py_IDENTIFIER(close); -_Py_IDENTIFIER(_dealloc_warn); -_Py_IDENTIFIER(flush); -_Py_IDENTIFIER(isatty); -_Py_IDENTIFIER(mode); -_Py_IDENTIFIER(name); -_Py_IDENTIFIER(peek); -_Py_IDENTIFIER(read); -_Py_IDENTIFIER(read1); -_Py_IDENTIFIER(readable); -_Py_IDENTIFIER(readinto); -_Py_IDENTIFIER(readinto1); -_Py_IDENTIFIER(writable); -_Py_IDENTIFIER(write); - /* * BufferedIOBase class, inherits from IOBase. */ @@ -65,9 +50,10 @@ _bufferediobase_readinto_generic(PyObject *self, Py_buffer *buffer, char readint Py_ssize_t len; PyObject *data; - data = _PyObject_CallMethodId(self, - readinto1 ? &PyId_read1 : &PyId_read, - "n", buffer->len); + PyObject *attr = readinto1 + ? &_Py_ID(read1) + : &_Py_ID(read); + data = _PyObject_CallMethod(self, attr, "n", buffer->len); if (data == NULL) return NULL; @@ -436,8 +422,7 @@ buffered_dealloc_warn(buffered *self, PyObject *source) { if (self->ok && self->raw) { PyObject *r; - r = _PyObject_CallMethodIdOneArg(self->raw, &PyId__dealloc_warn, - source); + r = PyObject_CallMethodOneArg(self->raw, &_Py_ID(_dealloc_warn), source); if (r) Py_DECREF(r); else @@ -583,14 +568,14 @@ static PyObject * buffered_name_get(buffered *self, void *context) { CHECK_INITIALIZED(self) - return _PyObject_GetAttrId(self->raw, &PyId_name); + return PyObject_GetAttr(self->raw, &_Py_ID(name)); } static PyObject * buffered_mode_get(buffered *self, void *context) { CHECK_INITIALIZED(self) - return _PyObject_GetAttrId(self->raw, &PyId_mode); + return PyObject_GetAttr(self->raw, &_Py_ID(mode)); } /* Lower-level APIs */ @@ -1381,7 +1366,7 @@ buffered_repr(buffered *self) { PyObject *nameobj, *res; - if (_PyObject_LookupAttrId((PyObject *) self, &PyId_name, &nameobj) < 0) { + if (_PyObject_LookupAttr((PyObject *) self, &_Py_ID(name), &nameobj) < 0) { if (!PyErr_ExceptionMatches(PyExc_ValueError)) { return NULL; } @@ -2153,7 +2138,7 @@ bufferedrwpair_dealloc(rwpair *self) } static PyObject * -_forward_call(buffered *self, _Py_Identifier *name, PyObject *args) +_forward_call(buffered *self, PyObject *name, PyObject *args) { PyObject *func, *ret; if (self == NULL) { @@ -2162,9 +2147,9 @@ _forward_call(buffered *self, _Py_Identifier *name, PyObject *args) return NULL; } - func = _PyObject_GetAttrId((PyObject *)self, name); + func = PyObject_GetAttr((PyObject *)self, name); if (func == NULL) { - PyErr_SetString(PyExc_AttributeError, name->string); + PyErr_SetObject(PyExc_AttributeError, name); return NULL; } @@ -2176,67 +2161,67 @@ _forward_call(buffered *self, _Py_Identifier *name, PyObject *args) static PyObject * bufferedrwpair_read(rwpair *self, PyObject *args) { - return _forward_call(self->reader, &PyId_read, args); + return _forward_call(self->reader, &_Py_ID(read), args); } static PyObject * bufferedrwpair_peek(rwpair *self, PyObject *args) { - return _forward_call(self->reader, &PyId_peek, args); + return _forward_call(self->reader, &_Py_ID(peek), args); } static PyObject * bufferedrwpair_read1(rwpair *self, PyObject *args) { - return _forward_call(self->reader, &PyId_read1, args); + return _forward_call(self->reader, &_Py_ID(read1), args); } static PyObject * bufferedrwpair_readinto(rwpair *self, PyObject *args) { - return _forward_call(self->reader, &PyId_readinto, args); + return _forward_call(self->reader, &_Py_ID(readinto), args); } static PyObject * bufferedrwpair_readinto1(rwpair *self, PyObject *args) { - return _forward_call(self->reader, &PyId_readinto1, args); + return _forward_call(self->reader, &_Py_ID(readinto1), args); } static PyObject * bufferedrwpair_write(rwpair *self, PyObject *args) { - return _forward_call(self->writer, &PyId_write, args); + return _forward_call(self->writer, &_Py_ID(write), args); } static PyObject * bufferedrwpair_flush(rwpair *self, PyObject *Py_UNUSED(ignored)) { - return _forward_call(self->writer, &PyId_flush, NULL); + return _forward_call(self->writer, &_Py_ID(flush), NULL); } static PyObject * bufferedrwpair_readable(rwpair *self, PyObject *Py_UNUSED(ignored)) { - return _forward_call(self->reader, &PyId_readable, NULL); + return _forward_call(self->reader, &_Py_ID(readable), NULL); } static PyObject * bufferedrwpair_writable(rwpair *self, PyObject *Py_UNUSED(ignored)) { - return _forward_call(self->writer, &PyId_writable, NULL); + return _forward_call(self->writer, &_Py_ID(writable), NULL); } static PyObject * bufferedrwpair_close(rwpair *self, PyObject *Py_UNUSED(ignored)) { PyObject *exc = NULL, *val, *tb; - PyObject *ret = _forward_call(self->writer, &PyId_close, NULL); + PyObject *ret = _forward_call(self->writer, &_Py_ID(close), NULL); if (ret == NULL) PyErr_Fetch(&exc, &val, &tb); else Py_DECREF(ret); - ret = _forward_call(self->reader, &PyId_close, NULL); + ret = _forward_call(self->reader, &_Py_ID(close), NULL); if (exc != NULL) { _PyErr_ChainExceptions(exc, val, tb); Py_CLEAR(ret); @@ -2247,7 +2232,7 @@ bufferedrwpair_close(rwpair *self, PyObject *Py_UNUSED(ignored)) static PyObject * bufferedrwpair_isatty(rwpair *self, PyObject *Py_UNUSED(ignored)) { - PyObject *ret = _forward_call(self->writer, &PyId_isatty, NULL); + PyObject *ret = _forward_call(self->writer, &_Py_ID(isatty), NULL); if (ret != Py_False) { /* either True or exception */ @@ -2255,7 +2240,7 @@ bufferedrwpair_isatty(rwpair *self, PyObject *Py_UNUSED(ignored)) } Py_DECREF(ret); - return _forward_call(self->reader, &PyId_isatty, NULL); + return _forward_call(self->reader, &_Py_ID(isatty), NULL); } static PyObject * diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c index dd215e8939944..8b1cff56d75fa 100644 --- a/Modules/_io/fileio.c +++ b/Modules/_io/fileio.c @@ -72,8 +72,6 @@ typedef struct { PyTypeObject PyFileIO_Type; -_Py_IDENTIFIER(name); - #define PyFileIO_Check(op) (PyObject_TypeCheck((op), &PyFileIO_Type)) /* Forward declarations */ @@ -146,9 +144,8 @@ _io_FileIO_close_impl(fileio *self) PyObject *res; PyObject *exc, *val, *tb; int rc; - _Py_IDENTIFIER(close); - res = _PyObject_CallMethodIdOneArg((PyObject*)&PyRawIOBase_Type, - &PyId_close, (PyObject *)self); + res = PyObject_CallMethodOneArg((PyObject*)&PyRawIOBase_Type, + &_Py_ID(close), (PyObject *)self); if (!self->closefd) { self->fd = -1; return res; @@ -476,7 +473,7 @@ _Py_COMP_DIAG_POP _setmode(self->fd, O_BINARY); #endif - if (_PyObject_SetAttrId((PyObject *)self, &PyId_name, nameobj) < 0) + if (PyObject_SetAttr((PyObject *)self, &_Py_ID(name), nameobj) < 0) goto error; if (self->appending) { @@ -1085,7 +1082,7 @@ fileio_repr(fileio *self) if (self->fd < 0) return PyUnicode_FromFormat("<_io.FileIO [closed]>"); - if (_PyObject_LookupAttrId((PyObject *) self, &PyId_name, &nameobj) < 0) { + if (_PyObject_LookupAttr((PyObject *) self, &_Py_ID(name), &nameobj) < 0) { return NULL; } if (nameobj == NULL) { diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c index 5b687b78176e8..74663e007b1e5 100644 --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -69,9 +69,6 @@ PyDoc_STRVAR(iobase_doc, of the IOBase object rather than the virtual `closed` attribute as returned by whatever subclass. */ -_Py_IDENTIFIER(__IOBase_closed); -_Py_IDENTIFIER(read); - /* Internal methods */ static PyObject * @@ -114,9 +111,7 @@ static PyObject * _io__IOBase_tell_impl(PyObject *self) /*[clinic end generated code: output=89a1c0807935abe2 input=04e615fec128801f]*/ { - _Py_IDENTIFIER(seek); - - return _PyObject_CallMethodId(self, &PyId_seek, "ii", 0, 1); + return _PyObject_CallMethod(self, &_Py_ID(seek), "ii", 0, 1); } PyDoc_STRVAR(iobase_truncate_doc, @@ -138,7 +133,7 @@ iobase_is_closed(PyObject *self) int ret; /* This gets the derived attribute, which is *not* __IOBase_closed in most cases! */ - ret = _PyObject_LookupAttrId(self, &PyId___IOBase_closed, &res); + ret = _PyObject_LookupAttr(self, &_Py_ID(__IOBase_closed), &res); Py_XDECREF(res); return ret; } @@ -239,7 +234,7 @@ _io__IOBase_close_impl(PyObject *self) res = PyObject_CallMethodNoArgs(self, _PyIO_str_flush); PyErr_Fetch(&exc, &val, &tb); - rc = _PyObject_SetAttrId(self, &PyId___IOBase_closed, Py_True); + rc = PyObject_SetAttr(self, &_Py_ID(__IOBase_closed), Py_True); _PyErr_ChainExceptions(exc, val, tb); if (rc < 0) { Py_CLEAR(res); @@ -260,7 +255,6 @@ iobase_finalize(PyObject *self) PyObject *res; PyObject *error_type, *error_value, *error_traceback; int closed; - _Py_IDENTIFIER(_finalizing); /* Save the current exception, if any. */ PyErr_Fetch(&error_type, &error_value, &error_traceback); @@ -280,7 +274,7 @@ iobase_finalize(PyObject *self) if (closed == 0) { /* Signal close() that it was called as part of the object finalization process. */ - if (_PyObject_SetAttrId(self, &PyId__finalizing, Py_True)) + if (PyObject_SetAttr(self, &_Py_ID(_finalizing), Py_True)) PyErr_Clear(); res = PyObject_CallMethodNoArgs((PyObject *)self, _PyIO_str_close); /* Silencing I/O errors is bad, but printing spurious tracebacks is @@ -597,7 +591,7 @@ _io__IOBase_readline_impl(PyObject *self, Py_ssize_t limit) Py_DECREF(readahead); } - b = _PyObject_CallMethodId(self, &PyId_read, "n", nreadahead); + b = _PyObject_CallMethod(self, &_Py_ID(read), "n", nreadahead); if (b == NULL) { /* NOTE: PyErr_SetFromErrno() calls PyErr_CheckSignals() when EINTR occurs so we needn't do it ourselves. */ @@ -697,10 +691,8 @@ _io__IOBase_readlines_impl(PyObject *self, Py_ssize_t hint) /* XXX special-casing this made sense in the Python version in order to remove the bytecode interpretation overhead, but it could probably be removed here. */ - _Py_IDENTIFIER(extend); - PyObject *ret = _PyObject_CallMethodIdObjArgs(result, &PyId_extend, - self, NULL); - + PyObject *ret = PyObject_CallMethodObjArgs(result, &_Py_ID(extend), + self, NULL); if (ret == NULL) { goto error; } @@ -919,9 +911,7 @@ _io__RawIOBase_read_impl(PyObject *self, Py_ssize_t n) PyObject *b, *res; if (n < 0) { - _Py_IDENTIFIER(readall); - - return _PyObject_CallMethodIdNoArgs(self, &PyId_readall); + return PyObject_CallMethodNoArgs(self, &_Py_ID(readall)); } /* TODO: allocate a bytes object directly instead and manually construct @@ -967,8 +957,8 @@ _io__RawIOBase_readall_impl(PyObject *self) return NULL; while (1) { - PyObject *data = _PyObject_CallMethodId(self, &PyId_read, - "i", DEFAULT_BUFFER_SIZE); + PyObject *data = _PyObject_CallMethod(self, &_Py_ID(read), + "i", DEFAULT_BUFFER_SIZE); if (!data) { /* NOTE: PyErr_SetFromErrno() calls PyErr_CheckSignals() when EINTR occurs so we needn't do it ourselves. */ diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c index eb05ae1a16eb0..d5ceec7981f3a 100644 --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -23,26 +23,6 @@ class _io.TextIOWrapper "textio *" "&TextIOWrapper_TYpe" [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=2097a4fc85670c26]*/ -_Py_IDENTIFIER(close); -_Py_IDENTIFIER(_dealloc_warn); -_Py_IDENTIFIER(decode); -_Py_IDENTIFIER(fileno); -_Py_IDENTIFIER(flush); -_Py_IDENTIFIER(isatty); -_Py_IDENTIFIER(mode); -_Py_IDENTIFIER(name); -_Py_IDENTIFIER(raw); -_Py_IDENTIFIER(read); -_Py_IDENTIFIER(readable); -_Py_IDENTIFIER(replace); -_Py_IDENTIFIER(reset); -_Py_IDENTIFIER(seek); -_Py_IDENTIFIER(seekable); -_Py_IDENTIFIER(setstate); -_Py_IDENTIFIER(strict); -_Py_IDENTIFIER(tell); -_Py_IDENTIFIER(writable); - /* TextIOBase */ PyDoc_STRVAR(textiobase_doc, @@ -255,9 +235,7 @@ _io_IncrementalNewlineDecoder___init___impl(nldecoder_object *self, Py_INCREF(decoder); if (errors == NULL) { - self->errors = _PyUnicode_FromId(&PyId_strict); - if (self->errors == NULL) - return -1; + self->errors = &_Py_ID(strict); } else { self->errors = errors; @@ -586,11 +564,13 @@ _io_IncrementalNewlineDecoder_setstate(nldecoder_object *self, self->pendingcr = (int) (flag & 1); flag >>= 1; - if (self->decoder != Py_None) - return _PyObject_CallMethodId(self->decoder, - &PyId_setstate, "((OK))", buffer, flag); - else + if (self->decoder != Py_None) { + return _PyObject_CallMethod(self->decoder, &_Py_ID(setstate), + "((OK))", buffer, flag); + } + else { Py_RETURN_NONE; + } } /*[clinic input] @@ -865,7 +845,7 @@ _textiowrapper_set_decoder(textio *self, PyObject *codec_info, PyObject *res; int r; - res = _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_readable); + res = PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(readable)); if (res == NULL) return -1; @@ -920,7 +900,7 @@ _textiowrapper_set_encoder(textio *self, PyObject *codec_info, PyObject *res; int r; - res = _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_writable); + res = PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(writable)); if (res == NULL) return -1; @@ -939,7 +919,7 @@ _textiowrapper_set_encoder(textio *self, PyObject *codec_info, return -1; /* Get the normalized named of the codec */ - if (_PyObject_LookupAttrId(codec_info, &PyId_name, &res) < 0) { + if (_PyObject_LookupAttr(codec_info, &_Py_ID(name), &res) < 0) { return -1; } if (res != NULL && PyUnicode_Check(res)) { @@ -1099,10 +1079,7 @@ _io_TextIOWrapper___init___impl(textio *self, PyObject *buffer, } if (errors == Py_None) { - errors = _PyUnicode_FromId(&PyId_strict); /* borrowed */ - if (errors == NULL) { - return -1; - } + errors = &_Py_ID(strict); } else if (!PyUnicode_Check(errors)) { // Check 'errors' argument here because Argument Clinic doesn't support @@ -1142,7 +1119,7 @@ _io_TextIOWrapper___init___impl(textio *self, PyObject *buffer, state = IO_STATE(); if (state == NULL) goto error; - fileno = _PyObject_CallMethodIdNoArgs(buffer, &PyId_fileno); + fileno = PyObject_CallMethodNoArgs(buffer, &_Py_ID(fileno)); /* Ignore only AttributeError and UnsupportedOperation */ if (fileno == NULL) { if (PyErr_ExceptionMatches(PyExc_AttributeError) || @@ -1228,7 +1205,7 @@ _io_TextIOWrapper___init___impl(textio *self, PyObject *buffer, Py_IS_TYPE(buffer, &PyBufferedWriter_Type) || Py_IS_TYPE(buffer, &PyBufferedRandom_Type)) { - if (_PyObject_LookupAttrId(buffer, &PyId_raw, &raw) < 0) + if (_PyObject_LookupAttr(buffer, &_Py_ID(raw), &raw) < 0) goto error; /* Cache the raw FileIO object to speed up 'closed' checks */ if (raw != NULL) { @@ -1239,7 +1216,7 @@ _io_TextIOWrapper___init___impl(textio *self, PyObject *buffer, } } - res = _PyObject_CallMethodIdNoArgs(buffer, &PyId_seekable); + res = PyObject_CallMethodNoArgs(buffer, &_Py_ID(seekable)); if (res == NULL) goto error; r = PyObject_IsTrue(res); @@ -1302,10 +1279,7 @@ textiowrapper_change_encoding(textio *self, PyObject *encoding, } } else if (errors == Py_None) { - errors = _PyUnicode_FromId(&PyId_strict); - if (errors == NULL) { - return -1; - } + errors = &_Py_ID(strict); } const char *c_errors = PyUnicode_AsUTF8(errors); @@ -1640,8 +1614,8 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) haslf = 1; if (haslf && self->writetranslate && self->writenl != NULL) { - PyObject *newtext = _PyObject_CallMethodId( - text, &PyId_replace, "ss", "\n", self->writenl); + PyObject *newtext = _PyObject_CallMethod(text, &_Py_ID(replace), + "ss", "\n", self->writenl); Py_DECREF(text); if (newtext == NULL) return NULL; @@ -1740,7 +1714,7 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) Py_CLEAR(self->snapshot); if (self->decoder) { - ret = _PyObject_CallMethodIdNoArgs(self->decoder, &PyId_reset); + ret = PyObject_CallMethodNoArgs(self->decoder, &_Py_ID(reset)); if (ret == NULL) return NULL; Py_DECREF(ret); @@ -1944,7 +1918,7 @@ _io_TextIOWrapper_read_impl(textio *self, Py_ssize_t n) if (n < 0) { /* Read everything */ - PyObject *bytes = _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_read); + PyObject *bytes = PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(read)); PyObject *decoded; if (bytes == NULL) goto fail; @@ -2404,13 +2378,16 @@ _textiowrapper_decoder_setstate(textio *self, cookie_type *cookie) at start is not (b"", 0) but e.g. (b"", 2) (meaning, in the case of utf-16, that we are expecting a BOM). */ - if (cookie->start_pos == 0 && cookie->dec_flags == 0) + if (cookie->start_pos == 0 && cookie->dec_flags == 0) { res = PyObject_CallMethodNoArgs(self->decoder, _PyIO_str_reset); - else - res = _PyObject_CallMethodId(self->decoder, &PyId_setstate, - "((yi))", "", cookie->dec_flags); - if (res == NULL) + } + else { + res = _PyObject_CallMethod(self->decoder, &_Py_ID(setstate), + "((yi))", "", cookie->dec_flags); + } + if (res == NULL) { return -1; + } Py_DECREF(res); return 0; } @@ -2487,7 +2464,7 @@ _io_TextIOWrapper_seek_impl(textio *self, PyObject *cookieObj, int whence) * sync the underlying buffer with the current position. */ Py_DECREF(cookieObj); - cookieObj = _PyObject_CallMethodIdNoArgs((PyObject *)self, &PyId_tell); + cookieObj = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(tell)); if (cookieObj == NULL) goto fail; break; @@ -2503,7 +2480,7 @@ _io_TextIOWrapper_seek_impl(textio *self, PyObject *cookieObj, int whence) goto fail; } - res = _PyObject_CallMethodIdNoArgs((PyObject *)self, &PyId_flush); + res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(flush)); if (res == NULL) goto fail; Py_DECREF(res); @@ -2511,13 +2488,13 @@ _io_TextIOWrapper_seek_impl(textio *self, PyObject *cookieObj, int whence) textiowrapper_set_decoded_chars(self, NULL); Py_CLEAR(self->snapshot); if (self->decoder) { - res = _PyObject_CallMethodIdNoArgs(self->decoder, &PyId_reset); + res = PyObject_CallMethodNoArgs(self->decoder, &_Py_ID(reset)); if (res == NULL) goto fail; Py_DECREF(res); } - res = _PyObject_CallMethodId(self->buffer, &PyId_seek, "ii", 0, 2); + res = _PyObject_CallMethod(self->buffer, &_Py_ID(seek), "ii", 0, 2); Py_CLEAR(cookieObj); if (res == NULL) goto fail; @@ -2583,8 +2560,8 @@ _io_TextIOWrapper_seek_impl(textio *self, PyObject *cookieObj, int whence) if (cookie.chars_to_skip) { /* Just like _read_chunk, feed the decoder and save a snapshot. */ - PyObject *input_chunk = _PyObject_CallMethodId( - self->buffer, &PyId_read, "i", cookie.bytes_to_feed); + PyObject *input_chunk = _PyObject_CallMethod(self->buffer, &_Py_ID(read), + "i", cookie.bytes_to_feed); PyObject *decoded; if (input_chunk == NULL) @@ -2605,7 +2582,7 @@ _io_TextIOWrapper_seek_impl(textio *self, PyObject *cookieObj, int whence) } Py_XSETREF(self->snapshot, snapshot); - decoded = _PyObject_CallMethodIdObjArgs(self->decoder, &PyId_decode, + decoded = PyObject_CallMethodObjArgs(self->decoder, &_Py_ID(decode), input_chunk, cookie.need_eof ? Py_True : Py_False, NULL); if (check_decoded(decoded) < 0) @@ -2673,12 +2650,12 @@ _io_TextIOWrapper_tell_impl(textio *self) if (_textiowrapper_writeflush(self) < 0) return NULL; - res = _PyObject_CallMethodIdNoArgs((PyObject *)self, &PyId_flush); + res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(flush)); if (res == NULL) goto fail; Py_DECREF(res); - posobj = _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_tell); + posobj = PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(tell)); if (posobj == NULL) goto fail; @@ -2750,8 +2727,8 @@ _io_TextIOWrapper_tell_impl(textio *self) } while (0) #define DECODER_DECODE(start, len, res) do { \ - PyObject *_decoded = _PyObject_CallMethodId( \ - self->decoder, &PyId_decode, "y#", start, len); \ + PyObject *_decoded = _PyObject_CallMethod( \ + self->decoder, &_Py_ID(decode), "y#", start, len); \ if (check_decoded(_decoded) < 0) \ goto fail; \ res = PyUnicode_GET_LENGTH(_decoded); \ @@ -2832,8 +2809,8 @@ _io_TextIOWrapper_tell_impl(textio *self) } if (input == input_end) { /* We didn't get enough decoded data; signal EOF to get more. */ - PyObject *decoded = _PyObject_CallMethodId( - self->decoder, &PyId_decode, "yO", "", /* final = */ Py_True); + PyObject *decoded = _PyObject_CallMethod( + self->decoder, &_Py_ID(decode), "yO", "", /* final = */ Py_True); if (check_decoded(decoded) < 0) goto fail; chars_decoded += PyUnicode_GET_LENGTH(decoded); @@ -2848,7 +2825,8 @@ _io_TextIOWrapper_tell_impl(textio *self) } finally: - res = _PyObject_CallMethodIdOneArg(self->decoder, &PyId_setstate, saved_state); + res = PyObject_CallMethodOneArg( + self->decoder, &_Py_ID(setstate), saved_state); Py_DECREF(saved_state); if (res == NULL) return NULL; @@ -2862,7 +2840,8 @@ _io_TextIOWrapper_tell_impl(textio *self) if (saved_state) { PyObject *type, *value, *traceback; PyErr_Fetch(&type, &value, &traceback); - res = _PyObject_CallMethodIdOneArg(self->decoder, &PyId_setstate, saved_state); + res = PyObject_CallMethodOneArg( + self->decoder, &_Py_ID(setstate), saved_state); _PyErr_ChainExceptions(type, value, traceback); Py_DECREF(saved_state); Py_XDECREF(res); @@ -2913,7 +2892,7 @@ textiowrapper_repr(textio *self) } goto error; } - if (_PyObject_LookupAttrId((PyObject *) self, &PyId_name, &nameobj) < 0) { + if (_PyObject_LookupAttr((PyObject *) self, &_Py_ID(name), &nameobj) < 0) { if (!PyErr_ExceptionMatches(PyExc_ValueError)) { goto error; } @@ -2929,7 +2908,7 @@ textiowrapper_repr(textio *self) if (res == NULL) goto error; } - if (_PyObject_LookupAttrId((PyObject *) self, &PyId_mode, &modeobj) < 0) { + if (_PyObject_LookupAttr((PyObject *) self, &_Py_ID(mode), &modeobj) < 0) { goto error; } if (modeobj != NULL) { @@ -2969,7 +2948,7 @@ _io_TextIOWrapper_fileno_impl(textio *self) /*[clinic end generated code: output=21490a4c3da13e6c input=c488ca83d0069f9b]*/ { CHECK_ATTACHED(self); - return _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_fileno); + return PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(fileno)); } /*[clinic input] @@ -2981,7 +2960,7 @@ _io_TextIOWrapper_seekable_impl(textio *self) /*[clinic end generated code: output=ab223dbbcffc0f00 input=8b005ca06e1fca13]*/ { CHECK_ATTACHED(self); - return _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_seekable); + return PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(seekable)); } /*[clinic input] @@ -2993,7 +2972,7 @@ _io_TextIOWrapper_readable_impl(textio *self) /*[clinic end generated code: output=72ff7ba289a8a91b input=0704ea7e01b0d3eb]*/ { CHECK_ATTACHED(self); - return _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_readable); + return PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(readable)); } /*[clinic input] @@ -3005,7 +2984,7 @@ _io_TextIOWrapper_writable_impl(textio *self) /*[clinic end generated code: output=a728c71790d03200 input=c41740bc9d8636e8]*/ { CHECK_ATTACHED(self); - return _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_writable); + return PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(writable)); } /*[clinic input] @@ -3017,7 +2996,7 @@ _io_TextIOWrapper_isatty_impl(textio *self) /*[clinic end generated code: output=12be1a35bace882e input=fb68d9f2c99bbfff]*/ { CHECK_ATTACHED(self); - return _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_isatty); + return PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(isatty)); } /*[clinic input] @@ -3033,7 +3012,7 @@ _io_TextIOWrapper_flush_impl(textio *self) self->telling = self->seekable; if (_textiowrapper_writeflush(self) < 0) return NULL; - return _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_flush); + return PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(flush)); } /*[clinic input] @@ -3062,21 +3041,20 @@ _io_TextIOWrapper_close_impl(textio *self) else { PyObject *exc = NULL, *val, *tb; if (self->finalizing) { - res = _PyObject_CallMethodIdOneArg(self->buffer, - &PyId__dealloc_warn, - (PyObject *)self); + res = PyObject_CallMethodOneArg(self->buffer, &_Py_ID(_dealloc_warn), + (PyObject *)self); if (res) Py_DECREF(res); else PyErr_Clear(); } - res = _PyObject_CallMethodIdNoArgs((PyObject *)self, &PyId_flush); + res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(flush)); if (res == NULL) PyErr_Fetch(&exc, &val, &tb); else Py_DECREF(res); - res = _PyObject_CallMethodIdNoArgs(self->buffer, &PyId_close); + res = PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(close)); if (exc != NULL) { _PyErr_ChainExceptions(exc, val, tb); Py_CLEAR(res); @@ -3127,7 +3105,7 @@ static PyObject * textiowrapper_name_get(textio *self, void *context) { CHECK_ATTACHED(self); - return _PyObject_GetAttrId(self->buffer, &PyId_name); + return PyObject_GetAttr(self->buffer, &_Py_ID(name)); } static PyObject * diff --git a/Modules/_io/winconsoleio.c b/Modules/_io/winconsoleio.c index 4002d28fc9b35..5c1a6dd86fc54 100644 --- a/Modules/_io/winconsoleio.c +++ b/Modules/_io/winconsoleio.c @@ -156,8 +156,6 @@ typedef struct { PyTypeObject PyWindowsConsoleIO_Type; -_Py_IDENTIFIER(name); - int _PyWindowsConsoleIO_closed(PyObject *self) { @@ -196,9 +194,8 @@ _io__WindowsConsoleIO_close_impl(winconsoleio *self) PyObject *res; PyObject *exc, *val, *tb; int rc; - _Py_IDENTIFIER(close); - res = _PyObject_CallMethodIdOneArg((PyObject*)&PyRawIOBase_Type, - &PyId_close, (PyObject*)self); + res = PyObject_CallMethodOneArg((PyObject*)&PyRawIOBase_Type, + &_Py_ID(close), (PyObject*)self); if (!self->closefd) { self->fd = -1; return res; @@ -394,7 +391,7 @@ _io__WindowsConsoleIO___init___impl(winconsoleio *self, PyObject *nameobj, self->blksize = DEFAULT_BUFFER_SIZE; memset(self->buf, 0, 4); - if (_PyObject_SetAttrId((PyObject *)self, &PyId_name, nameobj) < 0) + if (PyObject_SetAttr((PyObject *)self, &_Py_ID(name), nameobj) < 0) goto error; goto done; diff --git a/Modules/_json.c b/Modules/_json.c index 1c9c50665d466..f7a3e38de9926 100644 --- a/Modules/_json.c +++ b/Modules/_json.c @@ -7,6 +7,7 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #include "Python.h" #include "structmember.h" // PyMemberDef diff --git a/Modules/_lzmamodule.c b/Modules/_lzmamodule.c index e50f55b75c5dc..e3fc90e5175eb 100644 --- a/Modules/_lzmamodule.c +++ b/Modules/_lzmamodule.c @@ -6,6 +6,7 @@ */ #define PY_SSIZE_T_CLEAN +#define NEEDS_PY_IDENTIFIER #include "Python.h" #include "structmember.h" // PyMemberDef diff --git a/Modules/_operator.c b/Modules/_operator.c index b3a8bef2eaedd..10adabcbf0e85 100644 --- a/Modules/_operator.c +++ b/Modules/_operator.c @@ -1,5 +1,6 @@ #include "Python.h" #include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_runtime.h" // _Py_ID() #include "clinic/_operator.c.h" typedef struct { @@ -1693,11 +1694,10 @@ methodcaller_reduce(methodcallerobject *mc, PyObject *Py_UNUSED(ignored)) PyObject *constructor; PyObject *newargs[2]; - _Py_IDENTIFIER(partial); functools = PyImport_ImportModule("functools"); if (!functools) return NULL; - partial = _PyObject_GetAttrId(functools, &PyId_partial); + partial = PyObject_GetAttr(functools, &_Py_ID(partial)); Py_DECREF(functools); if (!partial) return NULL; diff --git a/Modules/_pickle.c b/Modules/_pickle.c index 0d9e57aa90d22..19e8a71073c97 100644 --- a/Modules/_pickle.c +++ b/Modules/_pickle.c @@ -11,6 +11,8 @@ #include "Python.h" #include "pycore_floatobject.h" // _PyFloat_Pack8() #include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_runtime.h" // _Py_ID() +#include "pycore_pystate.h" // _PyThreadState_GET() #include "structmember.h" // PyMemberDef #include // strtol() @@ -35,6 +37,13 @@ enum { DEFAULT_PROTOCOL = 4 }; +#ifdef MS_WINDOWS +// These are already typedefs from windows.h, pulled in via pycore_runtime.h. +#define FLOAT FLOAT_ +#define INT INT_ +#define LONG LONG_ +#endif + /* Pickle opcodes. These must be kept updated with pickle.py. Extensive docs are in pickletools.py. */ enum opcode { @@ -225,9 +234,8 @@ _Pickle_InitState(PickleState *st) PyObject *compat_pickle = NULL; PyObject *codecs = NULL; PyObject *functools = NULL; - _Py_IDENTIFIER(getattr); - st->getattr = _PyEval_GetBuiltinId(&PyId_getattr); + st->getattr = _PyEval_GetBuiltin(&_Py_ID(getattr)); if (st->getattr == NULL) goto error; @@ -373,7 +381,7 @@ _Pickle_FastCall(PyObject *func, PyObject *obj) /* Retrieve and deconstruct a method for avoiding a reference cycle (pickler -> bound method of pickler -> pickler) */ static int -init_method_ref(PyObject *self, _Py_Identifier *name, +init_method_ref(PyObject *self, PyObject *name, PyObject **method_func, PyObject **method_self) { PyObject *func, *func2; @@ -381,7 +389,7 @@ init_method_ref(PyObject *self, _Py_Identifier *name, /* *method_func and *method_self should be consistent. All refcount decrements should be occurred after setting *method_self and *method_func. */ - ret = _PyObject_LookupAttrId(self, name, &func); + ret = _PyObject_LookupAttr(self, name, &func); if (func == NULL) { *method_self = NULL; Py_CLEAR(*method_func); @@ -1175,9 +1183,8 @@ _Pickler_SetProtocol(PicklerObject *self, PyObject *protocol, int fix_imports) static int _Pickler_SetOutputStream(PicklerObject *self, PyObject *file) { - _Py_IDENTIFIER(write); assert(file != NULL); - if (_PyObject_LookupAttrId(file, &PyId_write, &self->write) < 0) { + if (_PyObject_LookupAttr(file, &_Py_ID(write), &self->write) < 0) { return -1; } if (self->write == NULL) { @@ -1636,20 +1643,15 @@ _Unpickler_New(void) static int _Unpickler_SetInputStream(UnpicklerObject *self, PyObject *file) { - _Py_IDENTIFIER(peek); - _Py_IDENTIFIER(read); - _Py_IDENTIFIER(readinto); - _Py_IDENTIFIER(readline); - /* Optional file methods */ - if (_PyObject_LookupAttrId(file, &PyId_peek, &self->peek) < 0) { + if (_PyObject_LookupAttr(file, &_Py_ID(peek), &self->peek) < 0) { return -1; } - if (_PyObject_LookupAttrId(file, &PyId_readinto, &self->readinto) < 0) { + if (_PyObject_LookupAttr(file, &_Py_ID(readinto), &self->readinto) < 0) { return -1; } - (void)_PyObject_LookupAttrId(file, &PyId_read, &self->read); - (void)_PyObject_LookupAttrId(file, &PyId_readline, &self->readline); + (void)_PyObject_LookupAttr(file, &_Py_ID(read), &self->read); + (void)_PyObject_LookupAttr(file, &_Py_ID(readline), &self->readline); if (!self->readline || !self->read) { if (!PyErr_Occurred()) { PyErr_SetString(PyExc_TypeError, @@ -1809,11 +1811,10 @@ memo_put(PicklerObject *self, PyObject *obj) static PyObject * get_dotted_path(PyObject *obj, PyObject *name) { - _Py_static_string(PyId_dot, "."); PyObject *dotted_path; Py_ssize_t i, n; - dotted_path = PyUnicode_Split(name, _PyUnicode_FromId(&PyId_dot), -1); + dotted_path = PyUnicode_Split(name, &_Py_STR(dot), -1); if (dotted_path == NULL) return NULL; n = PyList_GET_SIZE(dotted_path); @@ -1914,11 +1915,8 @@ whichmodule(PyObject *global, PyObject *dotted_path) PyObject *module = NULL; Py_ssize_t i; PyObject *modules; - _Py_IDENTIFIER(__module__); - _Py_IDENTIFIER(modules); - _Py_IDENTIFIER(__main__); - if (_PyObject_LookupAttrId(global, &PyId___module__, &module_name) < 0) { + if (_PyObject_LookupAttr(global, &_Py_ID(__module__), &module_name) < 0) { return NULL; } if (module_name) { @@ -1932,7 +1930,8 @@ whichmodule(PyObject *global, PyObject *dotted_path) assert(module_name == NULL); /* Fallback on walking sys.modules */ - modules = _PySys_GetObjectId(&PyId_modules); + PyThreadState *tstate = _PyThreadState_GET(); + modules = _PySys_GetAttr(tstate, &_Py_ID(modules)); if (modules == NULL) { PyErr_SetString(PyExc_RuntimeError, "unable to get sys.modules"); return NULL; @@ -1977,8 +1976,8 @@ whichmodule(PyObject *global, PyObject *dotted_path) } /* If no module is found, use __main__. */ - module_name = _PyUnicode_FromId(&PyId___main__); - Py_XINCREF(module_name); + module_name = &_Py_ID(__main__); + Py_INCREF(module_name); return module_name; } @@ -2427,13 +2426,12 @@ save_bytes(PicklerObject *self, PyObject *obj) PyUnicode_DecodeLatin1(PyBytes_AS_STRING(obj), PyBytes_GET_SIZE(obj), "strict"); - _Py_IDENTIFIER(latin1); if (unicode_str == NULL) return -1; reduce_value = Py_BuildValue("(O(OO))", st->codecs_encode, unicode_str, - _PyUnicode_FromId(&PyId_latin1)); + &_Py_ID(latin1)); Py_DECREF(unicode_str); } @@ -3335,9 +3333,7 @@ save_dict(PicklerObject *self, PyObject *obj) status = batch_dict_exact(self, obj); Py_LeaveRecursiveCall(); } else { - _Py_IDENTIFIER(items); - - items = _PyObject_CallMethodIdNoArgs(obj, &PyId_items); + items = PyObject_CallMethodNoArgs(obj, &_Py_ID(items)); if (items == NULL) goto error; iter = PyObject_GetIter(items); @@ -3589,8 +3585,6 @@ save_global(PicklerObject *self, PyObject *obj, PyObject *name) PyObject *cls; PickleState *st = _Pickle_GetGlobalState(); int status = 0; - _Py_IDENTIFIER(__name__); - _Py_IDENTIFIER(__qualname__); const char global_op = GLOBAL; @@ -3599,10 +3593,10 @@ save_global(PicklerObject *self, PyObject *obj, PyObject *name) global_name = name; } else { - if (_PyObject_LookupAttrId(obj, &PyId___qualname__, &global_name) < 0) + if (_PyObject_LookupAttr(obj, &_Py_ID(__qualname__), &global_name) < 0) goto error; if (global_name == NULL) { - global_name = _PyObject_GetAttrId(obj, &PyId___name__); + global_name = PyObject_GetAttr(obj, &_Py_ID(__name__)); if (global_name == NULL) goto error; } @@ -3923,9 +3917,8 @@ static PyObject * get_class(PyObject *obj) { PyObject *cls; - _Py_IDENTIFIER(__class__); - if (_PyObject_LookupAttrId(obj, &PyId___class__, &cls) == 0) { + if (_PyObject_LookupAttr(obj, &_Py_ID(__class__), &cls) == 0) { cls = (PyObject *) Py_TYPE(obj); Py_INCREF(cls); } @@ -4008,18 +4001,14 @@ save_reduce(PicklerObject *self, PyObject *args, PyObject *obj) if (self->proto >= 2) { PyObject *name; - _Py_IDENTIFIER(__name__); - if (_PyObject_LookupAttrId(callable, &PyId___name__, &name) < 0) { + if (_PyObject_LookupAttr(callable, &_Py_ID(__name__), &name) < 0) { return -1; } if (name != NULL && PyUnicode_Check(name)) { - _Py_IDENTIFIER(__newobj_ex__); - use_newobj_ex = _PyUnicode_EqualToASCIIId( - name, &PyId___newobj_ex__); + use_newobj_ex = _PyUnicode_Equal(name, &_Py_ID(__newobj_ex__)); if (!use_newobj_ex) { - _Py_IDENTIFIER(__newobj__); - use_newobj = _PyUnicode_EqualToASCIIId(name, &PyId___newobj__); + use_newobj = _PyUnicode_Equal(name, &_Py_ID(__newobj__)); } } Py_XDECREF(name); @@ -4071,13 +4060,12 @@ save_reduce(PicklerObject *self, PyObject *args, PyObject *obj) PyObject *newargs; PyObject *cls_new; Py_ssize_t i; - _Py_IDENTIFIER(__new__); newargs = PyTuple_New(PyTuple_GET_SIZE(args) + 2); if (newargs == NULL) return -1; - cls_new = _PyObject_GetAttrId(cls, &PyId___new__); + cls_new = PyObject_GetAttr(cls, &_Py_ID(__new__)); if (cls_new == NULL) { Py_DECREF(newargs); return -1; @@ -4412,9 +4400,6 @@ save(PicklerObject *self, PyObject *obj, int pers_save) goto done; } else { - _Py_IDENTIFIER(__reduce__); - _Py_IDENTIFIER(__reduce_ex__); - /* XXX: If the __reduce__ method is defined, __reduce_ex__ is automatically defined as __reduce__. While this is convenient, this make it impossible to know which method was actually called. Of @@ -4424,7 +4409,7 @@ save(PicklerObject *self, PyObject *obj, int pers_save) don't actually have to check for a __reduce__ method. */ /* Check for a __reduce_ex__ method. */ - if (_PyObject_LookupAttrId(obj, &PyId___reduce_ex__, &reduce_func) < 0) { + if (_PyObject_LookupAttr(obj, &_Py_ID(__reduce_ex__), &reduce_func) < 0) { goto error; } if (reduce_func != NULL) { @@ -4436,7 +4421,7 @@ save(PicklerObject *self, PyObject *obj, int pers_save) } else { /* Check for a __reduce__ method. */ - if (_PyObject_LookupAttrId(obj, &PyId___reduce__, &reduce_func) < 0) { + if (_PyObject_LookupAttr(obj, &_Py_ID(__reduce__), &reduce_func) < 0) { goto error; } if (reduce_func != NULL) { @@ -4489,10 +4474,9 @@ dump(PicklerObject *self, PyObject *obj) const char stop_op = STOP; int status = -1; PyObject *tmp; - _Py_IDENTIFIER(reducer_override); - if (_PyObject_LookupAttrId((PyObject *)self, &PyId_reducer_override, - &tmp) < 0) { + if (_PyObject_LookupAttr((PyObject *)self, &_Py_ID(reducer_override), + &tmp) < 0) { goto error; } /* Cache the reducer_override method, if it exists. */ @@ -4727,9 +4711,6 @@ _pickle_Pickler___init___impl(PicklerObject *self, PyObject *file, PyObject *buffer_callback) /*[clinic end generated code: output=0abedc50590d259b input=a7c969699bf5dad3]*/ { - _Py_IDENTIFIER(persistent_id); - _Py_IDENTIFIER(dispatch_table); - /* In case of multiple __init__() calls, clear previous content. */ if (self->write != NULL) (void)Pickler_clear(self); @@ -4762,14 +4743,14 @@ _pickle_Pickler___init___impl(PicklerObject *self, PyObject *file, self->fast_nesting = 0; self->fast_memo = NULL; - if (init_method_ref((PyObject *)self, &PyId_persistent_id, + if (init_method_ref((PyObject *)self, &_Py_ID(persistent_id), &self->pers_func, &self->pers_func_self) < 0) { return -1; } - if (_PyObject_LookupAttrId((PyObject *)self, - &PyId_dispatch_table, &self->dispatch_table) < 0) { + if (_PyObject_LookupAttr((PyObject *)self, &_Py_ID(dispatch_table), + &self->dispatch_table) < 0) { return -1; } @@ -5123,10 +5104,8 @@ static PyTypeObject Pickler_Type = { static PyObject * find_class(UnpicklerObject *self, PyObject *module_name, PyObject *global_name) { - _Py_IDENTIFIER(find_class); - - return _PyObject_CallMethodIdObjArgs((PyObject *)self, &PyId_find_class, - module_name, global_name, NULL); + return PyObject_CallMethodObjArgs((PyObject *)self, &_Py_ID(find_class), + module_name, global_name, NULL); } static Py_ssize_t @@ -5813,14 +5792,12 @@ instantiate(PyObject *cls, PyObject *args) into a newly created tuple. */ assert(PyTuple_Check(args)); if (!PyTuple_GET_SIZE(args) && PyType_Check(cls)) { - _Py_IDENTIFIER(__getinitargs__); - _Py_IDENTIFIER(__new__); PyObject *func; - if (_PyObject_LookupAttrId(cls, &PyId___getinitargs__, &func) < 0) { + if (_PyObject_LookupAttr(cls, &_Py_ID(__getinitargs__), &func) < 0) { return NULL; } if (func == NULL) { - return _PyObject_CallMethodIdOneArg(cls, &PyId___new__, cls); + return PyObject_CallMethodOneArg(cls, &_Py_ID(__new__), cls); } Py_DECREF(func); } @@ -6465,9 +6442,8 @@ do_append(UnpicklerObject *self, Py_ssize_t x) } else { PyObject *extend_func; - _Py_IDENTIFIER(extend); - if (_PyObject_LookupAttrId(list, &PyId_extend, &extend_func) < 0) { + if (_PyObject_LookupAttr(list, &_Py_ID(extend), &extend_func) < 0) { return -1; } if (extend_func != NULL) { @@ -6484,12 +6460,11 @@ do_append(UnpicklerObject *self, Py_ssize_t x) } else { PyObject *append_func; - _Py_IDENTIFIER(append); /* Even if the PEP 307 requires extend() and append() methods, fall back on append() if the object has no extend() method for backward compatibility. */ - append_func = _PyObject_GetAttrId(list, &PyId_append); + append_func = PyObject_GetAttr(list, &_Py_ID(append)); if (append_func == NULL) return -1; for (i = x; i < len; i++) { @@ -6612,9 +6587,8 @@ load_additems(UnpicklerObject *self) } else { PyObject *add_func; - _Py_IDENTIFIER(add); - add_func = _PyObject_GetAttrId(set, &PyId_add); + add_func = PyObject_GetAttr(set, &_Py_ID(add)); if (add_func == NULL) return -1; for (i = mark; i < len; i++) { @@ -6642,7 +6616,6 @@ load_build(UnpicklerObject *self) PyObject *state, *inst, *slotstate; PyObject *setstate; int status = 0; - _Py_IDENTIFIER(__setstate__); /* Stack is ... instance, state. We want to leave instance at * the stack top, possibly mutated via instance.__setstate__(state). @@ -6656,7 +6629,7 @@ load_build(UnpicklerObject *self) inst = self->stack->data[Py_SIZE(self->stack) - 1]; - if (_PyObject_LookupAttrId(inst, &PyId___setstate__, &setstate) < 0) { + if (_PyObject_LookupAttr(inst, &_Py_ID(__setstate__), &setstate) < 0) { Py_DECREF(state); return -1; } @@ -6692,14 +6665,13 @@ load_build(UnpicklerObject *self) PyObject *dict; PyObject *d_key, *d_value; Py_ssize_t i; - _Py_IDENTIFIER(__dict__); if (!PyDict_Check(state)) { PickleState *st = _Pickle_GetGlobalState(); PyErr_SetString(st->UnpicklingError, "state is not a dictionary"); goto error; } - dict = _PyObject_GetAttrId(inst, &PyId___dict__); + dict = PyObject_GetAttr(inst, &_Py_ID(__dict__)); if (dict == NULL) goto error; @@ -7251,8 +7223,6 @@ _pickle_Unpickler___init___impl(UnpicklerObject *self, PyObject *file, const char *errors, PyObject *buffers) /*[clinic end generated code: output=09f0192649ea3f85 input=ca4c1faea9553121]*/ { - _Py_IDENTIFIER(persistent_load); - /* In case of multiple __init__() calls, clear previous content. */ if (self->read != NULL) (void)Unpickler_clear(self); @@ -7268,7 +7238,7 @@ _pickle_Unpickler___init___impl(UnpicklerObject *self, PyObject *file, self->fix_imports = fix_imports; - if (init_method_ref((PyObject *)self, &PyId_persistent_load, + if (init_method_ref((PyObject *)self, &_Py_ID(persistent_load), &self->pers_func, &self->pers_func_self) < 0) { return -1; diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index caefdf483a0df..cb8cd2a3d1807 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -21,6 +21,8 @@ * 3. This notice may not be removed or altered from any source distribution. */ +#define NEEDS_PY_IDENTIFIER + #include "module.h" #include "structmember.h" // PyMemberDef #include "connection.h" diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c index 4700afbbf1188..395b10f33ff3a 100644 --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -21,6 +21,8 @@ * 3. This notice may not be removed or altered from any source distribution. */ +#define NEEDS_PY_IDENTIFIER + #include "cursor.h" #include "module.h" #include "util.h" diff --git a/Modules/_sqlite/microprotocols.c b/Modules/_sqlite/microprotocols.c index 95c799d306f30..697d1a788a3b0 100644 --- a/Modules/_sqlite/microprotocols.c +++ b/Modules/_sqlite/microprotocols.c @@ -23,6 +23,8 @@ * 3. This notice may not be removed or altered from any source distribution. */ +#define NEEDS_PY_IDENTIFIER + #include #include "cursor.h" diff --git a/Modules/_sqlite/module.c b/Modules/_sqlite/module.c index 3b9f79799b5c5..cfa4b970e7320 100644 --- a/Modules/_sqlite/module.c +++ b/Modules/_sqlite/module.c @@ -21,6 +21,8 @@ * 3. This notice may not be removed or altered from any source distribution. */ +#define NEEDS_PY_IDENTIFIER + #include "connection.h" #include "statement.h" #include "cursor.h" diff --git a/Modules/_ssl.c b/Modules/_ssl.c index e17876e49a8be..07f0580ef6fc5 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -22,6 +22,7 @@ #define OPENSSL_NO_DEPRECATED 1 #define PY_SSIZE_T_CLEAN +#define NEEDS_PY_IDENTIFIER #include "Python.h" diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 498a7c2fd686b..0be42f33e23c9 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -13,6 +13,7 @@ #undef Py_BUILD_CORE_MODULE #undef Py_BUILD_CORE_BUILTIN +#define NEEDS_PY_IDENTIFIER /* Always enable assertions */ #undef NDEBUG @@ -3865,7 +3866,7 @@ slot_tp_del(PyObject *self) PyErr_Fetch(&error_type, &error_value, &error_traceback); /* Execute __del__ method, if any. */ - del = _PyObject_LookupSpecial(self, &PyId___tp_del__); + del = _PyObject_LookupSpecialId(self, &PyId___tp_del__); if (del != NULL) { res = PyObject_CallNoArgs(del); if (res == NULL) diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index 7052d4c887a66..072b79a75f3af 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -17,11 +17,6 @@ // ThreadError is just an alias to PyExc_RuntimeError #define ThreadError PyExc_RuntimeError -_Py_IDENTIFIER(__dict__); - -_Py_IDENTIFIER(stderr); -_Py_IDENTIFIER(flush); - // Forward declarations static struct PyModuleDef thread_module; @@ -938,12 +933,7 @@ local_setattro(localobject *self, PyObject *name, PyObject *v) return -1; } - PyObject *str_dict = _PyUnicode_FromId(&PyId___dict__); // borrowed ref - if (str_dict == NULL) { - return -1; - } - - int r = PyObject_RichCompareBool(name, str_dict, Py_EQ); + int r = PyObject_RichCompareBool(name, &_Py_ID(__dict__), Py_EQ); if (r == -1) { return -1; } @@ -994,12 +984,7 @@ local_getattro(localobject *self, PyObject *name) if (ldict == NULL) return NULL; - PyObject *str_dict = _PyUnicode_FromId(&PyId___dict__); // borrowed ref - if (str_dict == NULL) { - return NULL; - } - - int r = PyObject_RichCompareBool(name, str_dict, Py_EQ); + int r = PyObject_RichCompareBool(name, &_Py_ID(__dict__), Py_EQ); if (r == 1) { return Py_NewRef(ldict); } @@ -1413,7 +1398,6 @@ static int thread_excepthook_file(PyObject *file, PyObject *exc_type, PyObject *exc_value, PyObject *exc_traceback, PyObject *thread) { - _Py_IDENTIFIER(name); /* print(f"Exception in thread {thread.name}:", file=file) */ if (PyFile_WriteString("Exception in thread ", file) < 0) { return -1; @@ -1421,7 +1405,7 @@ thread_excepthook_file(PyObject *file, PyObject *exc_type, PyObject *exc_value, PyObject *name = NULL; if (thread != Py_None) { - if (_PyObject_LookupAttrId(thread, &PyId_name, &name) < 0) { + if (_PyObject_LookupAttr(thread, &_Py_ID(name), &name) < 0) { return -1; } } @@ -1459,7 +1443,7 @@ thread_excepthook_file(PyObject *file, PyObject *exc_type, PyObject *exc_value, _PyErr_Display(file, exc_type, exc_value, exc_traceback); /* Call file.flush() */ - PyObject *res = _PyObject_CallMethodIdNoArgs(file, &PyId_flush); + PyObject *res = PyObject_CallMethodNoArgs(file, &_Py_ID(flush)); if (!res) { return -1; } @@ -1514,7 +1498,8 @@ thread_excepthook(PyObject *module, PyObject *args) PyObject *exc_tb = PyStructSequence_GET_ITEM(args, 2); PyObject *thread = PyStructSequence_GET_ITEM(args, 3); - PyObject *file = _PySys_GetObjectId(&PyId_stderr); + PyThreadState *tstate = _PyThreadState_GET(); + PyObject *file = _PySys_GetAttr(tstate, &_Py_ID(stderr)); if (file == NULL || file == Py_None) { if (thread == Py_None) { /* do nothing if sys.stderr is None and thread is None */ diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c index 8e430632813e6..4aaab0c531435 100644 --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -6,6 +6,7 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #define PY_SSIZE_T_CLEAN #include "Python.h" diff --git a/Modules/cjkcodecs/multibytecodec.c b/Modules/cjkcodecs/multibytecodec.c index ba558d0dbf2f1..df48a0deec7e9 100644 --- a/Modules/cjkcodecs/multibytecodec.c +++ b/Modules/cjkcodecs/multibytecodec.c @@ -5,6 +5,7 @@ */ #define PY_SSIZE_T_CLEAN +#define NEEDS_PY_IDENTIFIER #include "Python.h" #include "structmember.h" // PyMemberDef #include "multibytecodec.h" diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c index 1888337cf9f39..46e18c17d8d35 100644 --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -32,11 +32,6 @@ #define PUTS(fd, str) _Py_write_noraise(fd, str, strlen(str)) -_Py_IDENTIFIER(enable); -_Py_IDENTIFIER(fileno); -_Py_IDENTIFIER(flush); -_Py_IDENTIFIER(stderr); - #ifdef HAVE_SIGACTION typedef struct sigaction _Py_sighandler_t; #else @@ -155,7 +150,8 @@ faulthandler_get_fileno(PyObject **file_ptr) PyObject *file = *file_ptr; if (file == NULL || file == Py_None) { - file = _PySys_GetObjectId(&PyId_stderr); + PyThreadState *tstate = _PyThreadState_GET(); + file = _PySys_GetAttr(tstate, &_Py_ID(stderr)); if (file == NULL) { PyErr_SetString(PyExc_RuntimeError, "unable to get sys.stderr"); return -1; @@ -178,7 +174,7 @@ faulthandler_get_fileno(PyObject **file_ptr) return fd; } - result = _PyObject_CallMethodIdNoArgs(file, &PyId_fileno); + result = PyObject_CallMethodNoArgs(file, &_Py_ID(fileno)); if (result == NULL) return -1; @@ -196,7 +192,7 @@ faulthandler_get_fileno(PyObject **file_ptr) return -1; } - result = _PyObject_CallMethodIdNoArgs(file, &PyId_flush); + result = PyObject_CallMethodNoArgs(file, &_Py_ID(flush)); if (result != NULL) Py_DECREF(result); else { @@ -1336,7 +1332,7 @@ faulthandler_init_enable(void) return -1; } - PyObject *res = _PyObject_CallMethodIdNoArgs(module, &PyId_enable); + PyObject *res = PyObject_CallMethodNoArgs(module, &_Py_ID(enable)); Py_DECREF(module); if (res == NULL) { return -1; diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index 342a3e6555ba0..0ec65d57381d8 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -503,9 +503,8 @@ _grouper_next(_grouperobject *igo) static PyObject * _grouper_reduce(_grouperobject *lz, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(iter); if (((groupbyobject *)lz->parent)->currgrouper != lz) { - return Py_BuildValue("N(())", _PyEval_GetBuiltinId(&PyId_iter)); + return Py_BuildValue("N(())", _PyEval_GetBuiltin(&_Py_ID(iter))); } return Py_BuildValue("O(OO)", Py_TYPE(lz), lz->parent, lz->tgtkey); } @@ -1015,7 +1014,6 @@ itertools_tee_impl(PyObject *module, PyObject *iterable, Py_ssize_t n) { Py_ssize_t i; PyObject *it, *copyable, *copyfunc, *result; - _Py_IDENTIFIER(__copy__); if (n < 0) { PyErr_SetString(PyExc_ValueError, "n must be >= 0"); @@ -1032,7 +1030,7 @@ itertools_tee_impl(PyObject *module, PyObject *iterable, Py_ssize_t n) return NULL; } - if (_PyObject_LookupAttrId(it, &PyId___copy__, ©func) < 0) { + if (_PyObject_LookupAttr(it, &_Py_ID(__copy__), ©func) < 0) { Py_DECREF(it); Py_DECREF(result); return NULL; @@ -1047,7 +1045,7 @@ itertools_tee_impl(PyObject *module, PyObject *iterable, Py_ssize_t n) Py_DECREF(result); return NULL; } - copyfunc = _PyObject_GetAttrId(copyable, &PyId___copy__); + copyfunc = PyObject_GetAttr(copyable, &_Py_ID(__copy__)); if (copyfunc == NULL) { Py_DECREF(copyable); Py_DECREF(result); @@ -1179,9 +1177,8 @@ cycle_reduce(cycleobject *lz, PyObject *Py_UNUSED(ignored)) if (it == NULL) return NULL; if (lz->index != 0) { - _Py_IDENTIFIER(__setstate__); - PyObject *res = _PyObject_CallMethodId(it, &PyId___setstate__, - "n", lz->index); + PyObject *res = _PyObject_CallMethod(it, &_Py_ID(__setstate__), + "n", lz->index); if (res == NULL) { Py_DECREF(it); return NULL; @@ -4545,7 +4542,6 @@ static PyTypeObject ziplongest_type; static PyObject * zip_longest_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { - _Py_IDENTIFIER(fillvalue); ziplongestobject *lz; Py_ssize_t i; PyObject *ittuple; /* tuple of iterators */ @@ -4556,7 +4552,7 @@ zip_longest_new(PyTypeObject *type, PyObject *args, PyObject *kwds) if (kwds != NULL && PyDict_CheckExact(kwds) && PyDict_GET_SIZE(kwds) > 0) { fillvalue = NULL; if (PyDict_GET_SIZE(kwds) == 1) { - fillvalue = _PyDict_GetItemIdWithError(kwds, &PyId_fillvalue); + fillvalue = PyDict_GetItemWithError(kwds, &_Py_ID(fillvalue)); } if (fillvalue == NULL) { if (!PyErr_Occurred()) { diff --git a/Modules/main.c b/Modules/main.c index b9bcea393abe3..2443f5631b94b 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -158,11 +158,10 @@ pymain_get_importer(const wchar_t *filename, PyObject **importer_p, int *exitcod static int pymain_sys_path_add_path0(PyInterpreterState *interp, PyObject *path0) { - _Py_IDENTIFIER(path); PyObject *sys_path; PyObject *sysdict = interp->sysdict; if (sysdict != NULL) { - sys_path = _PyDict_GetItemIdWithError(sysdict, &PyId_path); + sys_path = PyDict_GetItemWithError(sysdict, &_Py_ID(path)); if (sys_path == NULL && PyErr_Occurred()) { return -1; } diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 0c7d4de068621..721c9a63d51b2 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -55,6 +55,7 @@ raised for division by zero and mod by zero. #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #include "Python.h" #include "pycore_bitutils.h" // _Py_bit_length() @@ -1216,7 +1217,7 @@ math_ceil(PyObject *module, PyObject *number) _Py_IDENTIFIER(__ceil__); if (!PyFloat_CheckExact(number)) { - PyObject *method = _PyObject_LookupSpecial(number, &PyId___ceil__); + PyObject *method = _PyObject_LookupSpecialId(number, &PyId___ceil__); if (method != NULL) { PyObject *result = _PyObject_CallNoArgs(method); Py_DECREF(method); @@ -1288,7 +1289,7 @@ math_floor(PyObject *module, PyObject *number) } else { - PyObject *method = _PyObject_LookupSpecial(number, &PyId___floor__); + PyObject *method = _PyObject_LookupSpecialId(number, &PyId___floor__); if (method != NULL) { PyObject *result = _PyObject_CallNoArgs(method); Py_DECREF(method); @@ -2166,7 +2167,7 @@ math_trunc(PyObject *module, PyObject *x) return NULL; } - trunc = _PyObject_LookupSpecial(x, &PyId___trunc__); + trunc = _PyObject_LookupSpecialId(x, &PyId___trunc__); if (trunc == NULL) { if (!PyErr_Occurred()) PyErr_Format(PyExc_TypeError, diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index 7c9c28f7fab59..0a216941d0248 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -21,6 +21,7 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #define PY_SSIZE_T_CLEAN #include diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c index c9e788fd70457..172a6e4297531 100644 --- a/Modules/ossaudiodev.c +++ b/Modules/ossaudiodev.c @@ -20,6 +20,7 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #define PY_SSIZE_T_CLEAN #include "Python.h" diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index ea0435d7d52e6..6ef3610fce7c5 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -22,11 +22,12 @@ # include "pycore_bitutils.h" // _Py_popcount32() #endif #include "pycore_call.h" // _PyObject_CallNoArgs() -#include "pycore_fileutils.h" // _Py_closerange() -#include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_ceval.h" // _PyEval_ReInitThreads() +#include "pycore_fileutils.h" // _Py_closerange() #include "pycore_import.h" // _PyImport_ReInitLock() #include "pycore_initconfig.h" // _PyStatus_EXCEPTION() +#include "pycore_moduleobject.h" // _PyModule_GetState() +#include "pycore_object.h" // _PyObject_LookupSpecial() #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "structmember.h" // PyMemberDef @@ -323,8 +324,6 @@ corresponding Unix manual entries for more information on calls."); # endif /* _MSC_VER */ #endif /* ! __WATCOMC__ || __QNX__ */ -_Py_IDENTIFIER(__fspath__); - /*[clinic input] # one of the few times we lie about this name! module os @@ -1159,7 +1158,7 @@ path_converter(PyObject *o, void *p) /* Inline PyOS_FSPath() for better error messages. */ PyObject *func, *res; - func = _PyObject_LookupSpecial(o, &PyId___fspath__); + func = _PyObject_LookupSpecial(o, &_Py_ID(__fspath__)); if (NULL == func) { goto error_format; } @@ -14437,7 +14436,7 @@ PyOS_FSPath(PyObject *path) return path; } - func = _PyObject_LookupSpecial(path, &PyId___fspath__); + func = _PyObject_LookupSpecial(path, &_Py_ID(__fspath__)); if (NULL == func) { return PyErr_Format(PyExc_TypeError, "expected str, bytes or os.PathLike object, " diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index f2baab757f90c..f224f91f38fc3 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1,3 +1,5 @@ +#define NEEDS_PY_IDENTIFIER + #include "Python.h" #include diff --git a/Modules/timemodule.c b/Modules/timemodule.c index 4b7500aabded0..5b2d9b768ddd6 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -4,6 +4,7 @@ #include "pycore_fileutils.h" // _Py_BEGIN_SUPPRESS_IPH #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_namespace.h" // _PyNamespace_New() +#include "pycore_runtime.h" // _Py_ID() #include @@ -910,13 +911,12 @@ static PyObject * time_strptime(PyObject *self, PyObject *args) { PyObject *module, *func, *result; - _Py_IDENTIFIER(_strptime_time); module = PyImport_ImportModule("_strptime"); if (!module) return NULL; - func = _PyObject_GetAttrId(module, &PyId__strptime_time); + func = PyObject_GetAttr(module, &_Py_ID(_strptime_time)); Py_DECREF(module); if (!func) { return NULL; diff --git a/Modules/unicodedata.c b/Modules/unicodedata.c index bdbddcf10b778..7369db53d2e0b 100644 --- a/Modules/unicodedata.c +++ b/Modules/unicodedata.c @@ -15,6 +15,7 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif +#define NEEDS_PY_IDENTIFIER #define PY_SSIZE_T_CLEAN diff --git a/Objects/abstract.c b/Objects/abstract.c index ed99fd6c5aafb..6ad66a88b4619 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -92,7 +92,6 @@ PyObject_LengthHint(PyObject *o, Py_ssize_t defaultvalue) { PyObject *hint, *result; Py_ssize_t res; - _Py_IDENTIFIER(__length_hint__); if (_PyObject_HasLen(o)) { res = PyObject_Length(o); if (res < 0) { @@ -107,7 +106,7 @@ PyObject_LengthHint(PyObject *o, Py_ssize_t defaultvalue) return res; } } - hint = _PyObject_LookupSpecial(o, &PyId___length_hint__); + hint = _PyObject_LookupSpecial(o, &_Py_ID(__length_hint__)); if (hint == NULL) { if (PyErr_Occurred()) { return -1; @@ -177,14 +176,13 @@ PyObject_GetItem(PyObject *o, PyObject *key) if (PyType_Check(o)) { PyObject *meth, *result; - _Py_IDENTIFIER(__class_getitem__); // Special case type[int], but disallow other types so str[int] fails if ((PyTypeObject*)o == &PyType_Type) { return Py_GenericAlias(o, key); } - if (_PyObject_LookupAttrId(o, &PyId___class_getitem__, &meth) < 0) { + if (_PyObject_LookupAttr(o, &_Py_ID(__class_getitem__), &meth) < 0) { return NULL; } if (meth) { @@ -770,7 +768,6 @@ PyObject_Format(PyObject *obj, PyObject *format_spec) PyObject *meth; PyObject *empty = NULL; PyObject *result = NULL; - _Py_IDENTIFIER(__format__); if (format_spec != NULL && !PyUnicode_Check(format_spec)) { PyErr_Format(PyExc_SystemError, @@ -797,7 +794,7 @@ PyObject_Format(PyObject *obj, PyObject *format_spec) } /* Find the (unbound!) __format__ method */ - meth = _PyObject_LookupSpecial(obj, &PyId___format__); + meth = _PyObject_LookupSpecial(obj, &_Py_ID(__format__)); if (meth == NULL) { PyThreadState *tstate = _PyThreadState_GET(); if (!_PyErr_Occurred(tstate)) { @@ -1520,7 +1517,6 @@ PyNumber_Long(PyObject *o) PyNumberMethods *m; PyObject *trunc_func; Py_buffer view; - _Py_IDENTIFIER(__trunc__); if (o == NULL) { return null_error(); @@ -1562,7 +1558,7 @@ PyNumber_Long(PyObject *o) if (m && m->nb_index) { return PyNumber_Index(o); } - trunc_func = _PyObject_LookupSpecial(o, &PyId___trunc__); + trunc_func = _PyObject_LookupSpecial(o, &_Py_ID(__trunc__)); if (trunc_func) { if (PyErr_WarnEx(PyExc_DeprecationWarning, "The delegation of int() to __trunc__ is deprecated.", 1)) { @@ -2406,12 +2402,12 @@ PyMapping_HasKey(PyObject *o, PyObject *key) a helper for PyMapping_Keys(), PyMapping_Items() and PyMapping_Values(). */ static PyObject * -method_output_as_list(PyObject *o, _Py_Identifier *meth_id) +method_output_as_list(PyObject *o, PyObject *meth) { PyObject *it, *result, *meth_output; assert(o != NULL); - meth_output = _PyObject_CallMethodIdNoArgs(o, meth_id); + meth_output = PyObject_CallMethodNoArgs(o, meth); if (meth_output == NULL || PyList_CheckExact(meth_output)) { return meth_output; } @@ -2422,7 +2418,7 @@ method_output_as_list(PyObject *o, _Py_Identifier *meth_id) _PyErr_Format(tstate, PyExc_TypeError, "%.200s.%U() returned a non-iterable (type %.200s)", Py_TYPE(o)->tp_name, - _PyUnicode_FromId(meth_id), + meth, Py_TYPE(meth_output)->tp_name); } Py_DECREF(meth_output); @@ -2437,43 +2433,37 @@ method_output_as_list(PyObject *o, _Py_Identifier *meth_id) PyObject * PyMapping_Keys(PyObject *o) { - _Py_IDENTIFIER(keys); - if (o == NULL) { return null_error(); } if (PyDict_CheckExact(o)) { return PyDict_Keys(o); } - return method_output_as_list(o, &PyId_keys); + return method_output_as_list(o, &_Py_ID(keys)); } PyObject * PyMapping_Items(PyObject *o) { - _Py_IDENTIFIER(items); - if (o == NULL) { return null_error(); } if (PyDict_CheckExact(o)) { return PyDict_Items(o); } - return method_output_as_list(o, &PyId_items); + return method_output_as_list(o, &_Py_ID(items)); } PyObject * PyMapping_Values(PyObject *o) { - _Py_IDENTIFIER(values); - if (o == NULL) { return null_error(); } if (PyDict_CheckExact(o)) { return PyDict_Values(o); } - return method_output_as_list(o, &PyId_values); + return method_output_as_list(o, &_Py_ID(values)); } /* isinstance(), issubclass() */ @@ -2505,10 +2495,9 @@ PyMapping_Values(PyObject *o) static PyObject * abstract_get_bases(PyObject *cls) { - _Py_IDENTIFIER(__bases__); PyObject *bases; - (void)_PyObject_LookupAttrId(cls, &PyId___bases__, &bases); + (void)_PyObject_LookupAttr(cls, &_Py_ID(__bases__), &bases); if (bases != NULL && !PyTuple_Check(bases)) { Py_DECREF(bases); return NULL; @@ -2589,11 +2578,10 @@ object_isinstance(PyObject *inst, PyObject *cls) { PyObject *icls; int retval; - _Py_IDENTIFIER(__class__); if (PyType_Check(cls)) { retval = PyObject_TypeCheck(inst, (PyTypeObject *)cls); if (retval == 0) { - retval = _PyObject_LookupAttrId(inst, &PyId___class__, &icls); + retval = _PyObject_LookupAttr(inst, &_Py_ID(__class__), &icls); if (icls != NULL) { if (icls != (PyObject *)(Py_TYPE(inst)) && PyType_Check(icls)) { retval = PyType_IsSubtype( @@ -2611,7 +2599,7 @@ object_isinstance(PyObject *inst, PyObject *cls) if (!check_class(cls, "isinstance() arg 2 must be a type, a tuple of types, or a union")) return -1; - retval = _PyObject_LookupAttrId(inst, &PyId___class__, &icls); + retval = _PyObject_LookupAttr(inst, &_Py_ID(__class__), &icls); if (icls != NULL) { retval = abstract_issubclass(icls, cls); Py_DECREF(icls); @@ -2624,8 +2612,6 @@ object_isinstance(PyObject *inst, PyObject *cls) static int object_recursive_isinstance(PyThreadState *tstate, PyObject *inst, PyObject *cls) { - _Py_IDENTIFIER(__instancecheck__); - /* Quick test for an exact match */ if (Py_IS_TYPE(inst, (PyTypeObject *)cls)) { return 1; @@ -2656,7 +2642,7 @@ object_recursive_isinstance(PyThreadState *tstate, PyObject *inst, PyObject *cls return r; } - PyObject *checker = _PyObject_LookupSpecial(cls, &PyId___instancecheck__); + PyObject *checker = _PyObject_LookupSpecial(cls, &_Py_ID(__instancecheck__)); if (checker != NULL) { if (_Py_EnterRecursiveCall(tstate, " in __instancecheck__")) { Py_DECREF(checker); @@ -2715,7 +2701,6 @@ recursive_issubclass(PyObject *derived, PyObject *cls) static int object_issubclass(PyThreadState *tstate, PyObject *derived, PyObject *cls) { - _Py_IDENTIFIER(__subclasscheck__); PyObject *checker; /* We know what type's __subclasscheck__ does. */ @@ -2744,7 +2729,7 @@ object_issubclass(PyThreadState *tstate, PyObject *derived, PyObject *cls) return r; } - checker = _PyObject_LookupSpecial(cls, &PyId___subclasscheck__); + checker = _PyObject_LookupSpecial(cls, &_Py_ID(__subclasscheck__)); if (checker != NULL) { int ok = -1; if (_Py_EnterRecursiveCall(tstate, " in __subclasscheck__")) { @@ -2879,7 +2864,6 @@ PyIter_Next(PyObject *iter) PySendResult PyIter_Send(PyObject *iter, PyObject *arg, PyObject **result) { - _Py_IDENTIFIER(send); assert(arg != NULL); assert(result != NULL); if (Py_TYPE(iter)->tp_as_async && Py_TYPE(iter)->tp_as_async->am_send) { @@ -2891,7 +2875,7 @@ PyIter_Send(PyObject *iter, PyObject *arg, PyObject **result) *result = Py_TYPE(iter)->tp_iternext(iter); } else { - *result = _PyObject_CallMethodIdOneArg(iter, &PyId_send, arg); + *result = PyObject_CallMethodOneArg(iter, &_Py_ID(send), arg); } if (*result != NULL) { return PYGEN_NEXT; diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c index 8fce83dbfb50b..0ebb2ece39d5d 100644 --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -2112,10 +2112,9 @@ static PyObject * _common_reduce(PyByteArrayObject *self, int proto) { PyObject *dict; - _Py_IDENTIFIER(__dict__); char *buf; - if (_PyObject_LookupAttrId((PyObject *)self, &PyId___dict__, &dict) < 0) { + if (_PyObject_LookupAttr((PyObject *)self, &_Py_ID(__dict__), &dict) < 0) { return NULL; } if (dict == NULL) { @@ -2428,12 +2427,11 @@ PyDoc_STRVAR(length_hint_doc, static PyObject * bytearrayiter_reduce(bytesiterobject *it, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(iter); if (it->it_seq != NULL) { - return Py_BuildValue("N(O)n", _PyEval_GetBuiltinId(&PyId_iter), + return Py_BuildValue("N(O)n", _PyEval_GetBuiltin(&_Py_ID(iter)), it->it_seq, it->it_index); } else { - return Py_BuildValue("N(())", _PyEval_GetBuiltinId(&PyId_iter)); + return Py_BuildValue("N(())", _PyEval_GetBuiltin(&_Py_ID(iter))); } } diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c index cfc358f4deba2..3d8a21696d1c8 100644 --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -23,8 +23,6 @@ class bytes "PyBytesObject *" "&PyBytes_Type" #include "clinic/bytesobject.c.h" -_Py_IDENTIFIER(__bytes__); - /* PyBytesObject_SIZE gives the basic size of a bytes object; any memory allocation for a bytes object of length n should request PyBytesObject_SIZE + n bytes. @@ -530,7 +528,7 @@ format_obj(PyObject *v, const char **pbuf, Py_ssize_t *plen) return v; } /* does it support __bytes__? */ - func = _PyObject_LookupSpecial(v, &PyId___bytes__); + func = _PyObject_LookupSpecial(v, &_Py_ID(__bytes__)); if (func != NULL) { result = _PyObject_CallNoArgs(func); Py_DECREF(func); @@ -2582,7 +2580,7 @@ bytes_new_impl(PyTypeObject *type, PyObject *x, const char *encoding, /* We'd like to call PyObject_Bytes here, but we need to check for an integer argument before deferring to PyBytes_FromObject, something PyObject_Bytes doesn't do. */ - else if ((func = _PyObject_LookupSpecial(x, &PyId___bytes__)) != NULL) { + else if ((func = _PyObject_LookupSpecial(x, &_Py_ID(__bytes__))) != NULL) { bytes = _PyObject_CallNoArgs(func); Py_DECREF(func); if (bytes == NULL) @@ -3122,12 +3120,11 @@ PyDoc_STRVAR(length_hint_doc, static PyObject * striter_reduce(striterobject *it, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(iter); if (it->it_seq != NULL) { - return Py_BuildValue("N(O)n", _PyEval_GetBuiltinId(&PyId_iter), + return Py_BuildValue("N(O)n", _PyEval_GetBuiltin(&_Py_ID(iter)), it->it_seq, it->it_index); } else { - return Py_BuildValue("N(())", _PyEval_GetBuiltinId(&PyId_iter)); + return Py_BuildValue("N(())", _PyEval_GetBuiltin(&_Py_ID(iter))); } } diff --git a/Objects/call.c b/Objects/call.c index 310a2d732d170..c3939ea0d07da 100644 --- a/Objects/call.c +++ b/Objects/call.c @@ -607,7 +607,6 @@ callmethod(PyThreadState *tstate, PyObject* callable, const char *format, va_lis return _PyObject_CallFunctionVa(tstate, callable, format, va, is_size_t); } - PyObject * PyObject_CallMethod(PyObject *obj, const char *name, const char *format, ...) { @@ -658,6 +657,30 @@ PyEval_CallMethod(PyObject *obj, const char *name, const char *format, ...) } +PyObject * +_PyObject_CallMethod(PyObject *obj, PyObject *name, + const char *format, ...) +{ + PyThreadState *tstate = _PyThreadState_GET(); + if (obj == NULL || name == NULL) { + return null_error(tstate); + } + + PyObject *callable = PyObject_GetAttr(obj, name); + if (callable == NULL) { + return NULL; + } + + va_list va; + va_start(va, format); + PyObject *retval = callmethod(tstate, callable, format, va, 1); + va_end(va); + + Py_DECREF(callable); + return retval; +} + + PyObject * _PyObject_CallMethodId(PyObject *obj, _Py_Identifier *name, const char *format, ...) @@ -682,6 +705,17 @@ _PyObject_CallMethodId(PyObject *obj, _Py_Identifier *name, } +PyObject * _PyObject_CallMethodFormat(PyThreadState *tstate, PyObject *callable, + const char *format, ...) +{ + va_list va; + va_start(va, format); + PyObject *retval = callmethod(tstate, callable, format, va, 0); + va_end(va); + return retval; +} + + PyObject * _PyObject_CallMethod_SizeT(PyObject *obj, const char *name, const char *format, ...) diff --git a/Objects/classobject.c b/Objects/classobject.c index 9d4fc99f1f858..d7ccf31244e8b 100644 --- a/Objects/classobject.c +++ b/Objects/classobject.c @@ -9,8 +9,6 @@ #define TP_DESCR_GET(t) ((t)->tp_descr_get) -_Py_IDENTIFIER(__name__); -_Py_IDENTIFIER(__qualname__); PyObject * PyMethod_Function(PyObject *im) @@ -123,14 +121,13 @@ method_reduce(PyMethodObject *im, PyObject *Py_UNUSED(ignored)) PyObject *self = PyMethod_GET_SELF(im); PyObject *func = PyMethod_GET_FUNCTION(im); PyObject *funcname; - _Py_IDENTIFIER(getattr); - funcname = _PyObject_GetAttrId(func, &PyId___name__); + funcname = PyObject_GetAttr(func, &_Py_ID(__name__)); if (funcname == NULL) { return NULL; } - return Py_BuildValue("N(ON)", _PyEval_GetBuiltinId(&PyId_getattr), - self, funcname); + return Py_BuildValue( + "N(ON)", _PyEval_GetBuiltin(&_Py_ID(getattr)), self, funcname); } static PyMethodDef method_methods[] = { @@ -280,9 +277,9 @@ method_repr(PyMethodObject *a) PyObject *funcname, *result; const char *defname = "?"; - if (_PyObject_LookupAttrId(func, &PyId___qualname__, &funcname) < 0 || + if (_PyObject_LookupAttr(func, &_Py_ID(__qualname__), &funcname) < 0 || (funcname == NULL && - _PyObject_LookupAttrId(func, &PyId___name__, &funcname) < 0)) + _PyObject_LookupAttr(func, &_Py_ID(__name__), &funcname) < 0)) { return NULL; } @@ -515,7 +512,7 @@ instancemethod_repr(PyObject *self) return NULL; } - if (_PyObject_LookupAttrId(func, &PyId___name__, &funcname) < 0) { + if (_PyObject_LookupAttr(func, &_Py_ID(__name__), &funcname) < 0) { return NULL; } if (funcname != NULL && !PyUnicode_Check(funcname)) { diff --git a/Objects/complexobject.c b/Objects/complexobject.c index e0766de258805..9bd68d50c30ae 100644 --- a/Objects/complexobject.c +++ b/Objects/complexobject.c @@ -281,9 +281,8 @@ static PyObject * try_complex_special_method(PyObject *op) { PyObject *f; - _Py_IDENTIFIER(__complex__); - f = _PyObject_LookupSpecial(op, &PyId___complex__); + f = _PyObject_LookupSpecial(op, &_Py_ID(__complex__)); if (f) { PyObject *res = _PyObject_CallNoArgs(f); Py_DECREF(f); diff --git a/Objects/descrobject.c b/Objects/descrobject.c index 962136beae05d..95162683edd7d 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -7,8 +7,6 @@ #include "pycore_tuple.h" // _PyTuple_ITEMS() #include "structmember.h" // PyMemberDef -_Py_IDENTIFIER(getattr); - /*[clinic input] class mappingproxy "mappingproxyobject *" "&PyDictProxy_Type" class property "propertyobject *" "&PyProperty_Type" @@ -571,7 +569,6 @@ static PyObject * calculate_qualname(PyDescrObject *descr) { PyObject *type_qualname, *res; - _Py_IDENTIFIER(__qualname__); if (descr->d_name == NULL || !PyUnicode_Check(descr->d_name)) { PyErr_SetString(PyExc_TypeError, @@ -579,8 +576,8 @@ calculate_qualname(PyDescrObject *descr) return NULL; } - type_qualname = _PyObject_GetAttrId((PyObject *)descr->d_type, - &PyId___qualname__); + type_qualname = PyObject_GetAttr( + (PyObject *)descr->d_type, &_Py_ID(__qualname__)); if (type_qualname == NULL) return NULL; @@ -608,7 +605,7 @@ descr_get_qualname(PyDescrObject *descr, void *Py_UNUSED(ignored)) static PyObject * descr_reduce(PyDescrObject *descr, PyObject *Py_UNUSED(ignored)) { - return Py_BuildValue("N(OO)", _PyEval_GetBuiltinId(&PyId_getattr), + return Py_BuildValue("N(OO)", _PyEval_GetBuiltin(&_Py_ID(getattr)), PyDescr_TYPE(descr), PyDescr_NAME(descr)); } @@ -1086,8 +1083,7 @@ mappingproxy_get(mappingproxyobject *pp, PyObject *const *args, Py_ssize_t nargs { return NULL; } - _Py_IDENTIFIER(get); - return _PyObject_VectorcallMethodId(&PyId_get, newargs, + return _PyObject_VectorcallMethod(&_Py_ID(get), newargs, 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); } @@ -1095,36 +1091,31 @@ mappingproxy_get(mappingproxyobject *pp, PyObject *const *args, Py_ssize_t nargs static PyObject * mappingproxy_keys(mappingproxyobject *pp, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(keys); - return _PyObject_CallMethodIdNoArgs(pp->mapping, &PyId_keys); + return PyObject_CallMethodNoArgs(pp->mapping, &_Py_ID(keys)); } static PyObject * mappingproxy_values(mappingproxyobject *pp, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(values); - return _PyObject_CallMethodIdNoArgs(pp->mapping, &PyId_values); + return PyObject_CallMethodNoArgs(pp->mapping, &_Py_ID(values)); } static PyObject * mappingproxy_items(mappingproxyobject *pp, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(items); - return _PyObject_CallMethodIdNoArgs(pp->mapping, &PyId_items); + return PyObject_CallMethodNoArgs(pp->mapping, &_Py_ID(items)); } static PyObject * mappingproxy_copy(mappingproxyobject *pp, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(copy); - return _PyObject_CallMethodIdNoArgs(pp->mapping, &PyId_copy); + return PyObject_CallMethodNoArgs(pp->mapping, &_Py_ID(copy)); } static PyObject * mappingproxy_reversed(mappingproxyobject *pp, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(__reversed__); - return _PyObject_CallMethodIdNoArgs(pp->mapping, &PyId___reversed__); + return PyObject_CallMethodNoArgs(pp->mapping, &_Py_ID(__reversed__)); } /* WARNING: mappingproxy methods must not give access @@ -1321,7 +1312,7 @@ wrapper_repr(wrapperobject *wp) static PyObject * wrapper_reduce(wrapperobject *wp, PyObject *Py_UNUSED(ignored)) { - return Py_BuildValue("N(OO)", _PyEval_GetBuiltinId(&PyId_getattr), + return Py_BuildValue("N(OO)", _PyEval_GetBuiltin(&_Py_ID(getattr)), wp->self, PyDescr_NAME(wp->descr)); } @@ -1756,9 +1747,8 @@ property_init_impl(propertyobject *self, PyObject *fget, PyObject *fset, /* if no docstring given and the getter has one, use that one */ if ((doc == NULL || doc == Py_None) && fget != NULL) { - _Py_IDENTIFIER(__doc__); PyObject *get_doc; - int rc = _PyObject_LookupAttrId(fget, &PyId___doc__, &get_doc); + int rc = _PyObject_LookupAttr(fget, &_Py_ID(__doc__), &get_doc); if (rc <= 0) { return rc; } @@ -1770,7 +1760,8 @@ property_init_impl(propertyobject *self, PyObject *fget, PyObject *fset, in dict of the subclass instance instead, otherwise it gets shadowed by __doc__ in the class's dict. */ - int err = _PyObject_SetAttrId((PyObject *)self, &PyId___doc__, get_doc); + int err = PyObject_SetAttr( + (PyObject *)self, &_Py_ID(__doc__), get_doc); Py_DECREF(get_doc); if (err < 0) return -1; diff --git a/Objects/dictobject.c b/Objects/dictobject.c index c41bdb86dbfe9..18de42cba6903 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -1484,6 +1484,17 @@ PyDict_GetItemWithError(PyObject *op, PyObject *key) return value; } +PyObject * +_PyDict_GetItemWithError(PyObject *dp, PyObject *kv) +{ + assert(PyUnicode_CheckExact(kv)); + Py_hash_t hash = kv->ob_type->tp_hash(kv); + if (hash == -1) { + return NULL; + } + return _PyDict_GetItem_KnownHash(dp, kv, hash); +} + PyObject * _PyDict_GetItemIdWithError(PyObject *dp, struct _Py_Identifier *key) { @@ -2166,8 +2177,8 @@ dict_subscript(PyDictObject *mp, PyObject *key) if (!PyDict_CheckExact(mp)) { /* Look up __missing__ method if we're a subclass. */ PyObject *missing, *res; - _Py_IDENTIFIER(__missing__); - missing = _PyObject_LookupSpecial((PyObject *)mp, &PyId___missing__); + missing = _PyObject_LookupSpecial( + (PyObject *)mp, &_Py_ID(__missing__)); if (missing != NULL) { res = PyObject_CallOneArg(missing, key); Py_DECREF(missing); @@ -2369,9 +2380,8 @@ dict_update_arg(PyObject *self, PyObject *arg) if (PyDict_CheckExact(arg)) { return PyDict_Merge(self, arg, 1); } - _Py_IDENTIFIER(keys); PyObject *func; - if (_PyObject_LookupAttrId(arg, &PyId_keys, &func) < 0) { + if (_PyObject_LookupAttr(arg, &_Py_ID(keys), &func) < 0) { return -1; } if (func != NULL) { @@ -4128,7 +4138,6 @@ dict___reversed___impl(PyDictObject *self) static PyObject * dictiter_reduce(dictiterobject *di, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(iter); /* copy the iterator state */ dictiterobject tmp = *di; Py_XINCREF(tmp.di_dict); @@ -4138,7 +4147,7 @@ dictiter_reduce(dictiterobject *di, PyObject *Py_UNUSED(ignored)) if (list == NULL) { return NULL; } - return Py_BuildValue("N(N)", _PyEval_GetBuiltinId(&PyId_iter), list); + return Py_BuildValue("N(N)", _PyEval_GetBuiltin(&_Py_ID(iter)), list); } PyTypeObject PyDictRevIterItem_Type = { @@ -4408,9 +4417,8 @@ dictviews_sub(PyObject *self, PyObject *other) return NULL; } - _Py_IDENTIFIER(difference_update); - PyObject *tmp = _PyObject_CallMethodIdOneArg( - result, &PyId_difference_update, other); + PyObject *tmp = PyObject_CallMethodOneArg( + result, &_Py_ID(difference_update), other); if (tmp == NULL) { Py_DECREF(result); return NULL; @@ -4446,8 +4454,8 @@ _PyDictView_Intersect(PyObject* self, PyObject *other) /* if other is a set and self is smaller than other, reuse set intersection logic */ if (PySet_CheckExact(other) && len_self <= PyObject_Size(other)) { - _Py_IDENTIFIER(intersection); - return _PyObject_CallMethodIdObjArgs(other, &PyId_intersection, self, NULL); + return PyObject_CallMethodObjArgs( + other, &_Py_ID(intersection), self, NULL); } /* if other is another dict view, and it is bigger than self, @@ -4587,9 +4595,8 @@ dictitems_xor(PyObject *self, PyObject *other) } key = val1 = val2 = NULL; - _Py_IDENTIFIER(items); - PyObject *remaining_pairs = _PyObject_CallMethodIdNoArgs(temp_dict, - &PyId_items); + PyObject *remaining_pairs = PyObject_CallMethodNoArgs( + temp_dict, &_Py_ID(items)); if (remaining_pairs == NULL) { goto error; } @@ -4621,9 +4628,8 @@ dictviews_xor(PyObject* self, PyObject *other) return NULL; } - _Py_IDENTIFIER(symmetric_difference_update); - PyObject *tmp = _PyObject_CallMethodIdOneArg( - result, &PyId_symmetric_difference_update, other); + PyObject *tmp = PyObject_CallMethodOneArg( + result, &_Py_ID(symmetric_difference_update), other); if (tmp == NULL) { Py_DECREF(result); return NULL; diff --git a/Objects/enumobject.c b/Objects/enumobject.c index 828f1f925a0a1..d84bac6f4c9af 100644 --- a/Objects/enumobject.c +++ b/Objects/enumobject.c @@ -356,9 +356,8 @@ reversed_new_impl(PyTypeObject *type, PyObject *seq) Py_ssize_t n; PyObject *reversed_meth; reversedobject *ro; - _Py_IDENTIFIER(__reversed__); - reversed_meth = _PyObject_LookupSpecial(seq, &PyId___reversed__); + reversed_meth = _PyObject_LookupSpecial(seq, &_Py_ID(__reversed__)); if (reversed_meth == Py_None) { Py_DECREF(reversed_meth); PyErr_Format(PyExc_TypeError, diff --git a/Objects/exceptions.c b/Objects/exceptions.c index ea8a31076b060..9b1c9f1018077 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -1499,16 +1499,14 @@ ImportError_getstate(PyImportErrorObject *self) { PyObject *dict = ((PyBaseExceptionObject *)self)->dict; if (self->name || self->path) { - _Py_IDENTIFIER(name); - _Py_IDENTIFIER(path); dict = dict ? PyDict_Copy(dict) : PyDict_New(); if (dict == NULL) return NULL; - if (self->name && _PyDict_SetItemId(dict, &PyId_name, self->name) < 0) { + if (self->name && PyDict_SetItem(dict, &_Py_ID(name), self->name) < 0) { Py_DECREF(dict); return NULL; } - if (self->path && _PyDict_SetItemId(dict, &PyId_path, self->path) < 0) { + if (self->path && PyDict_SetItem(dict, &_Py_ID(path), self->path) < 0) { Py_DECREF(dict); return NULL; } diff --git a/Objects/fileobject.c b/Objects/fileobject.c index 8ca56a802b976..8dba5b9aea6db 100644 --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -26,8 +26,6 @@ extern "C" { #endif -_Py_IDENTIFIER(open); - /* External C interface */ PyObject * @@ -40,9 +38,9 @@ PyFile_FromFd(int fd, const char *name, const char *mode, int buffering, const c io = PyImport_ImportModule("_io"); if (io == NULL) return NULL; - stream = _PyObject_CallMethodId(io, &PyId_open, "isisssO", fd, mode, - buffering, encoding, errors, - newline, closefd ? Py_True : Py_False); + stream = _PyObject_CallMethod(io, &_Py_ID(open), "isisssO", fd, mode, + buffering, encoding, errors, + newline, closefd ? Py_True : Py_False); Py_DECREF(io); if (stream == NULL) return NULL; @@ -54,7 +52,6 @@ PyFile_FromFd(int fd, const char *name, const char *mode, int buffering, const c PyObject * PyFile_GetLine(PyObject *f, int n) { - _Py_IDENTIFIER(readline); PyObject *result; if (f == NULL) { @@ -63,10 +60,10 @@ PyFile_GetLine(PyObject *f, int n) } if (n <= 0) { - result = _PyObject_CallMethodIdNoArgs(f, &PyId_readline); + result = PyObject_CallMethodNoArgs(f, &_Py_ID(readline)); } else { - result = _PyObject_CallMethodId(f, &PyId_readline, "i", n); + result = _PyObject_CallMethod(f, &_Py_ID(readline), "i", n); } if (result != NULL && !PyBytes_Check(result) && !PyUnicode_Check(result)) { @@ -120,13 +117,12 @@ int PyFile_WriteObject(PyObject *v, PyObject *f, int flags) { PyObject *writer, *value, *result; - _Py_IDENTIFIER(write); if (f == NULL) { PyErr_SetString(PyExc_TypeError, "writeobject with NULL file"); return -1; } - writer = _PyObject_GetAttrId(f, &PyId_write); + writer = PyObject_GetAttr(f, &_Py_ID(write)); if (writer == NULL) return -1; if (flags & Py_PRINT_RAW) { @@ -182,12 +178,11 @@ PyObject_AsFileDescriptor(PyObject *o) { int fd; PyObject *meth; - _Py_IDENTIFIER(fileno); if (PyLong_Check(o)) { fd = _PyLong_AsInt(o); } - else if (_PyObject_LookupAttrId(o, &PyId_fileno, &meth) < 0) { + else if (_PyObject_LookupAttr(o, &_Py_ID(fileno), &meth) < 0) { return -1; } else if (meth != NULL) { @@ -509,8 +504,7 @@ PyFile_OpenCodeObject(PyObject *path) } else { iomod = PyImport_ImportModule("_io"); if (iomod) { - f = _PyObject_CallMethodId(iomod, &PyId_open, "Os", - path, "rb"); + f = _PyObject_CallMethod(iomod, &_Py_ID(open), "Os", path, "rb"); Py_DECREF(iomod); } } diff --git a/Objects/frameobject.c b/Objects/frameobject.c index dd29541f04b0e..dc210030e8a15 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -770,8 +770,6 @@ PyTypeObject PyFrame_Type = { 0, /* tp_dict */ }; -_Py_IDENTIFIER(__builtins__); - static void init_frame(InterpreterFrame *frame, PyFunctionObject *func, PyObject *locals) { @@ -1074,7 +1072,7 @@ PyFrame_GetBack(PyFrameObject *frame) PyObject* _PyEval_BuiltinsFromGlobals(PyThreadState *tstate, PyObject *globals) { - PyObject *builtins = _PyDict_GetItemIdWithError(globals, &PyId___builtins__); + PyObject *builtins = PyDict_GetItemWithError(globals, &_Py_ID(__builtins__)); if (builtins) { if (PyModule_Check(builtins)) { builtins = _PyModule_GetDict(builtins); diff --git a/Objects/funcobject.c b/Objects/funcobject.c index a8f006da8ad33..deacfd55dd286 100644 --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -79,8 +79,7 @@ PyFunction_NewWithQualName(PyObject *code, PyObject *globals, PyObject *qualname Py_INCREF(doc); // __module__: Use globals['__name__'] if it exists, or NULL. - _Py_IDENTIFIER(__name__); - PyObject *module = _PyDict_GetItemIdWithError(globals, &PyId___name__); + PyObject *module = PyDict_GetItemWithError(globals, &_Py_ID(__name__)); PyObject *builtins = NULL; if (module == NULL && _PyErr_Occurred(tstate)) { goto error; @@ -808,12 +807,7 @@ functools_wraps(PyObject *wrapper, PyObject *wrapped) { #define COPY_ATTR(ATTR) \ do { \ - _Py_IDENTIFIER(ATTR); \ - PyObject *attr = _PyUnicode_FromId(&PyId_ ## ATTR); \ - if (attr == NULL) { \ - return -1; \ - } \ - if (functools_copy_attr(wrapper, wrapped, attr) < 0) { \ + if (functools_copy_attr(wrapper, wrapped, &_Py_ID(ATTR)) < 0) { \ return -1; \ } \ } while (0) \ diff --git a/Objects/genericaliasobject.c b/Objects/genericaliasobject.c index dbe5d89b73962..b41644910f5d2 100644 --- a/Objects/genericaliasobject.c +++ b/Objects/genericaliasobject.c @@ -41,10 +41,6 @@ ga_traverse(PyObject *self, visitproc visit, void *arg) static int ga_repr_item(_PyUnicodeWriter *writer, PyObject *p) { - _Py_IDENTIFIER(__module__); - _Py_IDENTIFIER(__qualname__); - _Py_IDENTIFIER(__origin__); - _Py_IDENTIFIER(__args__); PyObject *qualname = NULL; PyObject *module = NULL; PyObject *r = NULL; @@ -57,12 +53,12 @@ ga_repr_item(_PyUnicodeWriter *writer, PyObject *p) goto done; } - if (_PyObject_LookupAttrId(p, &PyId___origin__, &tmp) < 0) { + if (_PyObject_LookupAttr(p, &_Py_ID(__origin__), &tmp) < 0) { goto done; } if (tmp != NULL) { Py_DECREF(tmp); - if (_PyObject_LookupAttrId(p, &PyId___args__, &tmp) < 0) { + if (_PyObject_LookupAttr(p, &_Py_ID(__args__), &tmp) < 0) { goto done; } if (tmp != NULL) { @@ -72,13 +68,13 @@ ga_repr_item(_PyUnicodeWriter *writer, PyObject *p) } } - if (_PyObject_LookupAttrId(p, &PyId___qualname__, &qualname) < 0) { + if (_PyObject_LookupAttr(p, &_Py_ID(__qualname__), &qualname) < 0) { goto done; } if (qualname == NULL) { goto use_repr; } - if (_PyObject_LookupAttrId(p, &PyId___module__, &module) < 0) { + if (_PyObject_LookupAttr(p, &_Py_ID(__module__), &module) < 0) { goto done; } if (module == NULL || module == Py_None) { @@ -218,9 +214,9 @@ _Py_make_parameters(PyObject *args) iparam += tuple_add(parameters, iparam, t); } else { - _Py_IDENTIFIER(__parameters__); PyObject *subparams; - if (_PyObject_LookupAttrId(t, &PyId___parameters__, &subparams) < 0) { + if (_PyObject_LookupAttr(t, &_Py_ID(__parameters__), + &subparams) < 0) { Py_DECREF(parameters); return NULL; } @@ -260,9 +256,8 @@ _Py_make_parameters(PyObject *args) static PyObject * subs_tvars(PyObject *obj, PyObject *params, PyObject **argitems) { - _Py_IDENTIFIER(__parameters__); PyObject *subparams; - if (_PyObject_LookupAttrId(obj, &PyId___parameters__, &subparams) < 0) { + if (_PyObject_LookupAttr(obj, &_Py_ID(__parameters__), &subparams) < 0) { return NULL; } if (subparams && PyTuple_Check(subparams) && PyTuple_GET_SIZE(subparams)) { diff --git a/Objects/genobject.c b/Objects/genobject.c index b2d402eba6333..b34931a95a680 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -320,7 +320,6 @@ static int gen_close_iter(PyObject *yf) { PyObject *retval = NULL; - _Py_IDENTIFIER(close); if (PyGen_CheckExact(yf) || PyCoro_CheckExact(yf)) { retval = gen_close((PyGenObject *)yf, NULL); @@ -329,7 +328,7 @@ gen_close_iter(PyObject *yf) } else { PyObject *meth; - if (_PyObject_LookupAttrId(yf, &PyId_close, &meth) < 0) { + if (_PyObject_LookupAttr(yf, &_Py_ID(close), &meth) < 0) { PyErr_WriteUnraisable(yf); } if (meth) { @@ -417,7 +416,6 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, PyObject *typ, PyObject *val, PyObject *tb) { PyObject *yf = _PyGen_yf(gen); - _Py_IDENTIFIER(throw); if (yf) { InterpreterFrame *frame = (InterpreterFrame *)gen->gi_iframe; @@ -462,7 +460,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, } else { /* `yf` is an iterator or a coroutine-like object. */ PyObject *meth; - if (_PyObject_LookupAttrId(yf, &PyId_throw, &meth) < 0) { + if (_PyObject_LookupAttr(yf, &_Py_ID(throw), &meth) < 0) { Py_DECREF(yf); return NULL; } diff --git a/Objects/iterobject.c b/Objects/iterobject.c index 5db6bc10fb3ff..1732a037600c9 100644 --- a/Objects/iterobject.c +++ b/Objects/iterobject.c @@ -10,8 +10,6 @@ typedef struct { PyObject *it_seq; /* Set to NULL when iterator is exhausted */ } seqiterobject; -_Py_IDENTIFIER(iter); - PyObject * PySeqIter_New(PyObject *seq) { @@ -106,10 +104,10 @@ static PyObject * iter_reduce(seqiterobject *it, PyObject *Py_UNUSED(ignored)) { if (it->it_seq != NULL) - return Py_BuildValue("N(O)n", _PyEval_GetBuiltinId(&PyId_iter), + return Py_BuildValue("N(O)n", _PyEval_GetBuiltin(&_Py_ID(iter)), it->it_seq, it->it_index); else - return Py_BuildValue("N(())", _PyEval_GetBuiltinId(&PyId_iter)); + return Py_BuildValue("N(())", _PyEval_GetBuiltin(&_Py_ID(iter))); } PyDoc_STRVAR(reduce_doc, "Return state information for pickling."); @@ -245,10 +243,10 @@ static PyObject * calliter_reduce(calliterobject *it, PyObject *Py_UNUSED(ignored)) { if (it->it_callable != NULL && it->it_sentinel != NULL) - return Py_BuildValue("N(OO)", _PyEval_GetBuiltinId(&PyId_iter), + return Py_BuildValue("N(OO)", _PyEval_GetBuiltin(&_Py_ID(iter)), it->it_callable, it->it_sentinel); else - return Py_BuildValue("N(())", _PyEval_GetBuiltinId(&PyId_iter)); + return Py_BuildValue("N(())", _PyEval_GetBuiltin(&_Py_ID(iter))); } static PyMethodDef calliter_methods[] = { diff --git a/Objects/listobject.c b/Objects/listobject.c index 752d9e00bb7bf..1ba1c1b053143 100644 --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -3505,25 +3505,25 @@ listreviter_setstate(listreviterobject *it, PyObject *state) static PyObject * listiter_reduce_general(void *_it, int forward) { - _Py_IDENTIFIER(iter); - _Py_IDENTIFIER(reversed); PyObject *list; /* the objects are not the same, index is of different types! */ if (forward) { listiterobject *it = (listiterobject *)_it; - if (it->it_seq) - return Py_BuildValue("N(O)n", _PyEval_GetBuiltinId(&PyId_iter), + if (it->it_seq) { + return Py_BuildValue("N(O)n", _PyEval_GetBuiltin(&_Py_ID(iter)), it->it_seq, it->it_index); + } } else { listreviterobject *it = (listreviterobject *)_it; - if (it->it_seq) - return Py_BuildValue("N(O)n", _PyEval_GetBuiltinId(&PyId_reversed), + if (it->it_seq) { + return Py_BuildValue("N(O)n", _PyEval_GetBuiltin(&_Py_ID(reversed)), it->it_seq, it->it_index); + } } /* empty iterator, create an empty list */ list = PyList_New(0); if (list == NULL) return NULL; - return Py_BuildValue("N(N)", _PyEval_GetBuiltinId(&PyId_iter), list); + return Py_BuildValue("N(N)", _PyEval_GetBuiltin(&_Py_ID(iter)), list); } diff --git a/Objects/longobject.c b/Objects/longobject.c index e1f6a6402841b..3438906d84275 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -22,9 +22,6 @@ class int "PyObject *" "&PyLong_Type" [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=ec0275e3422a36e3]*/ -_Py_IDENTIFIER(little); -_Py_IDENTIFIER(big); - /* Is this PyLong of size 1, 0 or -1? */ #define IS_MEDIUM_VALUE(x) (((size_t)Py_SIZE(x)) + 1U < 3U) @@ -5775,9 +5772,9 @@ int_to_bytes_impl(PyObject *self, Py_ssize_t length, PyObject *byteorder, if (byteorder == NULL) little_endian = 0; - else if (_PyUnicode_EqualToASCIIId(byteorder, &PyId_little)) + else if (_PyUnicode_Equal(byteorder, &_Py_ID(little))) little_endian = 1; - else if (_PyUnicode_EqualToASCIIId(byteorder, &PyId_big)) + else if (_PyUnicode_Equal(byteorder, &_Py_ID(big))) little_endian = 0; else { PyErr_SetString(PyExc_ValueError, @@ -5837,9 +5834,9 @@ int_from_bytes_impl(PyTypeObject *type, PyObject *bytes_obj, if (byteorder == NULL) little_endian = 0; - else if (_PyUnicode_EqualToASCIIId(byteorder, &PyId_little)) + else if (_PyUnicode_Equal(byteorder, &_Py_ID(little))) little_endian = 1; - else if (_PyUnicode_EqualToASCIIId(byteorder, &PyId_big)) + else if (_PyUnicode_Equal(byteorder, &_Py_ID(big))) little_endian = 0; else { PyErr_SetString(PyExc_ValueError, diff --git a/Objects/methodobject.c b/Objects/methodobject.c index 2df63cfdf6a81..93fac22ec437c 100644 --- a/Objects/methodobject.c +++ b/Objects/methodobject.c @@ -179,12 +179,10 @@ meth_dealloc(PyCFunctionObject *m) static PyObject * meth_reduce(PyCFunctionObject *m, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(getattr); - if (m->m_self == NULL || PyModule_Check(m->m_self)) return PyUnicode_FromString(m->m_ml->ml_name); - return Py_BuildValue("N(Os)", _PyEval_GetBuiltinId(&PyId_getattr), + return Py_BuildValue("N(Os)", _PyEval_GetBuiltin(&_Py_ID(getattr)), m->m_self, m->m_ml->ml_name); } @@ -223,14 +221,13 @@ meth_get__qualname__(PyCFunctionObject *m, void *closure) Otherwise return type(m.__self__).__qualname__ + '.' + m.__name__ (e.g. [].append.__qualname__ == 'list.append') */ PyObject *type, *type_qualname, *res; - _Py_IDENTIFIER(__qualname__); if (m->m_self == NULL || PyModule_Check(m->m_self)) return PyUnicode_FromString(m->m_ml->ml_name); type = PyType_Check(m->m_self) ? m->m_self : (PyObject*)Py_TYPE(m->m_self); - type_qualname = _PyObject_GetAttrId(type, &PyId___qualname__); + type_qualname = PyObject_GetAttr(type, &_Py_ID(__qualname__)); if (type_qualname == NULL) return NULL; diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index 1d649a7932098..bd5e5611ec27e 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -10,13 +10,6 @@ static Py_ssize_t max_module_number; -_Py_IDENTIFIER(__doc__); -_Py_IDENTIFIER(__name__); -_Py_IDENTIFIER(__spec__); -_Py_IDENTIFIER(__dict__); -_Py_IDENTIFIER(__dir__); -_Py_IDENTIFIER(__annotations__); - static PyMemberDef module_members[] = { {"__dict__", T_OBJECT, offsetof(PyModuleObject, md_dict), READONLY}, {0} @@ -61,22 +54,19 @@ static int module_init_dict(PyModuleObject *mod, PyObject *md_dict, PyObject *name, PyObject *doc) { - _Py_IDENTIFIER(__package__); - _Py_IDENTIFIER(__loader__); - assert(md_dict != NULL); if (doc == NULL) doc = Py_None; - if (_PyDict_SetItemId(md_dict, &PyId___name__, name) != 0) + if (PyDict_SetItem(md_dict, &_Py_ID(__name__), name) != 0) return -1; - if (_PyDict_SetItemId(md_dict, &PyId___doc__, doc) != 0) + if (PyDict_SetItem(md_dict, &_Py_ID(__doc__), doc) != 0) return -1; - if (_PyDict_SetItemId(md_dict, &PyId___package__, Py_None) != 0) + if (PyDict_SetItem(md_dict, &_Py_ID(__package__), Py_None) != 0) return -1; - if (_PyDict_SetItemId(md_dict, &PyId___loader__, Py_None) != 0) + if (PyDict_SetItem(md_dict, &_Py_ID(__loader__), Py_None) != 0) return -1; - if (_PyDict_SetItemId(md_dict, &PyId___spec__, Py_None) != 0) + if (PyDict_SetItem(md_dict, &_Py_ID(__spec__), Py_None) != 0) return -1; if (PyUnicode_CheckExact(name)) { Py_INCREF(name); @@ -474,7 +464,7 @@ PyModule_SetDocString(PyObject *m, const char *doc) PyObject *v; v = PyUnicode_FromString(doc); - if (v == NULL || _PyObject_SetAttrId(m, &PyId___doc__, v) != 0) { + if (v == NULL || PyObject_SetAttr(m, &_Py_ID(__doc__), v) != 0) { Py_XDECREF(v); return -1; } @@ -503,7 +493,7 @@ PyModule_GetNameObject(PyObject *m) } d = ((PyModuleObject *)m)->md_dict; if (d == NULL || !PyDict_Check(d) || - (name = _PyDict_GetItemIdWithError(d, &PyId___name__)) == NULL || + (name = PyDict_GetItemWithError(d, &_Py_ID(__name__))) == NULL || !PyUnicode_Check(name)) { if (!PyErr_Occurred()) { @@ -528,7 +518,6 @@ PyModule_GetName(PyObject *m) PyObject* PyModule_GetFilenameObject(PyObject *m) { - _Py_IDENTIFIER(__file__); PyObject *d; PyObject *fileobj; if (!PyModule_Check(m)) { @@ -537,7 +526,7 @@ PyModule_GetFilenameObject(PyObject *m) } d = ((PyModuleObject *)m)->md_dict; if (d == NULL || - (fileobj = _PyDict_GetItemIdWithError(d, &PyId___file__)) == NULL || + (fileobj = PyDict_GetItemWithError(d, &_Py_ID(__file__))) == NULL || !PyUnicode_Check(fileobj)) { if (!PyErr_Occurred()) { @@ -726,8 +715,7 @@ int _PyModuleSpec_IsInitializing(PyObject *spec) { if (spec != NULL) { - _Py_IDENTIFIER(_initializing); - PyObject *value = _PyObject_GetAttrId(spec, &PyId__initializing); + PyObject *value = PyObject_GetAttr(spec, &_Py_ID(_initializing)); if (value != NULL) { int initializing = PyObject_IsTrue(value); Py_DECREF(value); @@ -750,8 +738,7 @@ _PyModuleSpec_IsUninitializedSubmodule(PyObject *spec, PyObject *name) return 0; } - _Py_IDENTIFIER(_uninitialized_submodules); - PyObject *value = _PyObject_GetAttrId(spec, &PyId__uninitialized_submodules); + PyObject *value = PyObject_GetAttr(spec, &_Py_ID(_uninitialized_submodules)); if (value == NULL) { return 0; } @@ -774,18 +761,17 @@ module_getattro(PyModuleObject *m, PyObject *name) } PyErr_Clear(); assert(m->md_dict != NULL); - _Py_IDENTIFIER(__getattr__); - getattr = _PyDict_GetItemIdWithError(m->md_dict, &PyId___getattr__); + getattr = PyDict_GetItemWithError(m->md_dict, &_Py_ID(__getattr__)); if (getattr) { return PyObject_CallOneArg(getattr, name); } if (PyErr_Occurred()) { return NULL; } - mod_name = _PyDict_GetItemIdWithError(m->md_dict, &PyId___name__); + mod_name = PyDict_GetItemWithError(m->md_dict, &_Py_ID(__name__)); if (mod_name && PyUnicode_Check(mod_name)) { Py_INCREF(mod_name); - PyObject *spec = _PyDict_GetItemIdWithError(m->md_dict, &PyId___spec__); + PyObject *spec = PyDict_GetItemWithError(m->md_dict, &_Py_ID(__spec__)); if (spec == NULL && PyErr_Occurred()) { Py_DECREF(mod_name); return NULL; @@ -861,11 +847,11 @@ static PyObject * module_dir(PyObject *self, PyObject *args) { PyObject *result = NULL; - PyObject *dict = _PyObject_GetAttrId(self, &PyId___dict__); + PyObject *dict = PyObject_GetAttr(self, &_Py_ID(__dict__)); if (dict != NULL) { if (PyDict_Check(dict)) { - PyObject *dirfunc = _PyDict_GetItemIdWithError(dict, &PyId___dir__); + PyObject *dirfunc = PyDict_GetItemWithError(dict, &_Py_ID(__dir__)); if (dirfunc) { result = _PyObject_CallNoArgs(dirfunc); } @@ -891,7 +877,7 @@ static PyMethodDef module_methods[] = { static PyObject * module_get_annotations(PyModuleObject *m, void *Py_UNUSED(ignored)) { - PyObject *dict = _PyObject_GetAttrId((PyObject *)m, &PyId___dict__); + PyObject *dict = PyObject_GetAttr((PyObject *)m, &_Py_ID(__dict__)); if ((dict == NULL) || !PyDict_Check(dict)) { PyErr_Format(PyExc_TypeError, ".__dict__ is not a dictionary"); @@ -901,8 +887,8 @@ module_get_annotations(PyModuleObject *m, void *Py_UNUSED(ignored)) PyObject *annotations; /* there's no _PyDict_GetItemId without WithError, so let's LBYL. */ - if (_PyDict_ContainsId(dict, &PyId___annotations__)) { - annotations = _PyDict_GetItemIdWithError(dict, &PyId___annotations__); + if (PyDict_Contains(dict, &_Py_ID(__annotations__))) { + annotations = PyDict_GetItemWithError(dict, &_Py_ID(__annotations__)); /* ** _PyDict_GetItemIdWithError could still fail, ** for instance with a well-timed Ctrl-C or a MemoryError. @@ -914,7 +900,8 @@ module_get_annotations(PyModuleObject *m, void *Py_UNUSED(ignored)) } else { annotations = PyDict_New(); if (annotations) { - int result = _PyDict_SetItemId(dict, &PyId___annotations__, annotations); + int result = PyDict_SetItem( + dict, &_Py_ID(__annotations__), annotations); if (result) { Py_CLEAR(annotations); } @@ -928,7 +915,7 @@ static int module_set_annotations(PyModuleObject *m, PyObject *value, void *Py_UNUSED(ignored)) { int ret = -1; - PyObject *dict = _PyObject_GetAttrId((PyObject *)m, &PyId___dict__); + PyObject *dict = PyObject_GetAttr((PyObject *)m, &_Py_ID(__dict__)); if ((dict == NULL) || !PyDict_Check(dict)) { PyErr_Format(PyExc_TypeError, ".__dict__ is not a dictionary"); @@ -937,17 +924,17 @@ module_set_annotations(PyModuleObject *m, PyObject *value, void *Py_UNUSED(ignor if (value != NULL) { /* set */ - ret = _PyDict_SetItemId(dict, &PyId___annotations__, value); + ret = PyDict_SetItem(dict, &_Py_ID(__annotations__), value); goto exit; } /* delete */ - if (!_PyDict_ContainsId(dict, &PyId___annotations__)) { + if (!PyDict_Contains(dict, &_Py_ID(__annotations__))) { PyErr_Format(PyExc_AttributeError, "__annotations__"); goto exit; } - ret = _PyDict_DelItemId(dict, &PyId___annotations__); + ret = PyDict_DelItem(dict, &_Py_ID(__annotations__)); exit: Py_XDECREF(dict); diff --git a/Objects/object.c b/Objects/object.c index 3082e70e7e230..3044c862fb9da 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -31,11 +31,6 @@ extern "C" { /* Defined in tracemalloc.c */ extern void _PyMem_DumpTraceback(int fd, const void *ptr); -_Py_IDENTIFIER(Py_Repr); -_Py_IDENTIFIER(__bytes__); -_Py_IDENTIFIER(__dir__); -_Py_IDENTIFIER(__isabstractmethod__); - int _PyObject_CheckConsistency(PyObject *op, int check_content) @@ -562,7 +557,7 @@ PyObject_Bytes(PyObject *v) return v; } - func = _PyObject_LookupSpecial(v, &PyId___bytes__); + func = _PyObject_LookupSpecial(v, &_Py_ID(__bytes__)); if (func != NULL) { result = _PyObject_CallNoArgs(func); Py_DECREF(func); @@ -600,12 +595,9 @@ def _PyObject_FunctionStr(x): PyObject * _PyObject_FunctionStr(PyObject *x) { - _Py_IDENTIFIER(__module__); - _Py_IDENTIFIER(__qualname__); - _Py_IDENTIFIER(builtins); assert(!PyErr_Occurred()); PyObject *qualname; - int ret = _PyObject_LookupAttrId(x, &PyId___qualname__, &qualname); + int ret = _PyObject_LookupAttr(x, &_Py_ID(__qualname__), &qualname); if (qualname == NULL) { if (ret < 0) { return NULL; @@ -614,13 +606,9 @@ _PyObject_FunctionStr(PyObject *x) } PyObject *module; PyObject *result = NULL; - ret = _PyObject_LookupAttrId(x, &PyId___module__, &module); + ret = _PyObject_LookupAttr(x, &_Py_ID(__module__), &module); if (module != NULL && module != Py_None) { - PyObject *builtinsname = _PyUnicode_FromId(&PyId_builtins); - if (builtinsname == NULL) { - goto done; - } - ret = PyObject_RichCompareBool(module, builtinsname, Py_NE); + ret = PyObject_RichCompareBool(module, &_Py_ID(builtins), Py_NE); if (ret < 0) { // error goto done; @@ -858,7 +846,7 @@ _PyObject_IsAbstract(PyObject *obj) if (obj == NULL) return 0; - res = _PyObject_LookupAttrId(obj, &PyId___isabstractmethod__, &isabstract); + res = _PyObject_LookupAttr(obj, &_Py_ID(__isabstractmethod__), &isabstract); if (res > 0) { res = PyObject_IsTrue(isabstract); Py_DECREF(isabstract); @@ -892,8 +880,6 @@ static inline int set_attribute_error_context(PyObject* v, PyObject* name) { assert(PyErr_Occurred()); - _Py_IDENTIFIER(name); - _Py_IDENTIFIER(obj); // Intercept AttributeError exceptions and augment them to offer // suggestions later. if (PyErr_ExceptionMatches(PyExc_AttributeError)){ @@ -901,8 +887,8 @@ set_attribute_error_context(PyObject* v, PyObject* name) PyErr_Fetch(&type, &value, &traceback); PyErr_NormalizeException(&type, &value, &traceback); if (PyErr_GivenExceptionMatches(value, PyExc_AttributeError) && - (_PyObject_SetAttrId(value, &PyId_name, name) || - _PyObject_SetAttrId(value, &PyId_obj, v))) { + (PyObject_SetAttr(value, &_Py_ID(name), name) || + PyObject_SetAttr(value, &_Py_ID(obj), v))) { return 1; } PyErr_Restore(type, value, traceback); @@ -1569,7 +1555,7 @@ static PyObject * _dir_object(PyObject *obj) { PyObject *result, *sorted; - PyObject *dirfunc = _PyObject_LookupSpecial(obj, &PyId___dir__); + PyObject *dirfunc = _PyObject_LookupSpecial(obj, &_Py_ID(__dir__)); assert(obj != NULL); if (dirfunc == NULL) { @@ -2148,7 +2134,7 @@ Py_ReprEnter(PyObject *obj) early on startup. */ if (dict == NULL) return 0; - list = _PyDict_GetItemIdWithError(dict, &PyId_Py_Repr); + list = PyDict_GetItemWithError(dict, &_Py_ID(Py_Repr)); if (list == NULL) { if (PyErr_Occurred()) { return -1; @@ -2156,7 +2142,7 @@ Py_ReprEnter(PyObject *obj) list = PyList_New(0); if (list == NULL) return -1; - if (_PyDict_SetItemId(dict, &PyId_Py_Repr, list) < 0) + if (PyDict_SetItem(dict, &_Py_ID(Py_Repr), list) < 0) return -1; Py_DECREF(list); } @@ -2184,7 +2170,7 @@ Py_ReprLeave(PyObject *obj) if (dict == NULL) goto finally; - list = _PyDict_GetItemIdWithError(dict, &PyId_Py_Repr); + list = PyDict_GetItemWithError(dict, &_Py_ID(Py_Repr)); if (list == NULL || !PyList_Check(list)) goto finally; diff --git a/Objects/odictobject.c b/Objects/odictobject.c index e27bcecd75ca3..c207593ab79f7 100644 --- a/Objects/odictobject.c +++ b/Objects/odictobject.c @@ -525,8 +525,6 @@ struct _odictnode { #define _odict_FOREACH(od, node) \ for (node = _odict_FIRST(od); node != NULL; node = _odictnode_NEXT(node)) -_Py_IDENTIFIER(items); - /* Return the index into the hash table, regardless of a valid node. */ static Py_ssize_t _odict_get_index_raw(PyODictObject *od, PyObject *key, Py_hash_t hash) @@ -949,12 +947,11 @@ PyDoc_STRVAR(odict_reduce__doc__, "Return state information for pickling"); static PyObject * odict_reduce(register PyODictObject *od, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(__dict__); PyObject *dict = NULL, *result = NULL; PyObject *items_iter, *items, *args = NULL; /* capture any instance state */ - dict = _PyObject_GetAttrId((PyObject *)od, &PyId___dict__); + dict = PyObject_GetAttr((PyObject *)od, &_Py_ID(__dict__)); if (dict == NULL) goto Done; else { @@ -973,7 +970,7 @@ odict_reduce(register PyODictObject *od, PyObject *Py_UNUSED(ignored)) if (args == NULL) goto Done; - items = _PyObject_CallMethodIdNoArgs((PyObject *)od, &PyId_items); + items = PyObject_CallMethodNoArgs((PyObject *)od, &_Py_ID(items)); if (items == NULL) goto Done; @@ -1431,8 +1428,8 @@ odict_repr(PyODictObject *self) } } else { - PyObject *items = _PyObject_CallMethodIdNoArgs((PyObject *)self, - &PyId_items); + PyObject *items = PyObject_CallMethodNoArgs( + (PyObject *)self, &_Py_ID(items)); if (items == NULL) goto Done; pieces = PySequence_List(items); @@ -1808,7 +1805,6 @@ PyDoc_STRVAR(reduce_doc, "Return state information for pickling"); static PyObject * odictiter_reduce(odictiterobject *di, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(iter); /* copy the iterator state */ odictiterobject tmp = *di; Py_XINCREF(tmp.di_odict); @@ -1821,7 +1817,7 @@ odictiter_reduce(odictiterobject *di, PyObject *Py_UNUSED(ignored)) if (list == NULL) { return NULL; } - return Py_BuildValue("N(N)", _PyEval_GetBuiltinId(&PyId_iter), list); + return Py_BuildValue("N(N)", _PyEval_GetBuiltin(&_Py_ID(iter)), list); } static PyMethodDef odictiter_methods[] = { @@ -2217,9 +2213,8 @@ mutablemapping_update_arg(PyObject *self, PyObject *arg) Py_DECREF(items); return res; } - _Py_IDENTIFIER(keys); PyObject *func; - if (_PyObject_LookupAttrId(arg, &PyId_keys, &func) < 0) { + if (_PyObject_LookupAttr(arg, &_Py_ID(keys), &func) < 0) { return -1; } if (func != NULL) { @@ -2251,7 +2246,7 @@ mutablemapping_update_arg(PyObject *self, PyObject *arg) } return 0; } - if (_PyObject_LookupAttrId(arg, &PyId_items, &func) < 0) { + if (_PyObject_LookupAttr(arg, &_Py_ID(items), &func) < 0) { return -1; } if (func != NULL) { diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c index a848d67a65152..5d583b2edf0e9 100644 --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -21,8 +21,6 @@ typedef struct { PyObject *length; } rangeobject; -_Py_IDENTIFIER(iter); - /* Helper function for validating step. Always returns a new reference or NULL on error. */ @@ -813,8 +811,8 @@ rangeiter_reduce(rangeiterobject *r, PyObject *Py_UNUSED(ignored)) if (range == NULL) goto err; /* return the result */ - return Py_BuildValue("N(N)l", _PyEval_GetBuiltinId(&PyId_iter), - range, r->index); + return Py_BuildValue( + "N(N)l", _PyEval_GetBuiltin(&_Py_ID(iter)), range, r->index); err: Py_XDECREF(start); Py_XDECREF(stop); @@ -967,8 +965,8 @@ longrangeiter_reduce(longrangeiterobject *r, PyObject *Py_UNUSED(ignored)) } /* return the result */ - return Py_BuildValue("N(N)O", _PyEval_GetBuiltinId(&PyId_iter), - range, r->index); + return Py_BuildValue( + "N(N)O", _PyEval_GetBuiltin(&_Py_ID(iter)), range, r->index); } static PyObject * diff --git a/Objects/setobject.c b/Objects/setobject.c index ca3cfe8196467..fe124945b1c7e 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -770,7 +770,6 @@ static PyObject *setiter_iternext(setiterobject *si); static PyObject * setiter_reduce(setiterobject *si, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(iter); /* copy the iterator state */ setiterobject tmp = *si; Py_XINCREF(tmp.si_set); @@ -781,7 +780,7 @@ setiter_reduce(setiterobject *si, PyObject *Py_UNUSED(ignored)) if (list == NULL) { return NULL; } - return Py_BuildValue("N(N)", _PyEval_GetBuiltinId(&PyId_iter), list); + return Py_BuildValue("N(N)", _PyEval_GetBuiltin(&_Py_ID(iter)), list); } PyDoc_STRVAR(reduce_doc, "Return state information for pickling."); @@ -1906,7 +1905,6 @@ static PyObject * set_reduce(PySetObject *so, PyObject *Py_UNUSED(ignored)) { PyObject *keys=NULL, *args=NULL, *result=NULL, *dict=NULL; - _Py_IDENTIFIER(__dict__); keys = PySequence_List((PyObject *)so); if (keys == NULL) @@ -1914,7 +1912,7 @@ set_reduce(PySetObject *so, PyObject *Py_UNUSED(ignored)) args = PyTuple_Pack(1, keys); if (args == NULL) goto done; - if (_PyObject_LookupAttrId((PyObject *)so, &PyId___dict__, &dict) < 0) { + if (_PyObject_LookupAttr((PyObject *)so, &_Py_ID(__dict__), &dict) < 0) { goto done; } if (dict == NULL) { diff --git a/Objects/structseq.c b/Objects/structseq.c index 092e1634656a3..229e3d893ff6a 100644 --- a/Objects/structseq.c +++ b/Objects/structseq.c @@ -23,17 +23,9 @@ static const char match_args_key[] = "__match_args__"; They are only allowed for indices < n_visible_fields. */ const char * const PyStructSequence_UnnamedField = "unnamed field"; -_Py_IDENTIFIER(n_sequence_fields); -_Py_IDENTIFIER(n_fields); -_Py_IDENTIFIER(n_unnamed_fields); - static Py_ssize_t -get_type_attr_as_size(PyTypeObject *tp, _Py_Identifier *id) +get_type_attr_as_size(PyTypeObject *tp, PyObject *name) { - PyObject *name = _PyUnicode_FromId(id); - if (name == NULL) { - return -1; - } PyObject *v = PyDict_GetItemWithError(tp->tp_dict, name); if (v == NULL && !PyErr_Occurred()) { PyErr_Format(PyExc_TypeError, @@ -44,11 +36,14 @@ get_type_attr_as_size(PyTypeObject *tp, _Py_Identifier *id) } #define VISIBLE_SIZE(op) Py_SIZE(op) -#define VISIBLE_SIZE_TP(tp) get_type_attr_as_size(tp, &PyId_n_sequence_fields) -#define REAL_SIZE_TP(tp) get_type_attr_as_size(tp, &PyId_n_fields) +#define VISIBLE_SIZE_TP(tp) \ + get_type_attr_as_size(tp, &_Py_ID(n_sequence_fields)) +#define REAL_SIZE_TP(tp) \ + get_type_attr_as_size(tp, &_Py_ID(n_fields)) #define REAL_SIZE(op) REAL_SIZE_TP(Py_TYPE(op)) -#define UNNAMED_FIELDS_TP(tp) get_type_attr_as_size(tp, &PyId_n_unnamed_fields) +#define UNNAMED_FIELDS_TP(tp) \ + get_type_attr_as_size(tp, &_Py_ID(n_unnamed_fields)) #define UNNAMED_FIELDS(op) UNNAMED_FIELDS_TP(Py_TYPE(op)) @@ -622,21 +617,3 @@ PyStructSequence_NewType(PyStructSequence_Desc *desc) { return _PyStructSequence_NewType(desc, 0); } - - -/* runtime lifecycle */ - -PyStatus _PyStructSequence_InitState(PyInterpreterState *interp) -{ - if (!_Py_IsMainInterpreter(interp)) { - return _PyStatus_OK(); - } - - if (_PyUnicode_FromId(&PyId_n_sequence_fields) == NULL - || _PyUnicode_FromId(&PyId_n_fields) == NULL - || _PyUnicode_FromId(&PyId_n_unnamed_fields) == NULL) - { - return _PyStatus_ERR("can't initialize structseq state"); - } - return _PyStatus_OK(); -} diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c index 86f541a96a5a1..ec9147cdcfedf 100644 --- a/Objects/tupleobject.c +++ b/Objects/tupleobject.c @@ -1193,12 +1193,11 @@ PyDoc_STRVAR(length_hint_doc, "Private method returning an estimate of len(list( static PyObject * tupleiter_reduce(tupleiterobject *it, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(iter); if (it->it_seq) - return Py_BuildValue("N(O)n", _PyEval_GetBuiltinId(&PyId_iter), + return Py_BuildValue("N(O)n", _PyEval_GetBuiltin(&_Py_ID(iter)), it->it_seq, it->it_index); else - return Py_BuildValue("N(())", _PyEval_GetBuiltinId(&PyId_iter)); + return Py_BuildValue("N(())", _PyEval_GetBuiltin(&_Py_ID(iter))); } static PyObject * diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 0c893eaa75b6f..ef3549ce68f7f 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -59,30 +59,6 @@ typedef struct PySlot_Offset { # define INTERN_NAME_STRINGS #endif -/* alphabetical order */ -_Py_IDENTIFIER(__abstractmethods__); -_Py_IDENTIFIER(__annotations__); -_Py_IDENTIFIER(__class__); -_Py_IDENTIFIER(__class_getitem__); -_Py_IDENTIFIER(__classcell__); -_Py_IDENTIFIER(__delitem__); -_Py_IDENTIFIER(__dict__); -_Py_IDENTIFIER(__doc__); -_Py_IDENTIFIER(__getattribute__); -_Py_IDENTIFIER(__getitem__); -_Py_IDENTIFIER(__hash__); -_Py_IDENTIFIER(__init_subclass__); -_Py_IDENTIFIER(__len__); -_Py_IDENTIFIER(__module__); -_Py_IDENTIFIER(__name__); -_Py_IDENTIFIER(__new__); -_Py_IDENTIFIER(__qualname__); -_Py_IDENTIFIER(__set_name__); -_Py_IDENTIFIER(__setitem__); -_Py_IDENTIFIER(__weakref__); -_Py_IDENTIFIER(builtins); -_Py_IDENTIFIER(mro); - static PyObject * slot_tp_new(PyTypeObject *type, PyObject *args, PyObject *kwds); @@ -90,7 +66,7 @@ static void clear_slotdefs(void); static PyObject * -lookup_maybe_method(PyObject *self, _Py_Identifier *attrid, int *unbound); +lookup_maybe_method(PyObject *self, PyObject *attr, int *unbound); static int slot_tp_setattro(PyObject *self, PyObject *name, PyObject *value); @@ -175,7 +151,7 @@ _PyType_CheckConsistency(PyTypeObject *type) if (type->tp_flags & Py_TPFLAGS_DISALLOW_INSTANTIATION) { CHECK(type->tp_new == NULL); - CHECK(_PyDict_ContainsId(type->tp_dict, &PyId___new__) == 0); + CHECK(PyDict_Contains(type->tp_dict, &_Py_ID(__new__)) == 0); } return 1; @@ -374,11 +350,11 @@ type_mro_modified(PyTypeObject *type, PyObject *bases) { if (custom) { mro_meth = lookup_maybe_method( - (PyObject *)type, &PyId_mro, &unbound); + (PyObject *)type, &_Py_ID(mro), &unbound); if (mro_meth == NULL) goto clear; type_mro_meth = lookup_maybe_method( - (PyObject *)&PyType_Type, &PyId_mro, &unbound); + (PyObject *)&PyType_Type, &_Py_ID(mro), &unbound); if (type_mro_meth == NULL) goto clear; if (mro_meth != type_mro_meth) @@ -572,7 +548,7 @@ type_module(PyTypeObject *type, void *context) PyObject *mod; if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) { - mod = _PyDict_GetItemIdWithError(type->tp_dict, &PyId___module__); + mod = PyDict_GetItemWithError(type->tp_dict, &_Py_ID(__module__)); if (mod == NULL) { if (!PyErr_Occurred()) { PyErr_Format(PyExc_AttributeError, "__module__"); @@ -590,8 +566,8 @@ type_module(PyTypeObject *type, void *context) PyUnicode_InternInPlace(&mod); } else { - mod = _PyUnicode_FromId(&PyId_builtins); - Py_XINCREF(mod); + mod = &_Py_ID(builtins); + Py_INCREF(mod); } } return mod; @@ -605,7 +581,7 @@ type_set_module(PyTypeObject *type, PyObject *value, void *context) PyType_Modified(type); - return _PyDict_SetItemId(type->tp_dict, &PyId___module__, value); + return PyDict_SetItem(type->tp_dict, &_Py_ID(__module__), value); } static PyObject * @@ -615,12 +591,11 @@ type_abstractmethods(PyTypeObject *type, void *context) /* type itself has an __abstractmethods__ descriptor (this). Don't return that. */ if (type != &PyType_Type) - mod = _PyDict_GetItemIdWithError(type->tp_dict, &PyId___abstractmethods__); + mod = PyDict_GetItemWithError(type->tp_dict, + &_Py_ID(__abstractmethods__)); if (!mod) { if (!PyErr_Occurred()) { - PyObject *message = _PyUnicode_FromId(&PyId___abstractmethods__); - if (message) - PyErr_SetObject(PyExc_AttributeError, message); + PyErr_SetObject(PyExc_AttributeError, &_Py_ID(__abstractmethods__)); } return NULL; } @@ -640,15 +615,13 @@ type_set_abstractmethods(PyTypeObject *type, PyObject *value, void *context) abstract = PyObject_IsTrue(value); if (abstract < 0) return -1; - res = _PyDict_SetItemId(type->tp_dict, &PyId___abstractmethods__, value); + res = PyDict_SetItem(type->tp_dict, &_Py_ID(__abstractmethods__), value); } else { abstract = 0; - res = _PyDict_DelItemId(type->tp_dict, &PyId___abstractmethods__); + res = PyDict_DelItem(type->tp_dict, &_Py_ID(__abstractmethods__)); if (res && PyErr_ExceptionMatches(PyExc_KeyError)) { - PyObject *message = _PyUnicode_FromId(&PyId___abstractmethods__); - if (message) - PyErr_SetObject(PyExc_AttributeError, message); + PyErr_SetObject(PyExc_AttributeError, &_Py_ID(__abstractmethods__)); return -1; } } @@ -902,7 +875,7 @@ type_get_doc(PyTypeObject *type, void *context) if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE) && type->tp_doc != NULL) { return _PyType_GetDocFromInternalDoc(type->tp_name, type->tp_doc); } - result = _PyDict_GetItemIdWithError(type->tp_dict, &PyId___doc__); + result = PyDict_GetItemWithError(type->tp_dict, &_Py_ID(__doc__)); if (result == NULL) { if (!PyErr_Occurred()) { result = Py_None; @@ -931,7 +904,7 @@ type_set_doc(PyTypeObject *type, PyObject *value, void *context) if (!check_set_special_type_attr(type, value, "__doc__")) return -1; PyType_Modified(type); - return _PyDict_SetItemId(type->tp_dict, &PyId___doc__, value); + return PyDict_SetItem(type->tp_dict, &_Py_ID(__doc__), value); } static PyObject * @@ -944,17 +917,18 @@ type_get_annotations(PyTypeObject *type, void *context) PyObject *annotations; /* there's no _PyDict_GetItemId without WithError, so let's LBYL. */ - if (_PyDict_ContainsId(type->tp_dict, &PyId___annotations__)) { - annotations = _PyDict_GetItemIdWithError(type->tp_dict, &PyId___annotations__); + if (PyDict_Contains(type->tp_dict, &_Py_ID(__annotations__))) { + annotations = PyDict_GetItemWithError( + type->tp_dict, &_Py_ID(__annotations__)); /* - ** _PyDict_GetItemIdWithError could still fail, + ** PyDict_GetItemWithError could still fail, ** for instance with a well-timed Ctrl-C or a MemoryError. ** so let's be totally safe. */ if (annotations) { if (Py_TYPE(annotations)->tp_descr_get) { - annotations = Py_TYPE(annotations)->tp_descr_get(annotations, NULL, - (PyObject *)type); + annotations = Py_TYPE(annotations)->tp_descr_get( + annotations, NULL, (PyObject *)type); } else { Py_INCREF(annotations); } @@ -962,7 +936,8 @@ type_get_annotations(PyTypeObject *type, void *context) } else { annotations = PyDict_New(); if (annotations) { - int result = _PyDict_SetItemId(type->tp_dict, &PyId___annotations__, annotations); + int result = PyDict_SetItem( + type->tp_dict, &_Py_ID(__annotations__), annotations); if (result) { Py_CLEAR(annotations); } else { @@ -986,14 +961,14 @@ type_set_annotations(PyTypeObject *type, PyObject *value, void *context) int result; if (value != NULL) { /* set */ - result = _PyDict_SetItemId(type->tp_dict, &PyId___annotations__, value); + result = PyDict_SetItem(type->tp_dict, &_Py_ID(__annotations__), value); } else { /* delete */ - if (!_PyDict_ContainsId(type->tp_dict, &PyId___annotations__)) { + if (!PyDict_Contains(type->tp_dict, &_Py_ID(__annotations__))) { PyErr_Format(PyExc_AttributeError, "__annotations__"); return -1; } - result = _PyDict_DelItemId(type->tp_dict, &PyId___annotations__); + result = PyDict_DelItem(type->tp_dict, &_Py_ID(__annotations__)); } if (result == 0) { @@ -1074,7 +1049,7 @@ type_repr(PyTypeObject *type) return NULL; } - if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins)) + if (mod != NULL && !_PyUnicode_Equal(mod, &_Py_ID(builtins))) rtn = PyUnicode_FromFormat("", mod, name); else rtn = PyUnicode_FromFormat("", type->tp_name); @@ -1604,11 +1579,11 @@ PyType_IsSubtype(PyTypeObject *a, PyTypeObject *b) */ PyObject * -_PyObject_LookupSpecial(PyObject *self, _Py_Identifier *attrid) +_PyObject_LookupSpecial(PyObject *self, PyObject *attr) { PyObject *res; - res = _PyType_LookupId(Py_TYPE(self), attrid); + res = _PyType_Lookup(Py_TYPE(self), attr); if (res != NULL) { descrgetfunc f; if ((f = Py_TYPE(res)->tp_descr_get) == NULL) @@ -1619,10 +1594,19 @@ _PyObject_LookupSpecial(PyObject *self, _Py_Identifier *attrid) return res; } +PyObject * +_PyObject_LookupSpecialId(PyObject *self, _Py_Identifier *attrid) +{ + PyObject *attr = _PyUnicode_FromId(attrid); /* borrowed */ + if (attr == NULL) + return NULL; + return _PyObject_LookupSpecial(self, attr); +} + static PyObject * -lookup_maybe_method(PyObject *self, _Py_Identifier *attrid, int *unbound) +lookup_maybe_method(PyObject *self, PyObject *attr, int *unbound) { - PyObject *res = _PyType_LookupId(Py_TYPE(self), attrid); + PyObject *res = _PyType_Lookup(Py_TYPE(self), attr); if (res == NULL) { return NULL; } @@ -1646,11 +1630,11 @@ lookup_maybe_method(PyObject *self, _Py_Identifier *attrid, int *unbound) } static PyObject * -lookup_method(PyObject *self, _Py_Identifier *attrid, int *unbound) +lookup_method(PyObject *self, PyObject *attr, int *unbound) { - PyObject *res = lookup_maybe_method(self, attrid, unbound); + PyObject *res = lookup_maybe_method(self, attr, unbound); if (res == NULL && !PyErr_Occurred()) { - PyErr_SetObject(PyExc_AttributeError, _PyUnicode_FromId(attrid)); + PyErr_SetObject(PyExc_AttributeError, attr); } return res; } @@ -1687,8 +1671,7 @@ call_unbound_noarg(int unbound, PyObject *func, PyObject *self) args is an argument vector of length nargs. The first element in this vector is the special object "self" which is used for the method lookup */ static PyObject * -vectorcall_method(_Py_Identifier *name, - PyObject *const *args, Py_ssize_t nargs) +vectorcall_method(PyObject *name, PyObject *const *args, Py_ssize_t nargs) { assert(nargs >= 1); @@ -1707,7 +1690,7 @@ vectorcall_method(_Py_Identifier *name, /* Clone of vectorcall_method() that returns NotImplemented * when the lookup fails. */ static PyObject * -vectorcall_maybe(PyThreadState *tstate, _Py_Identifier *name, +vectorcall_maybe(PyThreadState *tstate, PyObject *name, PyObject *const *args, Py_ssize_t nargs) { assert(nargs >= 1); @@ -1770,7 +1753,7 @@ static PyObject * class_name(PyObject *cls) { PyObject *name; - if (_PyObject_LookupAttrId(cls, &PyId___name__, &name) == 0) { + if (_PyObject_LookupAttr(cls, &_Py_ID(__name__), &name) == 0) { name = PyObject_Repr(cls); } return name; @@ -2093,8 +2076,8 @@ mro_invoke(PyTypeObject *type) if (custom) { int unbound; - PyObject *mro_meth = lookup_method((PyObject *)type, &PyId_mro, - &unbound); + PyObject *mro_meth = lookup_method( + (PyObject *)type, &_Py_ID(mro), &unbound); if (mro_meth == NULL) return NULL; mro_result = call_unbound_noarg(unbound, mro_meth, (PyObject *)type); @@ -2309,7 +2292,7 @@ get_dict_descriptor(PyTypeObject *type) { PyObject *descr; - descr = _PyType_LookupId(type, &PyId___dict__); + descr = _PyType_Lookup(type, &_Py_ID(__dict__)); if (descr == NULL || !PyDescr_IsData(descr)) return NULL; @@ -2558,7 +2541,7 @@ type_new_visit_slots(type_new_ctx *ctx) return -1; } assert(PyUnicode_Check(name)); - if (_PyUnicode_EqualToASCIIId(name, &PyId___dict__)) { + if (_PyUnicode_Equal(name, &_Py_ID(__dict__))) { if (!ctx->may_add_dict || ctx->add_dict != 0) { PyErr_SetString(PyExc_TypeError, "__dict__ slot disallowed: " @@ -2567,7 +2550,7 @@ type_new_visit_slots(type_new_ctx *ctx) } ctx->add_dict++; } - if (_PyUnicode_EqualToASCIIId(name, &PyId___weakref__)) { + if (_PyUnicode_Equal(name, &_Py_ID(__weakref__))) { if (!ctx->may_add_weak || ctx->add_weak != 0) { PyErr_SetString(PyExc_TypeError, "__weakref__ slot disallowed: " @@ -2601,10 +2584,8 @@ type_new_copy_slots(type_new_ctx *ctx, PyObject *dict) Py_ssize_t j = 0; for (Py_ssize_t i = 0; i < nslot; i++) { PyObject *slot = PyTuple_GET_ITEM(slots, i); - if ((ctx->add_dict && - _PyUnicode_EqualToASCIIId(slot, &PyId___dict__)) || - (ctx->add_weak && - _PyUnicode_EqualToASCIIString(slot, "__weakref__"))) + if ((ctx->add_dict && _PyUnicode_Equal(slot, &_Py_ID(__dict__))) || + (ctx->add_weak && _PyUnicode_Equal(slot, &_Py_ID(__weakref__)))) { continue; } @@ -2623,8 +2604,8 @@ type_new_copy_slots(type_new_ctx *ctx, PyObject *dict) /* CPython inserts __qualname__ and __classcell__ (when needed) into the namespace when creating a class. They will be deleted below so won't act as class variables. */ - if (!_PyUnicode_EqualToASCIIId(slot, &PyId___qualname__) && - !_PyUnicode_EqualToASCIIId(slot, &PyId___classcell__)) + if (!_PyUnicode_Equal(slot, &_Py_ID(__qualname__)) && + !_PyUnicode_Equal(slot, &_Py_ID(__classcell__))) { PyErr_Format(PyExc_ValueError, "%R in __slots__ conflicts with class variable", @@ -2819,7 +2800,7 @@ type_new_set_name(const type_new_ctx *ctx, PyTypeObject *type) static int type_new_set_module(PyTypeObject *type) { - int r = _PyDict_ContainsId(type->tp_dict, &PyId___module__); + int r = PyDict_Contains(type->tp_dict, &_Py_ID(__module__)); if (r < 0) { return -1; } @@ -2832,7 +2813,7 @@ type_new_set_module(PyTypeObject *type) return 0; } - PyObject *module = _PyDict_GetItemIdWithError(globals, &PyId___name__); + PyObject *module = PyDict_GetItemWithError(globals, &_Py_ID(__name__)); if (module == NULL) { if (PyErr_Occurred()) { return -1; @@ -2840,7 +2821,7 @@ type_new_set_module(PyTypeObject *type) return 0; } - if (_PyDict_SetItemId(type->tp_dict, &PyId___module__, module) < 0) { + if (PyDict_SetItem(type->tp_dict, &_Py_ID(__module__), module) < 0) { return -1; } return 0; @@ -2853,8 +2834,8 @@ static int type_new_set_ht_name(PyTypeObject *type) { PyHeapTypeObject *et = (PyHeapTypeObject *)type; - PyObject *qualname = _PyDict_GetItemIdWithError(type->tp_dict, - &PyId___qualname__); + PyObject *qualname = PyDict_GetItemWithError( + type->tp_dict, &_Py_ID(__qualname__)); if (qualname != NULL) { if (!PyUnicode_Check(qualname)) { PyErr_Format(PyExc_TypeError, @@ -2863,7 +2844,7 @@ type_new_set_ht_name(PyTypeObject *type) return -1; } et->ht_qualname = Py_NewRef(qualname); - if (_PyDict_DelItemId(type->tp_dict, &PyId___qualname__) < 0) { + if (PyDict_DelItem(type->tp_dict, &_Py_ID(__qualname__)) < 0) { return -1; } } @@ -2883,7 +2864,7 @@ type_new_set_ht_name(PyTypeObject *type) static int type_new_set_doc(PyTypeObject *type) { - PyObject *doc = _PyDict_GetItemIdWithError(type->tp_dict, &PyId___doc__); + PyObject *doc = PyDict_GetItemWithError(type->tp_dict, &_Py_ID(__doc__)); if (doc == NULL) { if (PyErr_Occurred()) { return -1; @@ -2916,9 +2897,9 @@ type_new_set_doc(PyTypeObject *type) static int -type_new_staticmethod(PyTypeObject *type, _Py_Identifier *attr_id) +type_new_staticmethod(PyTypeObject *type, PyObject *attr) { - PyObject *func = _PyDict_GetItemIdWithError(type->tp_dict, attr_id); + PyObject *func = PyDict_GetItemWithError(type->tp_dict, attr); if (func == NULL) { if (PyErr_Occurred()) { return -1; @@ -2933,7 +2914,7 @@ type_new_staticmethod(PyTypeObject *type, _Py_Identifier *attr_id) if (static_func == NULL) { return -1; } - if (_PyDict_SetItemId(type->tp_dict, attr_id, static_func) < 0) { + if (PyDict_SetItem(type->tp_dict, attr, static_func) < 0) { Py_DECREF(static_func); return -1; } @@ -2943,9 +2924,9 @@ type_new_staticmethod(PyTypeObject *type, _Py_Identifier *attr_id) static int -type_new_classmethod(PyTypeObject *type, _Py_Identifier *attr_id) +type_new_classmethod(PyTypeObject *type, PyObject *attr) { - PyObject *func = _PyDict_GetItemIdWithError(type->tp_dict, attr_id); + PyObject *func = PyDict_GetItemWithError(type->tp_dict, attr); if (func == NULL) { if (PyErr_Occurred()) { return -1; @@ -2961,7 +2942,7 @@ type_new_classmethod(PyTypeObject *type, _Py_Identifier *attr_id) return -1; } - if (_PyDict_SetItemId(type->tp_dict, attr_id, method) < 0) { + if (PyDict_SetItem(type->tp_dict, attr, method) < 0) { Py_DECREF(method); return -1; } @@ -3052,8 +3033,8 @@ type_new_set_slots(const type_new_ctx *ctx, PyTypeObject *type) static int type_new_set_classcell(PyTypeObject *type) { - PyObject *cell = _PyDict_GetItemIdWithError(type->tp_dict, - &PyId___classcell__); + PyObject *cell = PyDict_GetItemWithError( + type->tp_dict, &_Py_ID(__classcell__)); if (cell == NULL) { if (PyErr_Occurred()) { return -1; @@ -3070,7 +3051,7 @@ type_new_set_classcell(PyTypeObject *type) } (void)PyCell_Set(cell, (PyObject *) type); - if (_PyDict_DelItemId(type->tp_dict, &PyId___classcell__) < 0) { + if (PyDict_DelItem(type->tp_dict, &_Py_ID(__classcell__)) < 0) { return -1; } return 0; @@ -3098,16 +3079,16 @@ type_new_set_attrs(const type_new_ctx *ctx, PyTypeObject *type) /* Special-case __new__: if it's a plain function, make it a static function */ - if (type_new_staticmethod(type, &PyId___new__) < 0) { + if (type_new_staticmethod(type, &_Py_ID(__new__)) < 0) { return -1; } /* Special-case __init_subclass__ and __class_getitem__: if they are plain functions, make them classmethods */ - if (type_new_classmethod(type, &PyId___init_subclass__) < 0) { + if (type_new_classmethod(type, &_Py_ID(__init_subclass__)) < 0) { return -1; } - if (type_new_classmethod(type, &PyId___class_getitem__) < 0) { + if (type_new_classmethod(type, &_Py_ID(__class_getitem__)) < 0) { return -1; } @@ -3127,8 +3108,7 @@ type_new_set_attrs(const type_new_ctx *ctx, PyTypeObject *type) static int type_new_get_slots(type_new_ctx *ctx, PyObject *dict) { - _Py_IDENTIFIER(__slots__); - PyObject *slots = _PyDict_GetItemIdWithError(dict, &PyId___slots__); + PyObject *slots = PyDict_GetItemWithError(dict, &_Py_ID(__slots__)); if (slots == NULL) { if (PyErr_Occurred()) { return -1; @@ -3251,15 +3231,14 @@ type_new_get_bases(type_new_ctx *ctx, PyObject **type) return 0; } - _Py_IDENTIFIER(__mro_entries__); for (Py_ssize_t i = 0; i < nbases; i++) { PyObject *base = PyTuple_GET_ITEM(ctx->bases, i); if (PyType_Check(base)) { continue; } PyObject *mro_entries; - if (_PyObject_LookupAttrId(base, &PyId___mro_entries__, - &mro_entries) < 0) { + if (_PyObject_LookupAttr(base, &_Py_ID(__mro_entries__), + &mro_entries) < 0) { return -1; } if (mro_entries != NULL) { @@ -3608,7 +3587,7 @@ PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases) PyObject *__doc__ = PyUnicode_FromString(_PyType_DocWithoutSignature(type->tp_name, type->tp_doc)); if (!__doc__) goto fail; - r = _PyDict_SetItemId(type->tp_dict, &PyId___doc__, __doc__); + r = PyDict_SetItem(type->tp_dict, &_Py_ID(__doc__), __doc__); Py_DECREF(__doc__); if (r < 0) goto fail; @@ -3626,7 +3605,7 @@ PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases) } /* Set type.__module__ */ - r = _PyDict_ContainsId(type->tp_dict, &PyId___module__); + r = PyDict_Contains(type->tp_dict, &_Py_ID(__module__)); if (r < 0) { goto fail; } @@ -3638,7 +3617,7 @@ PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases) if (modname == NULL) { goto fail; } - r = _PyDict_SetItemId(type->tp_dict, &PyId___module__, modname); + r = PyDict_SetItem(type->tp_dict, &_Py_ID(__module__), modname); Py_DECREF(modname); if (r != 0) goto fail; @@ -4217,13 +4196,12 @@ merge_class_dict(PyObject *dict, PyObject *aclass) { PyObject *classdict; PyObject *bases; - _Py_IDENTIFIER(__bases__); assert(PyDict_Check(dict)); assert(aclass); /* Merge in the type's dict (if any). */ - if (_PyObject_LookupAttrId(aclass, &PyId___dict__, &classdict) < 0) { + if (_PyObject_LookupAttr(aclass, &_Py_ID(__dict__), &classdict) < 0) { return -1; } if (classdict != NULL) { @@ -4234,7 +4212,7 @@ merge_class_dict(PyObject *dict, PyObject *aclass) } /* Recursively merge in the base types' (if any) dicts. */ - if (_PyObject_LookupAttrId(aclass, &PyId___bases__, &bases) < 0) { + if (_PyObject_LookupAttr(aclass, &_Py_ID(__bases__), &bases) < 0) { return -1; } if (bases != NULL) { @@ -4557,8 +4535,6 @@ object_new(PyTypeObject *type, PyObject *args, PyObject *kwds) PyObject *abstract_methods; PyObject *sorted_methods; PyObject *joined; - PyObject *comma; - _Py_static_string(comma_id, ", "); Py_ssize_t method_count; /* Compute ", ".join(sorted(type.__abstractmethods__)) @@ -4574,12 +4550,7 @@ object_new(PyTypeObject *type, PyObject *args, PyObject *kwds) Py_DECREF(sorted_methods); return NULL; } - comma = _PyUnicode_FromId(&comma_id); - if (comma == NULL) { - Py_DECREF(sorted_methods); - return NULL; - } - joined = PyUnicode_Join(comma, sorted_methods); + joined = PyUnicode_Join(&_Py_STR(comma_sep), sorted_methods); method_count = PyObject_Length(sorted_methods); Py_DECREF(sorted_methods); if (joined == NULL) @@ -4632,7 +4603,7 @@ object_repr(PyObject *self) Py_XDECREF(mod); return NULL; } - if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins)) + if (mod != NULL && !_PyUnicode_Equal(mod, &_Py_ID(builtins))) rtn = PyUnicode_FromFormat("<%U.%U object at %p>", mod, name, self); else rtn = PyUnicode_FromFormat("<%s object at %p>", @@ -4925,27 +4896,19 @@ static PyGetSetDef object_getsets[] = { static PyObject * import_copyreg(void) { - PyObject *copyreg_str; - PyObject *copyreg_module; - _Py_IDENTIFIER(copyreg); - - copyreg_str = _PyUnicode_FromId(&PyId_copyreg); - if (copyreg_str == NULL) { - return NULL; - } /* Try to fetch cached copy of copyreg from sys.modules first in an attempt to avoid the import overhead. Previously this was implemented by storing a reference to the cached module in a static variable, but this broke when multiple embedded interpreters were in use (see issue #17408 and #19088). */ - copyreg_module = PyImport_GetModule(copyreg_str); + PyObject *copyreg_module = PyImport_GetModule(&_Py_ID(copyreg)); if (copyreg_module != NULL) { return copyreg_module; } if (PyErr_Occurred()) { return NULL; } - return PyImport_Import(copyreg_str); + return PyImport_Import(&_Py_ID(copyreg)); } static PyObject * @@ -4953,13 +4916,11 @@ _PyType_GetSlotNames(PyTypeObject *cls) { PyObject *copyreg; PyObject *slotnames; - _Py_IDENTIFIER(__slotnames__); - _Py_IDENTIFIER(_slotnames); assert(PyType_Check(cls)); /* Get the slot names from the cache in the class if possible. */ - slotnames = _PyDict_GetItemIdWithError(cls->tp_dict, &PyId___slotnames__); + slotnames = PyDict_GetItemWithError(cls->tp_dict, &_Py_ID(__slotnames__)); if (slotnames != NULL) { if (slotnames != Py_None && !PyList_Check(slotnames)) { PyErr_Format(PyExc_TypeError, @@ -4985,8 +4946,8 @@ _PyType_GetSlotNames(PyTypeObject *cls) /* Use _slotnames function from the copyreg module to find the slots by this class and its bases. This function will cache the result in __slotnames__. */ - slotnames = _PyObject_CallMethodIdOneArg(copyreg, &PyId__slotnames, - (PyObject *)cls); + slotnames = PyObject_CallMethodOneArg( + copyreg, &_Py_ID(_slotnames), (PyObject *)cls); Py_DECREF(copyreg); if (slotnames == NULL) return NULL; @@ -5006,9 +4967,8 @@ _PyObject_GetState(PyObject *obj, int required) { PyObject *state; PyObject *getstate; - _Py_IDENTIFIER(__getstate__); - if (_PyObject_LookupAttrId(obj, &PyId___getstate__, &getstate) < 0) { + if (_PyObject_LookupAttr(obj, &_Py_ID(__getstate__), &getstate) < 0) { return NULL; } if (getstate == NULL) { @@ -5144,8 +5104,6 @@ static int _PyObject_GetNewArguments(PyObject *obj, PyObject **args, PyObject **kwargs) { PyObject *getnewargs, *getnewargs_ex; - _Py_IDENTIFIER(__getnewargs_ex__); - _Py_IDENTIFIER(__getnewargs__); if (args == NULL || kwargs == NULL) { PyErr_BadInternalCall(); @@ -5154,7 +5112,7 @@ _PyObject_GetNewArguments(PyObject *obj, PyObject **args, PyObject **kwargs) /* We first attempt to fetch the arguments for __new__ by calling __getnewargs_ex__ on the object. */ - getnewargs_ex = _PyObject_LookupSpecial(obj, &PyId___getnewargs_ex__); + getnewargs_ex = _PyObject_LookupSpecial(obj, &_Py_ID(__getnewargs_ex__)); if (getnewargs_ex != NULL) { PyObject *newargs = _PyObject_CallNoArgs(getnewargs_ex); Py_DECREF(getnewargs_ex); @@ -5207,7 +5165,7 @@ _PyObject_GetNewArguments(PyObject *obj, PyObject **args, PyObject **kwargs) /* The object does not have __getnewargs_ex__ so we fallback on using __getnewargs__ instead. */ - getnewargs = _PyObject_LookupSpecial(obj, &PyId___getnewargs__); + getnewargs = _PyObject_LookupSpecial(obj, &_Py_ID(__getnewargs__)); if (getnewargs != NULL) { *args = _PyObject_CallNoArgs(getnewargs); Py_DECREF(getnewargs); @@ -5260,10 +5218,7 @@ _PyObject_GetItemsIter(PyObject *obj, PyObject **listitems, Py_INCREF(*dictitems); } else { - PyObject *items; - _Py_IDENTIFIER(items); - - items = _PyObject_CallMethodIdNoArgs(obj, &PyId_items); + PyObject *items = PyObject_CallMethodNoArgs(obj, &_Py_ID(items)); if (items == NULL) { Py_CLEAR(*listitems); return -1; @@ -5307,12 +5262,11 @@ reduce_newobj(PyObject *obj) } hasargs = (args != NULL); if (kwargs == NULL || PyDict_GET_SIZE(kwargs) == 0) { - _Py_IDENTIFIER(__newobj__); PyObject *cls; Py_ssize_t i, n; Py_XDECREF(kwargs); - newobj = _PyObject_GetAttrId(copyreg, &PyId___newobj__); + newobj = PyObject_GetAttr(copyreg, &_Py_ID(__newobj__)); Py_DECREF(copyreg); if (newobj == NULL) { Py_XDECREF(args); @@ -5336,9 +5290,7 @@ reduce_newobj(PyObject *obj) Py_XDECREF(args); } else if (args != NULL) { - _Py_IDENTIFIER(__newobj_ex__); - - newobj = _PyObject_GetAttrId(copyreg, &PyId___newobj_ex__); + newobj = PyObject_GetAttr(copyreg, &_Py_ID(__newobj_ex__)); Py_DECREF(copyreg); if (newobj == NULL) { Py_DECREF(args); @@ -5444,17 +5396,16 @@ object___reduce_ex___impl(PyObject *self, int protocol) { static PyObject *objreduce; PyObject *reduce, *res; - _Py_IDENTIFIER(__reduce__); if (objreduce == NULL) { - objreduce = _PyDict_GetItemIdWithError(PyBaseObject_Type.tp_dict, - &PyId___reduce__); + objreduce = PyDict_GetItemWithError( + PyBaseObject_Type.tp_dict, &_Py_ID(__reduce__)); if (objreduce == NULL && PyErr_Occurred()) { return NULL; } } - if (_PyObject_LookupAttrId(self, &PyId___reduce__, &reduce) < 0) { + if (_PyObject_LookupAttr(self, &_Py_ID(__reduce__), &reduce) < 0) { return NULL; } if (reduce != NULL) { @@ -5462,7 +5413,7 @@ object___reduce_ex___impl(PyObject *self, int protocol) int override; cls = (PyObject *) Py_TYPE(self); - clsreduce = _PyObject_GetAttrId(cls, &PyId___reduce__); + clsreduce = PyObject_GetAttr(cls, &_Py_ID(__reduce__)); if (clsreduce == NULL) { Py_DECREF(reduce); return NULL; @@ -5570,7 +5521,7 @@ object___dir___impl(PyObject *self) PyObject *itsclass = NULL; /* Get __dict__ (which may or may not be a real dict...) */ - if (_PyObject_LookupAttrId(self, &PyId___dict__, &dict) < 0) { + if (_PyObject_LookupAttr(self, &_Py_ID(__dict__), &dict) < 0) { return NULL; } if (dict == NULL) { @@ -5591,7 +5542,7 @@ object___dir___impl(PyObject *self) goto error; /* Merge in attrs reachable from its class. */ - if (_PyObject_LookupAttrId(self, &PyId___class__, &itsclass) < 0) { + if (_PyObject_LookupAttr(self, &_Py_ID(__class__), &itsclass) < 0) { goto error; } /* XXX(tomer): Perhaps fall back to Py_TYPE(obj) if no @@ -5857,12 +5808,11 @@ static int overrides_hash(PyTypeObject *type) { PyObject *dict = type->tp_dict; - _Py_IDENTIFIER(__eq__); assert(dict != NULL); - int r = _PyDict_ContainsId(dict, &PyId___eq__); + int r = PyDict_Contains(dict, &_Py_ID(__eq__)); if (r == 0) { - r = _PyDict_ContainsId(dict, &PyId___hash__); + r = PyDict_Contains(dict, &_Py_ID(__hash__)); } return r; } @@ -6177,7 +6127,7 @@ type_ready_set_dict(PyTypeObject *type) static int type_dict_set_doc(PyTypeObject *type) { - int r = _PyDict_ContainsId(type->tp_dict, &PyId___doc__); + int r = PyDict_Contains(type->tp_dict, &_Py_ID(__doc__)); if (r < 0) { return -1; } @@ -6193,14 +6143,14 @@ type_dict_set_doc(PyTypeObject *type) return -1; } - if (_PyDict_SetItemId(type->tp_dict, &PyId___doc__, doc) < 0) { + if (PyDict_SetItem(type->tp_dict, &_Py_ID(__doc__), doc) < 0) { Py_DECREF(doc); return -1; } Py_DECREF(doc); } else { - if (_PyDict_SetItemId(type->tp_dict, &PyId___doc__, Py_None) < 0) { + if (PyDict_SetItem(type->tp_dict, &_Py_ID(__doc__), Py_None) < 0) { return -1; } } @@ -6348,7 +6298,7 @@ type_ready_set_hash(PyTypeObject *type) return 0; } - int r = _PyDict_ContainsId(type->tp_dict, &PyId___hash__); + int r = PyDict_Contains(type->tp_dict, &_Py_ID(__hash__)); if (r < 0) { return -1; } @@ -6356,7 +6306,7 @@ type_ready_set_hash(PyTypeObject *type) return 0; } - if (_PyDict_SetItemId(type->tp_dict, &PyId___hash__, Py_None) < 0) { + if (PyDict_SetItem(type->tp_dict, &_Py_ID(__hash__), Py_None) < 0) { return -1; } type->tp_hash = PyObject_HashNotImplemented; @@ -7173,7 +7123,7 @@ static struct PyMethodDef tp_new_methoddef[] = { static int add_tp_new_wrapper(PyTypeObject *type) { - int r = _PyDict_ContainsId(type->tp_dict, &PyId___new__); + int r = PyDict_Contains(type->tp_dict, &_Py_ID(__new__)); if (r > 0) { return 0; } @@ -7185,7 +7135,7 @@ add_tp_new_wrapper(PyTypeObject *type) if (func == NULL) { return -1; } - r = _PyDict_SetItemId(type->tp_dict, &PyId___new__, func); + r = PyDict_SetItem(type->tp_dict, &_Py_ID(__new__), func); Py_DECREF(func); return r; } @@ -7193,33 +7143,31 @@ add_tp_new_wrapper(PyTypeObject *type) /* Slot wrappers that call the corresponding __foo__ slot. See comments below at override_slots() for more explanation. */ -#define SLOT0(FUNCNAME, OPSTR) \ +#define SLOT0(FUNCNAME, DUNDER) \ static PyObject * \ FUNCNAME(PyObject *self) \ { \ PyObject* stack[1] = {self}; \ - _Py_static_string(id, OPSTR); \ - return vectorcall_method(&id, stack, 1); \ + return vectorcall_method(&_Py_ID(DUNDER), stack, 1); \ } -#define SLOT1(FUNCNAME, OPSTR, ARG1TYPE) \ +#define SLOT1(FUNCNAME, DUNDER, ARG1TYPE) \ static PyObject * \ FUNCNAME(PyObject *self, ARG1TYPE arg1) \ { \ PyObject* stack[2] = {self, arg1}; \ - _Py_static_string(id, OPSTR); \ - return vectorcall_method(&id, stack, 2); \ + return vectorcall_method(&_Py_ID(DUNDER), stack, 2); \ } /* Boolean helper for SLOT1BINFULL(). right.__class__ is a nontrivial subclass of left.__class__. */ static int -method_is_overloaded(PyObject *left, PyObject *right, struct _Py_Identifier *name) +method_is_overloaded(PyObject *left, PyObject *right, PyObject *name) { PyObject *a, *b; int ok; - if (_PyObject_LookupAttrId((PyObject *)(Py_TYPE(right)), name, &b) < 0) { + if (_PyObject_LookupAttr((PyObject *)(Py_TYPE(right)), name, &b) < 0) { return -1; } if (b == NULL) { @@ -7227,7 +7175,7 @@ method_is_overloaded(PyObject *left, PyObject *right, struct _Py_Identifier *nam return 0; } - if (_PyObject_LookupAttrId((PyObject *)(Py_TYPE(left)), name, &a) < 0) { + if (_PyObject_LookupAttr((PyObject *)(Py_TYPE(left)), name, &a) < 0) { Py_DECREF(b); return -1; } @@ -7244,14 +7192,12 @@ method_is_overloaded(PyObject *left, PyObject *right, struct _Py_Identifier *nam } -#define SLOT1BINFULL(FUNCNAME, TESTFUNC, SLOTNAME, OPSTR, ROPSTR) \ +#define SLOT1BINFULL(FUNCNAME, TESTFUNC, SLOTNAME, DUNDER, RDUNDER) \ static PyObject * \ FUNCNAME(PyObject *self, PyObject *other) \ { \ PyObject* stack[2]; \ PyThreadState *tstate = _PyThreadState_GET(); \ - _Py_static_string(op_id, OPSTR); \ - _Py_static_string(rop_id, ROPSTR); \ int do_other = !Py_IS_TYPE(self, Py_TYPE(other)) && \ Py_TYPE(other)->tp_as_number != NULL && \ Py_TYPE(other)->tp_as_number->SLOTNAME == TESTFUNC; \ @@ -7259,14 +7205,14 @@ FUNCNAME(PyObject *self, PyObject *other) \ Py_TYPE(self)->tp_as_number->SLOTNAME == TESTFUNC) { \ PyObject *r; \ if (do_other && PyType_IsSubtype(Py_TYPE(other), Py_TYPE(self))) { \ - int ok = method_is_overloaded(self, other, &rop_id); \ + int ok = method_is_overloaded(self, other, &_Py_ID(RDUNDER)); \ if (ok < 0) { \ return NULL; \ } \ if (ok) { \ stack[0] = other; \ stack[1] = self; \ - r = vectorcall_maybe(tstate, &rop_id, stack, 2); \ + r = vectorcall_maybe(tstate, &_Py_ID(RDUNDER), stack, 2); \ if (r != Py_NotImplemented) \ return r; \ Py_DECREF(r); \ @@ -7275,7 +7221,7 @@ FUNCNAME(PyObject *self, PyObject *other) \ } \ stack[0] = self; \ stack[1] = other; \ - r = vectorcall_maybe(tstate, &op_id, stack, 2); \ + r = vectorcall_maybe(tstate, &_Py_ID(DUNDER), stack, 2); \ if (r != Py_NotImplemented || \ Py_IS_TYPE(other, Py_TYPE(self))) \ return r; \ @@ -7284,19 +7230,19 @@ FUNCNAME(PyObject *self, PyObject *other) \ if (do_other) { \ stack[0] = other; \ stack[1] = self; \ - return vectorcall_maybe(tstate, &rop_id, stack, 2); \ + return vectorcall_maybe(tstate, &_Py_ID(RDUNDER), stack, 2); \ } \ Py_RETURN_NOTIMPLEMENTED; \ } -#define SLOT1BIN(FUNCNAME, SLOTNAME, OPSTR, ROPSTR) \ - SLOT1BINFULL(FUNCNAME, FUNCNAME, SLOTNAME, OPSTR, ROPSTR) +#define SLOT1BIN(FUNCNAME, SLOTNAME, DUNDER, RDUNDER) \ + SLOT1BINFULL(FUNCNAME, FUNCNAME, SLOTNAME, DUNDER, RDUNDER) static Py_ssize_t slot_sq_length(PyObject *self) { PyObject* stack[1] = {self}; - PyObject *res = vectorcall_method(&PyId___len__, stack, 1); + PyObject *res = vectorcall_method(&_Py_ID(__len__), stack, 1); Py_ssize_t len; if (res == NULL) @@ -7328,7 +7274,7 @@ slot_sq_item(PyObject *self, Py_ssize_t i) return NULL; } PyObject *stack[2] = {self, ival}; - PyObject *retval = vectorcall_method(&PyId___getitem__, stack, 2); + PyObject *retval = vectorcall_method(&_Py_ID(__getitem__), stack, 2); Py_DECREF(ival); return retval; } @@ -7348,11 +7294,11 @@ slot_sq_ass_item(PyObject *self, Py_ssize_t index, PyObject *value) stack[0] = self; stack[1] = index_obj; if (value == NULL) { - res = vectorcall_method(&PyId___delitem__, stack, 2); + res = vectorcall_method(&_Py_ID(__delitem__), stack, 2); } else { stack[2] = value; - res = vectorcall_method(&PyId___setitem__, stack, 3); + res = vectorcall_method(&_Py_ID(__setitem__), stack, 3); } Py_DECREF(index_obj); @@ -7369,9 +7315,8 @@ slot_sq_contains(PyObject *self, PyObject *value) PyThreadState *tstate = _PyThreadState_GET(); PyObject *func, *res; int result = -1, unbound; - _Py_IDENTIFIER(__contains__); - func = lookup_maybe_method(self, &PyId___contains__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__contains__), &unbound); if (func == Py_None) { Py_DECREF(func); PyErr_Format(PyExc_TypeError, @@ -7398,7 +7343,7 @@ slot_sq_contains(PyObject *self, PyObject *value) #define slot_mp_length slot_sq_length -SLOT1(slot_mp_subscript, "__getitem__", PyObject *) +SLOT1(slot_mp_subscript, __getitem__, PyObject *) static int slot_mp_ass_subscript(PyObject *self, PyObject *key, PyObject *value) @@ -7409,11 +7354,11 @@ slot_mp_ass_subscript(PyObject *self, PyObject *key, PyObject *value) stack[0] = self; stack[1] = key; if (value == NULL) { - res = vectorcall_method(&PyId___delitem__, stack, 2); + res = vectorcall_method(&_Py_ID(__delitem__), stack, 2); } else { stack[2] = value; - res = vectorcall_method(&PyId___setitem__, stack, 3); + res = vectorcall_method(&_Py_ID(__setitem__), stack, 3); } if (res == NULL) @@ -7422,23 +7367,20 @@ slot_mp_ass_subscript(PyObject *self, PyObject *key, PyObject *value) return 0; } -SLOT1BIN(slot_nb_add, nb_add, "__add__", "__radd__") -SLOT1BIN(slot_nb_subtract, nb_subtract, "__sub__", "__rsub__") -SLOT1BIN(slot_nb_multiply, nb_multiply, "__mul__", "__rmul__") -SLOT1BIN(slot_nb_matrix_multiply, nb_matrix_multiply, "__matmul__", "__rmatmul__") -SLOT1BIN(slot_nb_remainder, nb_remainder, "__mod__", "__rmod__") -SLOT1BIN(slot_nb_divmod, nb_divmod, "__divmod__", "__rdivmod__") +SLOT1BIN(slot_nb_add, nb_add, __add__, __radd__) +SLOT1BIN(slot_nb_subtract, nb_subtract, __sub__, __rsub__) +SLOT1BIN(slot_nb_multiply, nb_multiply, __mul__, __rmul__) +SLOT1BIN(slot_nb_matrix_multiply, nb_matrix_multiply, __matmul__, __rmatmul__) +SLOT1BIN(slot_nb_remainder, nb_remainder, __mod__, __rmod__) +SLOT1BIN(slot_nb_divmod, nb_divmod, __divmod__, __rdivmod__) static PyObject *slot_nb_power(PyObject *, PyObject *, PyObject *); -SLOT1BINFULL(slot_nb_power_binary, slot_nb_power, - nb_power, "__pow__", "__rpow__") +SLOT1BINFULL(slot_nb_power_binary, slot_nb_power, nb_power, __pow__, __rpow__) static PyObject * slot_nb_power(PyObject *self, PyObject *other, PyObject *modulus) { - _Py_IDENTIFIER(__pow__); - if (modulus == Py_None) return slot_nb_power_binary(self, other); /* Three-arg power doesn't use __rpow__. But ternary_op @@ -7447,14 +7389,14 @@ slot_nb_power(PyObject *self, PyObject *other, PyObject *modulus) if (Py_TYPE(self)->tp_as_number != NULL && Py_TYPE(self)->tp_as_number->nb_power == slot_nb_power) { PyObject* stack[3] = {self, other, modulus}; - return vectorcall_method(&PyId___pow__, stack, 3); + return vectorcall_method(&_Py_ID(__pow__), stack, 3); } Py_RETURN_NOTIMPLEMENTED; } -SLOT0(slot_nb_negative, "__neg__") -SLOT0(slot_nb_positive, "__pos__") -SLOT0(slot_nb_absolute, "__abs__") +SLOT0(slot_nb_negative, __neg__) +SLOT0(slot_nb_positive, __pos__) +SLOT0(slot_nb_absolute, __abs__) static int slot_nb_bool(PyObject *self) @@ -7462,15 +7404,14 @@ slot_nb_bool(PyObject *self) PyObject *func, *value; int result, unbound; int using_len = 0; - _Py_IDENTIFIER(__bool__); - func = lookup_maybe_method(self, &PyId___bool__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__bool__), &unbound); if (func == NULL) { if (PyErr_Occurred()) { return -1; } - func = lookup_maybe_method(self, &PyId___len__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__len__), &unbound); if (func == NULL) { if (PyErr_Occurred()) { return -1; @@ -7513,53 +7454,50 @@ slot_nb_bool(PyObject *self) static PyObject * slot_nb_index(PyObject *self) { - _Py_IDENTIFIER(__index__); PyObject *stack[1] = {self}; - return vectorcall_method(&PyId___index__, stack, 1); + return vectorcall_method(&_Py_ID(__index__), stack, 1); } -SLOT0(slot_nb_invert, "__invert__") -SLOT1BIN(slot_nb_lshift, nb_lshift, "__lshift__", "__rlshift__") -SLOT1BIN(slot_nb_rshift, nb_rshift, "__rshift__", "__rrshift__") -SLOT1BIN(slot_nb_and, nb_and, "__and__", "__rand__") -SLOT1BIN(slot_nb_xor, nb_xor, "__xor__", "__rxor__") -SLOT1BIN(slot_nb_or, nb_or, "__or__", "__ror__") +SLOT0(slot_nb_invert, __invert__) +SLOT1BIN(slot_nb_lshift, nb_lshift, __lshift__, __rlshift__) +SLOT1BIN(slot_nb_rshift, nb_rshift, __rshift__, __rrshift__) +SLOT1BIN(slot_nb_and, nb_and, __and__, __rand__) +SLOT1BIN(slot_nb_xor, nb_xor, __xor__, __rxor__) +SLOT1BIN(slot_nb_or, nb_or, __or__, __ror__) -SLOT0(slot_nb_int, "__int__") -SLOT0(slot_nb_float, "__float__") -SLOT1(slot_nb_inplace_add, "__iadd__", PyObject *) -SLOT1(slot_nb_inplace_subtract, "__isub__", PyObject *) -SLOT1(slot_nb_inplace_multiply, "__imul__", PyObject *) -SLOT1(slot_nb_inplace_matrix_multiply, "__imatmul__", PyObject *) -SLOT1(slot_nb_inplace_remainder, "__imod__", PyObject *) +SLOT0(slot_nb_int, __int__) +SLOT0(slot_nb_float, __float__) +SLOT1(slot_nb_inplace_add, __iadd__, PyObject *) +SLOT1(slot_nb_inplace_subtract, __isub__, PyObject *) +SLOT1(slot_nb_inplace_multiply, __imul__, PyObject *) +SLOT1(slot_nb_inplace_matrix_multiply, __imatmul__, PyObject *) +SLOT1(slot_nb_inplace_remainder, __imod__, PyObject *) /* Can't use SLOT1 here, because nb_inplace_power is ternary */ static PyObject * slot_nb_inplace_power(PyObject *self, PyObject * arg1, PyObject *arg2) { PyObject *stack[2] = {self, arg1}; - _Py_IDENTIFIER(__ipow__); - return vectorcall_method(&PyId___ipow__, stack, 2); -} -SLOT1(slot_nb_inplace_lshift, "__ilshift__", PyObject *) -SLOT1(slot_nb_inplace_rshift, "__irshift__", PyObject *) -SLOT1(slot_nb_inplace_and, "__iand__", PyObject *) -SLOT1(slot_nb_inplace_xor, "__ixor__", PyObject *) -SLOT1(slot_nb_inplace_or, "__ior__", PyObject *) + return vectorcall_method(&_Py_ID(__ipow__), stack, 2); +} +SLOT1(slot_nb_inplace_lshift, __ilshift__, PyObject *) +SLOT1(slot_nb_inplace_rshift, __irshift__, PyObject *) +SLOT1(slot_nb_inplace_and, __iand__, PyObject *) +SLOT1(slot_nb_inplace_xor, __ixor__, PyObject *) +SLOT1(slot_nb_inplace_or, __ior__, PyObject *) SLOT1BIN(slot_nb_floor_divide, nb_floor_divide, - "__floordiv__", "__rfloordiv__") -SLOT1BIN(slot_nb_true_divide, nb_true_divide, "__truediv__", "__rtruediv__") -SLOT1(slot_nb_inplace_floor_divide, "__ifloordiv__", PyObject *) -SLOT1(slot_nb_inplace_true_divide, "__itruediv__", PyObject *) + __floordiv__, __rfloordiv__) +SLOT1BIN(slot_nb_true_divide, nb_true_divide, __truediv__, __rtruediv__) +SLOT1(slot_nb_inplace_floor_divide, __ifloordiv__, PyObject *) +SLOT1(slot_nb_inplace_true_divide, __itruediv__, PyObject *) static PyObject * slot_tp_repr(PyObject *self) { PyObject *func, *res; - _Py_IDENTIFIER(__repr__); int unbound; - func = lookup_maybe_method(self, &PyId___repr__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__repr__), &unbound); if (func != NULL) { res = call_unbound_noarg(unbound, func, self); Py_DECREF(func); @@ -7570,7 +7508,7 @@ slot_tp_repr(PyObject *self) Py_TYPE(self)->tp_name, self); } -SLOT0(slot_tp_str, "__str__") +SLOT0(slot_tp_str, __str__) static Py_hash_t slot_tp_hash(PyObject *self) @@ -7579,7 +7517,7 @@ slot_tp_hash(PyObject *self) Py_ssize_t h; int unbound; - func = lookup_maybe_method(self, &PyId___hash__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__hash__), &unbound); if (func == Py_None) { Py_DECREF(func); @@ -7624,10 +7562,9 @@ static PyObject * slot_tp_call(PyObject *self, PyObject *args, PyObject *kwds) { PyThreadState *tstate = _PyThreadState_GET(); - _Py_IDENTIFIER(__call__); int unbound; - PyObject *meth = lookup_method(self, &PyId___call__, &unbound); + PyObject *meth = lookup_method(self, &_Py_ID(__call__), &unbound); if (meth == NULL) { return NULL; } @@ -7659,7 +7596,7 @@ static PyObject * slot_tp_getattro(PyObject *self, PyObject *name) { PyObject *stack[2] = {self, name}; - return vectorcall_method(&PyId___getattribute__, stack, 2); + return vectorcall_method(&_Py_ID(__getattribute__), stack, 2); } static PyObject * @@ -7685,14 +7622,13 @@ slot_tp_getattr_hook(PyObject *self, PyObject *name) { PyTypeObject *tp = Py_TYPE(self); PyObject *getattr, *getattribute, *res; - _Py_IDENTIFIER(__getattr__); /* speed hack: we could use lookup_maybe, but that would resolve the method fully for each attribute lookup for classes with __getattr__, even when the attribute is present. So we use _PyType_Lookup and create the method only when needed, with call_attribute. */ - getattr = _PyType_LookupId(tp, &PyId___getattr__); + getattr = _PyType_Lookup(tp, &_Py_ID(__getattr__)); if (getattr == NULL) { /* No __getattr__ hook: use a simpler dispatcher */ tp->tp_getattro = slot_tp_getattro; @@ -7704,7 +7640,7 @@ slot_tp_getattr_hook(PyObject *self, PyObject *name) __getattr__, even when self has the default __getattribute__ method. So we use _PyType_Lookup and create the method only when needed, with call_attribute. */ - getattribute = _PyType_LookupId(tp, &PyId___getattribute__); + getattribute = _PyType_Lookup(tp, &_Py_ID(__getattribute__)); if (getattribute == NULL || (Py_IS_TYPE(getattribute, &PyWrapperDescr_Type) && ((PyWrapperDescrObject *)getattribute)->d_wrapped == @@ -7728,17 +7664,15 @@ slot_tp_setattro(PyObject *self, PyObject *name, PyObject *value) { PyObject *stack[3]; PyObject *res; - _Py_IDENTIFIER(__delattr__); - _Py_IDENTIFIER(__setattr__); stack[0] = self; stack[1] = name; if (value == NULL) { - res = vectorcall_method(&PyId___delattr__, stack, 2); + res = vectorcall_method(&_Py_ID(__delattr__), stack, 2); } else { stack[2] = value; - res = vectorcall_method(&PyId___setattr__, stack, 3); + res = vectorcall_method(&_Py_ID(__setattr__), stack, 3); } if (res == NULL) return -1; @@ -7746,13 +7680,13 @@ slot_tp_setattro(PyObject *self, PyObject *name, PyObject *value) return 0; } -static _Py_Identifier name_op[] = { - _Py_static_string_init("__lt__"), - _Py_static_string_init("__le__"), - _Py_static_string_init("__eq__"), - _Py_static_string_init("__ne__"), - _Py_static_string_init("__gt__"), - _Py_static_string_init("__ge__"), +static PyObject *name_op[] = { + &_Py_ID(__lt__), + &_Py_ID(__le__), + &_Py_ID(__eq__), + &_Py_ID(__ne__), + &_Py_ID(__gt__), + &_Py_ID(__ge__), }; static PyObject * @@ -7761,7 +7695,7 @@ slot_tp_richcompare(PyObject *self, PyObject *other, int op) PyThreadState *tstate = _PyThreadState_GET(); int unbound; - PyObject *func = lookup_maybe_method(self, &name_op[op], &unbound); + PyObject *func = lookup_maybe_method(self, name_op[op], &unbound); if (func == NULL) { PyErr_Clear(); Py_RETURN_NOTIMPLEMENTED; @@ -7778,9 +7712,8 @@ slot_tp_iter(PyObject *self) { int unbound; PyObject *func, *res; - _Py_IDENTIFIER(__iter__); - func = lookup_maybe_method(self, &PyId___iter__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__iter__), &unbound); if (func == Py_None) { Py_DECREF(func); PyErr_Format(PyExc_TypeError, @@ -7796,7 +7729,7 @@ slot_tp_iter(PyObject *self) } PyErr_Clear(); - func = lookup_maybe_method(self, &PyId___getitem__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__getitem__), &unbound); if (func == NULL) { PyErr_Format(PyExc_TypeError, "'%.200s' object is not iterable", @@ -7810,9 +7743,8 @@ slot_tp_iter(PyObject *self) static PyObject * slot_tp_iternext(PyObject *self) { - _Py_IDENTIFIER(__next__); PyObject *stack[1] = {self}; - return vectorcall_method(&PyId___next__, stack, 1); + return vectorcall_method(&_Py_ID(__next__), stack, 1); } static PyObject * @@ -7820,9 +7752,8 @@ slot_tp_descr_get(PyObject *self, PyObject *obj, PyObject *type) { PyTypeObject *tp = Py_TYPE(self); PyObject *get; - _Py_IDENTIFIER(__get__); - get = _PyType_LookupId(tp, &PyId___get__); + get = _PyType_Lookup(tp, &_Py_ID(__get__)); if (get == NULL) { /* Avoid further slowdowns */ if (tp->tp_descr_get == slot_tp_descr_get) @@ -7842,17 +7773,15 @@ slot_tp_descr_set(PyObject *self, PyObject *target, PyObject *value) { PyObject* stack[3]; PyObject *res; - _Py_IDENTIFIER(__delete__); - _Py_IDENTIFIER(__set__); stack[0] = self; stack[1] = target; if (value == NULL) { - res = vectorcall_method(&PyId___delete__, stack, 2); + res = vectorcall_method(&_Py_ID(__delete__), stack, 2); } else { stack[2] = value; - res = vectorcall_method(&PyId___set__, stack, 3); + res = vectorcall_method(&_Py_ID(__set__), stack, 3); } if (res == NULL) return -1; @@ -7865,9 +7794,8 @@ slot_tp_init(PyObject *self, PyObject *args, PyObject *kwds) { PyThreadState *tstate = _PyThreadState_GET(); - _Py_IDENTIFIER(__init__); int unbound; - PyObject *meth = lookup_method(self, &PyId___init__, &unbound); + PyObject *meth = lookup_method(self, &_Py_ID(__init__), &unbound); if (meth == NULL) { return -1; } @@ -7899,7 +7827,7 @@ slot_tp_new(PyTypeObject *type, PyObject *args, PyObject *kwds) PyThreadState *tstate = _PyThreadState_GET(); PyObject *func, *result; - func = _PyObject_GetAttrId((PyObject *)type, &PyId___new__); + func = PyObject_GetAttr((PyObject *)type, &_Py_ID(__new__)); if (func == NULL) { return NULL; } @@ -7912,7 +7840,6 @@ slot_tp_new(PyTypeObject *type, PyObject *args, PyObject *kwds) static void slot_tp_finalize(PyObject *self) { - _Py_IDENTIFIER(__del__); int unbound; PyObject *del, *res; PyObject *error_type, *error_value, *error_traceback; @@ -7921,7 +7848,7 @@ slot_tp_finalize(PyObject *self) PyErr_Fetch(&error_type, &error_value, &error_traceback); /* Execute __del__ method, if any. */ - del = lookup_maybe_method(self, &PyId___del__, &unbound); + del = lookup_maybe_method(self, &_Py_ID(__del__), &unbound); if (del != NULL) { res = call_unbound_noarg(unbound, del, self); if (res == NULL) @@ -7940,9 +7867,8 @@ slot_am_await(PyObject *self) { int unbound; PyObject *func, *res; - _Py_IDENTIFIER(__await__); - func = lookup_maybe_method(self, &PyId___await__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__await__), &unbound); if (func != NULL) { res = call_unbound_noarg(unbound, func, self); Py_DECREF(func); @@ -7959,9 +7885,8 @@ slot_am_aiter(PyObject *self) { int unbound; PyObject *func, *res; - _Py_IDENTIFIER(__aiter__); - func = lookup_maybe_method(self, &PyId___aiter__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__aiter__), &unbound); if (func != NULL) { res = call_unbound_noarg(unbound, func, self); Py_DECREF(func); @@ -7978,9 +7903,8 @@ slot_am_anext(PyObject *self) { int unbound; PyObject *func, *res; - _Py_IDENTIFIER(__anext__); - func = lookup_maybe_method(self, &PyId___anext__, &unbound); + func = lookup_maybe_method(self, &_Py_ID(__anext__), &unbound); if (func != NULL) { res = call_unbound_noarg(unbound, func, self); Py_DECREF(func); @@ -8612,7 +8536,8 @@ type_new_set_names(PyTypeObject *type) Py_ssize_t i = 0; PyObject *key, *value; while (PyDict_Next(names_to_set, &i, &key, &value)) { - PyObject *set_name = _PyObject_LookupSpecial(value, &PyId___set_name__); + PyObject *set_name = _PyObject_LookupSpecial(value, + &_Py_ID(__set_name__)); if (set_name == NULL) { if (PyErr_Occurred()) { goto error; @@ -8652,7 +8577,7 @@ type_new_init_subclass(PyTypeObject *type, PyObject *kwds) return -1; } - PyObject *func = _PyObject_GetAttrId(super, &PyId___init_subclass__); + PyObject *func = PyObject_GetAttr(super, &_Py_ID(__init_subclass__)); Py_DECREF(super); if (func == NULL) { return -1; @@ -8862,7 +8787,7 @@ super_getattro(PyObject *self, PyObject *name) (i.e. super, or a subclass), not the class of su->obj. */ if (PyUnicode_Check(name) && PyUnicode_GET_LENGTH(name) == 9 && - _PyUnicode_EqualToASCIIId(name, &PyId___class__)) + _PyUnicode_Equal(name, &_Py_ID(__class__))) goto skip; mro = starttype->tp_mro; @@ -8954,7 +8879,7 @@ supercheck(PyTypeObject *type, PyObject *obj) /* Try the slow way */ PyObject *class_attr; - if (_PyObject_LookupAttrId(obj, &PyId___class__, &class_attr) < 0) { + if (_PyObject_LookupAttr(obj, &_Py_ID(__class__), &class_attr) < 0) { return NULL; } if (class_attr != NULL && @@ -9045,7 +8970,7 @@ super_init_without_args(InterpreterFrame *cframe, PyCodeObject *co, assert((_PyLocals_GetKind(co->co_localspluskinds, i) & CO_FAST_FREE) != 0); PyObject *name = PyTuple_GET_ITEM(co->co_localsplusnames, i); assert(PyUnicode_Check(name)); - if (_PyUnicode_EqualToASCIIId(name, &PyId___class__)) { + if (_PyUnicode_Equal(name, &_Py_ID(__class__))) { PyObject *cell = _PyFrame_GetLocalsArray(cframe)[i]; if (cell == NULL || !PyCell_Check(cell)) { PyErr_SetString(PyExc_RuntimeError, diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 888871a90c875..908ad51492599 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -260,11 +260,7 @@ get_unicode_state(void) // Return a borrowed reference to the empty string singleton. static inline PyObject* unicode_get_empty(void) { - struct _Py_unicode_state *state = get_unicode_state(); - // unicode_get_empty() must not be called before _PyUnicode_Init() - // or after _PyUnicode_Fini() - assert(state->empty_string != NULL); - return state->empty_string; + return &_Py_STR(empty); } @@ -1388,25 +1384,6 @@ _PyUnicode_Dump(PyObject *op) } #endif -static int -unicode_create_empty_string_singleton(struct _Py_unicode_state *state) -{ - // Use size=1 rather than size=0, so PyUnicode_New(0, maxchar) can be - // optimized to always use state->empty_string without having to check if - // it is NULL or not. - PyObject *empty = PyUnicode_New(1, 0); - if (empty == NULL) { - return -1; - } - PyUnicode_1BYTE_DATA(empty)[0] = 0; - _PyUnicode_LENGTH(empty) = 0; - assert(_PyUnicode_CheckConsistency(empty, 1)); - - assert(state->empty_string == NULL); - state->empty_string = empty; - return 0; -} - PyObject * PyUnicode_New(Py_ssize_t size, Py_UCS4 maxchar) @@ -2009,10 +1986,11 @@ unicode_dealloc(PyObject *unicode) static int unicode_is_singleton(PyObject *unicode) { - struct _Py_unicode_state *state = get_unicode_state(); - if (unicode == state->empty_string) { + if (unicode == &_Py_STR(empty)) { return 1; } + + struct _Py_unicode_state *state = get_unicode_state(); PyASCIIObject *ascii = (PyASCIIObject *)unicode; if (ascii->state.kind != PyUnicode_WCHAR_KIND && ascii->length == 1) { Py_UCS4 ch = PyUnicode_READ_CHAR(unicode, 0); @@ -15551,11 +15529,14 @@ _PyUnicode_InitState(PyInterpreterState *interp) PyStatus _PyUnicode_InitGlobalObjects(PyInterpreterState *interp) { - struct _Py_unicode_state *state = &interp->unicode; - if (unicode_create_empty_string_singleton(state) < 0) { - return _PyStatus_NO_MEMORY(); + if (!_Py_IsMainInterpreter(interp)) { + return _PyStatus_OK(); } +#ifdef Py_DEBUG + assert(_PyUnicode_CheckConsistency(&_Py_STR(empty), 1)); +#endif + return _PyStatus_OK(); } @@ -15798,15 +15779,14 @@ PyDoc_STRVAR(length_hint_doc, "Private method returning an estimate of len(list( static PyObject * unicodeiter_reduce(unicodeiterobject *it, PyObject *Py_UNUSED(ignored)) { - _Py_IDENTIFIER(iter); if (it->it_seq != NULL) { - return Py_BuildValue("N(O)n", _PyEval_GetBuiltinId(&PyId_iter), + return Py_BuildValue("N(O)n", _PyEval_GetBuiltin(&_Py_ID(iter)), it->it_seq, it->it_index); } else { PyObject *u = (PyObject *)_PyUnicode_New(0); if (u == NULL) return NULL; - return Py_BuildValue("N(N)", _PyEval_GetBuiltinId(&PyId_iter), u); + return Py_BuildValue("N(N)", _PyEval_GetBuiltin(&_Py_ID(iter)), u); } } @@ -16137,7 +16117,6 @@ _PyUnicode_Fini(PyInterpreterState *interp) for (Py_ssize_t i = 0; i < 256; i++) { Py_CLEAR(state->latin1[i]); } - Py_CLEAR(state->empty_string); } diff --git a/Objects/unionobject.c b/Objects/unionobject.c index 80c70389ab30d..72a0a3f2cf8d2 100644 --- a/Objects/unionobject.c +++ b/Objects/unionobject.c @@ -255,10 +255,6 @@ _Py_union_type_or(PyObject* self, PyObject* other) static int union_repr_item(_PyUnicodeWriter *writer, PyObject *p) { - _Py_IDENTIFIER(__module__); - _Py_IDENTIFIER(__qualname__); - _Py_IDENTIFIER(__origin__); - _Py_IDENTIFIER(__args__); PyObject *qualname = NULL; PyObject *module = NULL; PyObject *tmp; @@ -269,13 +265,13 @@ union_repr_item(_PyUnicodeWriter *writer, PyObject *p) return _PyUnicodeWriter_WriteASCIIString(writer, "None", 4); } - if (_PyObject_LookupAttrId(p, &PyId___origin__, &tmp) < 0) { + if (_PyObject_LookupAttr(p, &_Py_ID(__origin__), &tmp) < 0) { goto exit; } if (tmp) { Py_DECREF(tmp); - if (_PyObject_LookupAttrId(p, &PyId___args__, &tmp) < 0) { + if (_PyObject_LookupAttr(p, &_Py_ID(__args__), &tmp) < 0) { goto exit; } if (tmp) { @@ -285,13 +281,13 @@ union_repr_item(_PyUnicodeWriter *writer, PyObject *p) } } - if (_PyObject_LookupAttrId(p, &PyId___qualname__, &qualname) < 0) { + if (_PyObject_LookupAttr(p, &_Py_ID(__qualname__), &qualname) < 0) { goto exit; } if (qualname == NULL) { goto use_repr; } - if (_PyObject_LookupAttrId(p, &PyId___module__, &module) < 0) { + if (_PyObject_LookupAttr(p, &_Py_ID(__module__), &module) < 0) { goto exit; } if (module == NULL || module == Py_None) { diff --git a/Objects/weakrefobject.c b/Objects/weakrefobject.c index 76121f9fe8872..71dfa640ebf57 100644 --- a/Objects/weakrefobject.c +++ b/Objects/weakrefobject.c @@ -163,7 +163,6 @@ static PyObject * weakref_repr(PyWeakReference *self) { PyObject *name, *repr; - _Py_IDENTIFIER(__name__); PyObject* obj = PyWeakref_GET_OBJECT(self); if (obj == Py_None) { @@ -171,7 +170,7 @@ weakref_repr(PyWeakReference *self) } Py_INCREF(obj); - if (_PyObject_LookupAttrId(obj, &PyId___name__, &name) < 0) { + if (_PyObject_LookupAttr(obj, &_Py_ID(__name__), &name) < 0) { Py_DECREF(obj); return NULL; } @@ -462,10 +461,9 @@ proxy_checkref(PyWeakReference *proxy) #define WRAP_METHOD(method, special) \ static PyObject * \ method(PyObject *proxy, PyObject *Py_UNUSED(ignored)) { \ - _Py_IDENTIFIER(special); \ UNWRAP(proxy); \ Py_INCREF(proxy); \ - PyObject* res = _PyObject_CallMethodIdNoArgs(proxy, &PyId_##special); \ + PyObject* res = PyObject_CallMethodNoArgs(proxy, &_Py_ID(special)); \ Py_DECREF(proxy); \ return res; \ } diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index d38df66c69ed0..b14506aaca352 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -443,8 +443,6 @@ static int fp_setreadl(struct tok_state *tok, const char* enc) { PyObject *readline, *io, *stream; - _Py_IDENTIFIER(open); - _Py_IDENTIFIER(readline); int fd; long pos; @@ -462,25 +460,28 @@ fp_setreadl(struct tok_state *tok, const char* enc) } io = PyImport_ImportModule("io"); - if (io == NULL) + if (io == NULL) { return 0; - - stream = _PyObject_CallMethodId(io, &PyId_open, "isisOOO", + } + stream = _PyObject_CallMethod(io, &_Py_ID(open), "isisOOO", fd, "r", -1, enc, Py_None, Py_None, Py_False); Py_DECREF(io); - if (stream == NULL) + if (stream == NULL) { return 0; + } - readline = _PyObject_GetAttrId(stream, &PyId_readline); + readline = PyObject_GetAttr(stream, &_Py_ID(readline)); Py_DECREF(stream); - if (readline == NULL) + if (readline == NULL) { return 0; + } Py_XSETREF(tok->decoding_readline, readline); if (pos > 0) { PyObject *bufobj = _PyObject_CallNoArgs(readline); - if (bufobj == NULL) + if (bufobj == NULL) { return 0; + } Py_DECREF(bufobj); } diff --git a/Programs/_testembed.c b/Programs/_testembed.c index 08e27d97723b3..3830dc3f8b6ec 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -1,6 +1,7 @@ #ifndef Py_BUILD_CORE_MODULE # define Py_BUILD_CORE_MODULE #endif +#define NEEDS_PY_IDENTIFIER /* Always enable assertion (even in release mode) */ #undef NDEBUG diff --git a/Programs/test_frozenmain.h b/Programs/test_frozenmain.h index 00f36ff9bd70f..01e046becd13b 100644 --- a/Programs/test_frozenmain.h +++ b/Programs/test_frozenmain.h @@ -21,7 +21,7 @@ unsigned char M_test_frozenmain[] = { 5,112,114,105,110,116,218,4,97,114,103,118,90,11,103,101, 116,95,99,111,110,102,105,103,115,114,2,0,0,0,218,3, 107,101,121,169,0,243,0,0,0,0,250,18,116,101,115,116, - 95,102,114,111,122,101,110,109,97,105,110,46,112,121,218,8, + 95,102,114,111,122,101,110,109,97,105,110,46,112,121,250,8, 60,109,111,100,117,108,101,62,114,11,0,0,0,1,0,0, 0,115,18,0,0,0,2,128,8,3,8,1,10,2,14,1, 14,1,8,1,28,7,4,249,115,20,0,0,0,2,128,8, diff --git a/Python/_warnings.c b/Python/_warnings.c index cf2110d31c3b5..a47e5fef6865f 100644 --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -14,30 +14,45 @@ PyDoc_STRVAR(warnings__doc__, MODULE_NAME " provides basic warning filtering support.\n" "It is a helper module to speed up interpreter start-up."); -_Py_IDENTIFIER(stderr); -#ifndef Py_DEBUG -_Py_IDENTIFIER(default); -_Py_IDENTIFIER(ignore); -#endif - /*************************************************************************/ typedef struct _warnings_runtime_state WarningsState; -_Py_IDENTIFIER(__name__); - -/* Given a module object, get its per-module state. */ -static WarningsState * -warnings_get_state(void) +static inline int +check_interp(PyInterpreterState *interp) { - PyInterpreterState *interp = _PyInterpreterState_GET(); if (interp == NULL) { PyErr_SetString(PyExc_RuntimeError, "warnings_get_state: could not identify " "current interpreter"); + return 0; + } + return 1; +} + +static inline PyInterpreterState * +get_current_interp(void) +{ + PyInterpreterState *interp = _PyInterpreterState_GET(); + return check_interp(interp) ? interp : NULL; +} + +static inline PyThreadState * +get_current_tstate(void) +{ + PyThreadState *tstate = _PyThreadState_GET(); + if (tstate == NULL) { + (void)check_interp(NULL); return NULL; } + return check_interp(tstate->interp) ? tstate : NULL; +} + +/* Given a module object, get its per-module state. */ +static WarningsState * +warnings_get_state(PyInterpreterState *interp) +{ return &interp->warnings; } @@ -52,13 +67,9 @@ warnings_clear_state(WarningsState *st) #ifndef Py_DEBUG static PyObject * -create_filter(PyObject *category, _Py_Identifier *id, const char *modname) +create_filter(PyObject *category, PyObject *action_str, const char *modname) { PyObject *modname_obj = NULL; - PyObject *action_str = _PyUnicode_FromId(id); - if (action_str == NULL) { - return NULL; - } /* Default to "no module name" for initial filter set */ if (modname != NULL) { @@ -79,7 +90,7 @@ create_filter(PyObject *category, _Py_Identifier *id, const char *modname) #endif static PyObject * -init_filters(void) +init_filters(PyInterpreterState *interp) { #ifdef Py_DEBUG /* Py_DEBUG builds show all warnings by default */ @@ -92,16 +103,15 @@ init_filters(void) } size_t pos = 0; /* Post-incremented in each use. */ - PyList_SET_ITEM(filters, pos++, - create_filter(PyExc_DeprecationWarning, &PyId_default, "__main__")); - PyList_SET_ITEM(filters, pos++, - create_filter(PyExc_DeprecationWarning, &PyId_ignore, NULL)); - PyList_SET_ITEM(filters, pos++, - create_filter(PyExc_PendingDeprecationWarning, &PyId_ignore, NULL)); - PyList_SET_ITEM(filters, pos++, - create_filter(PyExc_ImportWarning, &PyId_ignore, NULL)); - PyList_SET_ITEM(filters, pos++, - create_filter(PyExc_ResourceWarning, &PyId_ignore, NULL)); +#define ADD(TYPE, ACTION, MODNAME) \ + PyList_SET_ITEM(filters, pos++, \ + create_filter(TYPE, &_Py_ID(ACTION), MODNAME)); + ADD(PyExc_DeprecationWarning, default, "__main__"); + ADD(PyExc_DeprecationWarning, ignore, NULL); + ADD(PyExc_PendingDeprecationWarning, ignore, NULL); + ADD(PyExc_ImportWarning, ignore, NULL); + ADD(PyExc_ResourceWarning, ignore, NULL); +#undef ADD for (size_t x = 0; x < pos; x++) { if (PyList_GET_ITEM(filters, x) == NULL) { @@ -120,7 +130,7 @@ _PyWarnings_InitState(PyInterpreterState *interp) WarningsState *st = &interp->warnings; if (st->filters == NULL) { - st->filters = init_filters(); + st->filters = init_filters(interp); if (st->filters == NULL) { return -1; } @@ -148,10 +158,9 @@ _PyWarnings_InitState(PyInterpreterState *interp) /*************************************************************************/ static int -check_matched(PyObject *obj, PyObject *arg) +check_matched(PyInterpreterState *interp, PyObject *obj, PyObject *arg) { PyObject *result; - _Py_IDENTIFIER(match); int rc; /* A 'None' filter always matches */ @@ -168,7 +177,7 @@ check_matched(PyObject *obj, PyObject *arg) } /* Otherwise assume a regex filter and call its match() method */ - result = _PyObject_CallMethodIdOneArg(obj, &PyId_match, arg); + result = PyObject_CallMethodOneArg(obj, &_Py_ID(match), arg); if (result == NULL) return -1; @@ -177,25 +186,21 @@ check_matched(PyObject *obj, PyObject *arg) return rc; } +#define GET_WARNINGS_ATTR(interp, attr, try_import) \ + get_warnings_attr(interp, &_Py_ID(attr), try_import) + /* Returns a new reference. A NULL return value can mean false or an error. */ static PyObject * -get_warnings_attr(_Py_Identifier *attr_id, int try_import) +get_warnings_attr(PyInterpreterState *interp, PyObject *attr, int try_import) { - PyObject *warnings_str; PyObject *warnings_module, *obj; - _Py_IDENTIFIER(warnings); - - warnings_str = _PyUnicode_FromId(&PyId_warnings); - if (warnings_str == NULL) { - return NULL; - } /* don't try to import after the start of the Python finallization */ if (try_import && !_Py_IsFinalizing()) { - warnings_module = PyImport_Import(warnings_str); + warnings_module = PyImport_Import(&_Py_ID(warnings)); if (warnings_module == NULL) { /* Fallback to the C implementation if we cannot get the Python implementation */ @@ -210,27 +215,31 @@ get_warnings_attr(_Py_Identifier *attr_id, int try_import) gone, then we can't even use PyImport_GetModule without triggering an interpreter abort. */ - if (!_PyInterpreterState_GET()->modules) { + if (!interp->modules) { return NULL; } - warnings_module = PyImport_GetModule(warnings_str); + warnings_module = PyImport_GetModule(&_Py_ID(warnings)); if (warnings_module == NULL) return NULL; } - (void)_PyObject_LookupAttrId(warnings_module, attr_id, &obj); + (void)_PyObject_LookupAttr(warnings_module, attr, &obj); Py_DECREF(warnings_module); return obj; } static PyObject * -get_once_registry(WarningsState *st) +get_once_registry(PyInterpreterState *interp) { PyObject *registry; - _Py_IDENTIFIER(onceregistry); - registry = get_warnings_attr(&PyId_onceregistry, 0); + WarningsState *st = warnings_get_state(interp); + if (st == NULL) { + return NULL; + } + + registry = GET_WARNINGS_ATTR(interp, onceregistry, 0); if (registry == NULL) { if (PyErr_Occurred()) return NULL; @@ -251,12 +260,16 @@ get_once_registry(WarningsState *st) static PyObject * -get_default_action(WarningsState *st) +get_default_action(PyInterpreterState *interp) { PyObject *default_action; - _Py_IDENTIFIER(defaultaction); - default_action = get_warnings_attr(&PyId_defaultaction, 0); + WarningsState *st = warnings_get_state(interp); + if (st == NULL) { + return NULL; + } + + default_action = GET_WARNINGS_ATTR(interp, defaultaction, 0); if (default_action == NULL) { if (PyErr_Occurred()) { return NULL; @@ -279,19 +292,19 @@ get_default_action(WarningsState *st) /* The item is a new reference. */ static PyObject* -get_filter(PyObject *category, PyObject *text, Py_ssize_t lineno, +get_filter(PyInterpreterState *interp, PyObject *category, + PyObject *text, Py_ssize_t lineno, PyObject *module, PyObject **item) { PyObject *action; Py_ssize_t i; PyObject *warnings_filters; - _Py_IDENTIFIER(filters); - WarningsState *st = warnings_get_state(); + WarningsState *st = warnings_get_state(interp); if (st == NULL) { return NULL; } - warnings_filters = get_warnings_attr(&PyId_filters, 0); + warnings_filters = GET_WARNINGS_ATTR(interp, filters, 0); if (warnings_filters == NULL) { if (PyErr_Occurred()) return NULL; @@ -336,13 +349,13 @@ get_filter(PyObject *category, PyObject *text, Py_ssize_t lineno, return NULL; } - good_msg = check_matched(msg, text); + good_msg = check_matched(interp, msg, text); if (good_msg == -1) { Py_DECREF(tmp_item); return NULL; } - good_mod = check_matched(mod, module); + good_mod = check_matched(interp, mod, module); if (good_mod == -1) { Py_DECREF(tmp_item); return NULL; @@ -368,7 +381,7 @@ get_filter(PyObject *category, PyObject *text, Py_ssize_t lineno, Py_DECREF(tmp_item); } - action = get_default_action(st); + action = get_default_action(interp); if (action != NULL) { Py_INCREF(Py_None); *item = Py_None; @@ -380,19 +393,19 @@ get_filter(PyObject *category, PyObject *text, Py_ssize_t lineno, static int -already_warned(PyObject *registry, PyObject *key, int should_set) +already_warned(PyInterpreterState *interp, PyObject *registry, PyObject *key, + int should_set) { PyObject *version_obj, *already_warned; - _Py_IDENTIFIER(version); if (key == NULL) return -1; - WarningsState *st = warnings_get_state(); + WarningsState *st = warnings_get_state(interp); if (st == NULL) { return -1; } - version_obj = _PyDict_GetItemIdWithError(registry, &PyId_version); + version_obj = _PyDict_GetItemWithError(registry, &_Py_ID(version)); if (version_obj == NULL || !PyLong_CheckExact(version_obj) || PyLong_AsLong(version_obj) != st->filters_version) @@ -404,7 +417,7 @@ already_warned(PyObject *registry, PyObject *key, int should_set) version_obj = PyLong_FromLong(st->filters_version); if (version_obj == NULL) return -1; - if (_PyDict_SetItemId(registry, &PyId_version, version_obj) < 0) { + if (PyDict_SetItem(registry, &_Py_ID(version), version_obj) < 0) { Py_DECREF(version_obj); return -1; } @@ -463,8 +476,8 @@ normalize_module(PyObject *filename) } static int -update_registry(PyObject *registry, PyObject *text, PyObject *category, - int add_zero) +update_registry(PyInterpreterState *interp, PyObject *registry, PyObject *text, + PyObject *category, int add_zero) { PyObject *altkey; int rc; @@ -474,14 +487,14 @@ update_registry(PyObject *registry, PyObject *text, PyObject *category, else altkey = PyTuple_Pack(2, text, category); - rc = already_warned(registry, altkey, 1); + rc = already_warned(interp, registry, altkey, 1); Py_XDECREF(altkey); return rc; } static void -show_warning(PyObject *filename, int lineno, PyObject *text, - PyObject *category, PyObject *sourceline) +show_warning(PyThreadState *tstate, PyObject *filename, int lineno, + PyObject *text, PyObject *category, PyObject *sourceline) { PyObject *f_stderr; PyObject *name; @@ -489,12 +502,12 @@ show_warning(PyObject *filename, int lineno, PyObject *text, PyOS_snprintf(lineno_str, sizeof(lineno_str), ":%d: ", lineno); - name = _PyObject_GetAttrId(category, &PyId___name__); + name = PyObject_GetAttr(category, &_Py_ID(__name__)); if (name == NULL) { goto error; } - f_stderr = _PySys_GetObjectId(&PyId_stderr); + f_stderr = _PySys_GetAttr(tstate, &_Py_ID(stderr)); if (f_stderr == NULL) { fprintf(stderr, "lost sys.stderr\n"); goto error; @@ -553,22 +566,22 @@ show_warning(PyObject *filename, int lineno, PyObject *text, } static int -call_show_warning(PyObject *category, PyObject *text, PyObject *message, +call_show_warning(PyThreadState *tstate, PyObject *category, + PyObject *text, PyObject *message, PyObject *filename, int lineno, PyObject *lineno_obj, PyObject *sourceline, PyObject *source) { PyObject *show_fn, *msg, *res, *warnmsg_cls = NULL; - _Py_IDENTIFIER(_showwarnmsg); - _Py_IDENTIFIER(WarningMessage); + PyInterpreterState *interp = tstate->interp; /* If the source parameter is set, try to get the Python implementation. The Python implementation is able to log the traceback where the source was allocated, whereas the C implementation doesn't. */ - show_fn = get_warnings_attr(&PyId__showwarnmsg, source != NULL); + show_fn = GET_WARNINGS_ATTR(interp, _showwarnmsg, source != NULL); if (show_fn == NULL) { if (PyErr_Occurred()) return -1; - show_warning(filename, lineno, text, category, sourceline); + show_warning(tstate, filename, lineno, text, category, sourceline); return 0; } @@ -578,7 +591,7 @@ call_show_warning(PyObject *category, PyObject *text, PyObject *message, goto error; } - warnmsg_cls = get_warnings_attr(&PyId_WarningMessage, 0); + warnmsg_cls = GET_WARNINGS_ATTR(interp, WarningMessage, 0); if (warnmsg_cls == NULL) { if (!PyErr_Occurred()) { PyErr_SetString(PyExc_RuntimeError, @@ -610,7 +623,7 @@ call_show_warning(PyObject *category, PyObject *text, PyObject *message, } static PyObject * -warn_explicit(PyObject *category, PyObject *message, +warn_explicit(PyThreadState *tstate, PyObject *category, PyObject *message, PyObject *filename, int lineno, PyObject *module, PyObject *registry, PyObject *sourceline, PyObject *source) @@ -619,6 +632,7 @@ warn_explicit(PyObject *category, PyObject *message, PyObject *item = NULL; PyObject *action; int rc; + PyInterpreterState *interp = tstate->interp; /* module can be None if a warning is emitted late during Python shutdown. In this case, the Python warnings module was probably unloaded, filters @@ -674,7 +688,7 @@ warn_explicit(PyObject *category, PyObject *message, goto cleanup; if ((registry != NULL) && (registry != Py_None)) { - rc = already_warned(registry, key, 0); + rc = already_warned(interp, registry, key, 0); if (rc == -1) goto cleanup; else if (rc == 1) @@ -682,7 +696,7 @@ warn_explicit(PyObject *category, PyObject *message, /* Else this warning hasn't been generated before. */ } - action = get_filter(category, text, lineno, module, &item); + action = get_filter(interp, category, text, lineno, module, &item); if (action == NULL) goto cleanup; @@ -707,21 +721,17 @@ warn_explicit(PyObject *category, PyObject *message, if (_PyUnicode_EqualToASCIIString(action, "once")) { if (registry == NULL || registry == Py_None) { - WarningsState *st = warnings_get_state(); - if (st == NULL) { - goto cleanup; - } - registry = get_once_registry(st); + registry = get_once_registry(interp); if (registry == NULL) goto cleanup; } /* WarningsState.once_registry[(text, category)] = 1 */ - rc = update_registry(registry, text, category, 0); + rc = update_registry(interp, registry, text, category, 0); } else if (_PyUnicode_EqualToASCIIString(action, "module")) { /* registry[(text, category, 0)] = 1 */ if (registry != NULL && registry != Py_None) - rc = update_registry(registry, text, category, 0); + rc = update_registry(interp, registry, text, category, 0); } else if (!_PyUnicode_EqualToASCIIString(action, "default")) { PyErr_Format(PyExc_RuntimeError, @@ -734,8 +744,8 @@ warn_explicit(PyObject *category, PyObject *message, if (rc == 1) /* Already warned for this module. */ goto return_none; if (rc == 0) { - if (call_show_warning(category, text, message, filename, lineno, - lineno_obj, sourceline, source) < 0) + if (call_show_warning(tstate, category, text, message, filename, + lineno, lineno_obj, sourceline, source) < 0) goto cleanup; } else /* if (rc == -1) */ @@ -827,11 +837,14 @@ static int setup_context(Py_ssize_t stack_level, PyObject **filename, int *lineno, PyObject **module, PyObject **registry) { - _Py_IDENTIFIER(__warningregistry__); PyObject *globals; /* Setup globals, filename and lineno. */ - PyThreadState *tstate = _PyThreadState_GET(); + PyThreadState *tstate = get_current_tstate(); + if (tstate == NULL) { + return 0; + } + PyInterpreterState *interp = tstate->interp; PyFrameObject *f = PyThreadState_GetFrame(tstate); // Stack level comparisons to Python code is off by one as there is no // warnings-related stack level to avoid. @@ -849,7 +862,7 @@ setup_context(Py_ssize_t stack_level, PyObject **filename, int *lineno, } if (f == NULL) { - globals = tstate->interp->sysdict; + globals = interp->sysdict; *filename = PyUnicode_FromString("sys"); *lineno = 1; } @@ -866,7 +879,7 @@ setup_context(Py_ssize_t stack_level, PyObject **filename, int *lineno, /* Setup registry. */ assert(globals != NULL); assert(PyDict_Check(globals)); - *registry = _PyDict_GetItemIdWithError(globals, &PyId___warningregistry__); + *registry = _PyDict_GetItemWithError(globals, &_Py_ID(__warningregistry__)); if (*registry == NULL) { int rc; @@ -877,7 +890,7 @@ setup_context(Py_ssize_t stack_level, PyObject **filename, int *lineno, if (*registry == NULL) goto handle_error; - rc = _PyDict_SetItemId(globals, &PyId___warningregistry__, *registry); + rc = PyDict_SetItem(globals, &_Py_ID(__warningregistry__), *registry); if (rc < 0) goto handle_error; } @@ -885,7 +898,7 @@ setup_context(Py_ssize_t stack_level, PyObject **filename, int *lineno, Py_INCREF(*registry); /* Setup module. */ - *module = _PyDict_GetItemIdWithError(globals, &PyId___name__); + *module = _PyDict_GetItemWithError(globals, &_Py_ID(__name__)); if (*module == Py_None || (*module != NULL && PyUnicode_Check(*module))) { Py_INCREF(*module); } @@ -943,10 +956,15 @@ do_warn(PyObject *message, PyObject *category, Py_ssize_t stack_level, PyObject *filename, *module, *registry, *res; int lineno; + PyThreadState *tstate = get_current_tstate(); + if (tstate == NULL) { + return NULL; + } + if (!setup_context(stack_level, &filename, &lineno, &module, ®istry)) return NULL; - res = warn_explicit(category, message, filename, lineno, module, registry, + res = warn_explicit(tstate, category, message, filename, lineno, module, registry, NULL, source); Py_DECREF(filename); Py_DECREF(registry); @@ -977,10 +995,8 @@ warnings_warn_impl(PyObject *module, PyObject *message, PyObject *category, } static PyObject * -get_source_line(PyObject *module_globals, int lineno) +get_source_line(PyInterpreterState *interp, PyObject *module_globals, int lineno) { - _Py_IDENTIFIER(get_source); - _Py_IDENTIFIER(__loader__); PyObject *loader; PyObject *module_name; PyObject *get_source; @@ -989,12 +1005,12 @@ get_source_line(PyObject *module_globals, int lineno) PyObject *source_line; /* Check/get the requisite pieces needed for the loader. */ - loader = _PyDict_GetItemIdWithError(module_globals, &PyId___loader__); + loader = _PyDict_GetItemWithError(module_globals, &_Py_ID(__loader__)); if (loader == NULL) { return NULL; } Py_INCREF(loader); - module_name = _PyDict_GetItemIdWithError(module_globals, &PyId___name__); + module_name = _PyDict_GetItemWithError(module_globals, &_Py_ID(__name__)); if (!module_name) { Py_DECREF(loader); return NULL; @@ -1002,7 +1018,7 @@ get_source_line(PyObject *module_globals, int lineno) Py_INCREF(module_name); /* Make sure the loader implements the optional get_source() method. */ - (void)_PyObject_LookupAttrId(loader, &PyId_get_source, &get_source); + (void)_PyObject_LookupAttr(loader, &_Py_ID(get_source), &get_source); Py_DECREF(loader); if (!get_source) { Py_DECREF(module_name); @@ -1056,6 +1072,11 @@ warnings_warn_explicit(PyObject *self, PyObject *args, PyObject *kwds) ®istry, &module_globals, &sourceobj)) return NULL; + PyThreadState *tstate = get_current_tstate(); + if (tstate == NULL) { + return NULL; + } + if (module_globals && module_globals != Py_None) { if (!PyDict_Check(module_globals)) { PyErr_Format(PyExc_TypeError, @@ -1064,12 +1085,12 @@ warnings_warn_explicit(PyObject *self, PyObject *args, PyObject *kwds) return NULL; } - source_line = get_source_line(module_globals, lineno); + source_line = get_source_line(tstate->interp, module_globals, lineno); if (source_line == NULL && PyErr_Occurred()) { return NULL; } } - returned = warn_explicit(category, message, filename, lineno, module, + returned = warn_explicit(tstate, category, message, filename, lineno, module, registry, source_line, sourceobj); Py_XDECREF(source_line); return returned; @@ -1078,7 +1099,11 @@ warnings_warn_explicit(PyObject *self, PyObject *args, PyObject *kwds) static PyObject * warnings_filters_mutated(PyObject *self, PyObject *args) { - WarningsState *st = warnings_get_state(); + PyInterpreterState *interp = get_current_interp(); + if (interp == NULL) { + return NULL; + } + WarningsState *st = warnings_get_state(interp); if (st == NULL) { return NULL; } @@ -1208,7 +1233,11 @@ PyErr_WarnExplicitObject(PyObject *category, PyObject *message, PyObject *res; if (category == NULL) category = PyExc_RuntimeWarning; - res = warn_explicit(category, message, filename, lineno, + PyThreadState *tstate = get_current_tstate(); + if (tstate == NULL) { + return -1; + } + res = warn_explicit(tstate, category, message, filename, lineno, module, registry, NULL, NULL); if (res == NULL) return -1; @@ -1272,12 +1301,15 @@ PyErr_WarnExplicitFormat(PyObject *category, message = PyUnicode_FromFormatV(format, vargs); if (message != NULL) { PyObject *res; - res = warn_explicit(category, message, filename, lineno, - module, registry, NULL, NULL); - Py_DECREF(message); - if (res != NULL) { - Py_DECREF(res); - ret = 0; + PyThreadState *tstate = get_current_tstate(); + if (tstate != NULL) { + res = warn_explicit(tstate, category, message, filename, lineno, + module, registry, NULL, NULL); + Py_DECREF(message); + if (res != NULL) { + Py_DECREF(res); + ret = 0; + } } } va_end(vargs); @@ -1309,9 +1341,10 @@ _PyErr_WarnUnawaitedCoroutine(PyObject *coro) Since this is called from __del__ context, it's careful to never raise an exception. */ - _Py_IDENTIFIER(_warn_unawaited_coroutine); int warned = 0; - PyObject *fn = get_warnings_attr(&PyId__warn_unawaited_coroutine, 1); + PyInterpreterState *interp = _PyInterpreterState_GET(); + assert(interp != NULL); + PyObject *fn = GET_WARNINGS_ATTR(interp, _warn_unawaited_coroutine, 1); if (fn) { PyObject *res = PyObject_CallOneArg(fn, coro); Py_DECREF(fn); @@ -1352,7 +1385,11 @@ static PyMethodDef warnings_functions[] = { static int warnings_module_exec(PyObject *module) { - WarningsState *st = warnings_get_state(); + PyInterpreterState *interp = get_current_interp(); + if (interp == NULL) { + return -1; + } + WarningsState *st = warnings_get_state(interp); if (st == NULL) { return -1; } diff --git a/Python/ast_opt.c b/Python/ast_opt.c index 967e75d5771c5..2911370649790 100644 --- a/Python/ast_opt.c +++ b/Python/ast_opt.c @@ -268,15 +268,8 @@ parse_literal(PyObject *fmt, Py_ssize_t *ppos, PyArena *arena) PyObject *str = PyUnicode_Substring(fmt, start, pos); /* str = str.replace('%%', '%') */ if (str && has_percents) { - _Py_static_string(PyId_double_percent, "%%"); - _Py_static_string(PyId_percent, "%"); - PyObject *double_percent = _PyUnicode_FromId(&PyId_double_percent); - PyObject *percent = _PyUnicode_FromId(&PyId_percent); - if (!double_percent || !percent) { - Py_DECREF(str); - return NULL; - } - Py_SETREF(str, PyUnicode_Replace(str, double_percent, percent, -1)); + Py_SETREF(str, PyUnicode_Replace(str, &_Py_STR(dbl_percent), + &_Py_STR(percent), -1)); } if (!str) { return NULL; diff --git a/Python/bltinmodule.c b/Python/bltinmodule.c index ed612091d8394..b253f88a04bae 100644 --- a/Python/bltinmodule.c +++ b/Python/bltinmodule.c @@ -11,21 +11,6 @@ #include "pycore_tuple.h" // _PyTuple_FromArray() #include "pycore_ceval.h" // _PyEval_Vector() -_Py_IDENTIFIER(__builtins__); -_Py_IDENTIFIER(__dict__); -_Py_IDENTIFIER(__prepare__); -_Py_IDENTIFIER(__round__); -_Py_IDENTIFIER(__mro_entries__); -_Py_IDENTIFIER(encoding); -_Py_IDENTIFIER(errors); -_Py_IDENTIFIER(fileno); -_Py_IDENTIFIER(flush); -_Py_IDENTIFIER(metaclass); -_Py_IDENTIFIER(sort); -_Py_IDENTIFIER(stdin); -_Py_IDENTIFIER(stdout); -_Py_IDENTIFIER(stderr); - #include "clinic/bltinmodule.c.h" static PyObject* @@ -47,7 +32,7 @@ update_bases(PyObject *bases, PyObject *const *args, Py_ssize_t nargs) } continue; } - if (_PyObject_LookupAttrId(base, &PyId___mro_entries__, &meth) < 0) { + if (_PyObject_LookupAttr(base, &_Py_ID(__mro_entries__), &meth) < 0) { goto error; } if (!meth) { @@ -148,10 +133,10 @@ builtin___build_class__(PyObject *self, PyObject *const *args, Py_ssize_t nargs, goto error; } - meta = _PyDict_GetItemIdWithError(mkw, &PyId_metaclass); + meta = _PyDict_GetItemWithError(mkw, &_Py_ID(metaclass)); if (meta != NULL) { Py_INCREF(meta); - if (_PyDict_DelItemId(mkw, &PyId_metaclass) < 0) { + if (PyDict_DelItem(mkw, &_Py_ID(metaclass)) < 0) { goto error; } /* metaclass is explicitly given, check if it's indeed a class */ @@ -191,7 +176,7 @@ builtin___build_class__(PyObject *self, PyObject *const *args, Py_ssize_t nargs, } /* else: meta is not a class, so we cannot do the metaclass calculation, so we will use the explicitly given object as it is */ - if (_PyObject_LookupAttrId(meta, &PyId___prepare__, &prep) < 0) { + if (_PyObject_LookupAttr(meta, &_Py_ID(__prepare__), &prep) < 0) { ns = NULL; } else if (prep == NULL) { @@ -946,10 +931,9 @@ builtin_eval_impl(PyObject *module, PyObject *source, PyObject *globals, return NULL; } - int r = _PyDict_ContainsId(globals, &PyId___builtins__); + int r = PyDict_Contains(globals, &_Py_ID(__builtins__)); if (r == 0) { - r = _PyDict_SetItemId(globals, &PyId___builtins__, - PyEval_GetBuiltins()); + r = PyDict_SetItem(globals, &_Py_ID(__builtins__), PyEval_GetBuiltins()); } if (r < 0) { return NULL; @@ -1034,10 +1018,9 @@ builtin_exec_impl(PyObject *module, PyObject *source, PyObject *globals, Py_TYPE(locals)->tp_name); return NULL; } - int r = _PyDict_ContainsId(globals, &PyId___builtins__); + int r = PyDict_Contains(globals, &_Py_ID(__builtins__)); if (r == 0) { - r = _PyDict_SetItemId(globals, &PyId___builtins__, - PyEval_GetBuiltins()); + r = PyDict_SetItem(globals, &_Py_ID(__builtins__), PyEval_GetBuiltins()); } if (r < 0) { return NULL; @@ -1960,7 +1943,8 @@ builtin_print_impl(PyObject *module, PyObject *args, PyObject *sep, int i, err; if (file == Py_None) { - file = _PySys_GetObjectId(&PyId_stdout); + PyThreadState *tstate = _PyThreadState_GET(); + file = _PySys_GetAttr(tstate, &_Py_ID(stdout)); if (file == NULL) { PyErr_SetString(PyExc_RuntimeError, "lost sys.stdout"); return NULL; @@ -2020,7 +2004,7 @@ builtin_print_impl(PyObject *module, PyObject *args, PyObject *sep, } if (flush) { - PyObject *tmp = _PyObject_CallMethodIdNoArgs(file, &PyId_flush); + PyObject *tmp = PyObject_CallMethodNoArgs(file, &_Py_ID(flush)); if (tmp == NULL) { return NULL; } @@ -2050,9 +2034,13 @@ static PyObject * builtin_input_impl(PyObject *module, PyObject *prompt) /*[clinic end generated code: output=83db5a191e7a0d60 input=5e8bb70c2908fe3c]*/ { - PyObject *fin = _PySys_GetObjectId(&PyId_stdin); - PyObject *fout = _PySys_GetObjectId(&PyId_stdout); - PyObject *ferr = _PySys_GetObjectId(&PyId_stderr); + PyThreadState *tstate = _PyThreadState_GET(); + PyObject *fin = _PySys_GetAttr( + tstate, &_Py_ID(stdin)); + PyObject *fout = _PySys_GetAttr( + tstate, &_Py_ID(stdout)); + PyObject *ferr = _PySys_GetAttr( + tstate, &_Py_ID(stderr)); PyObject *tmp; long fd; int tty; @@ -2079,7 +2067,7 @@ builtin_input_impl(PyObject *module, PyObject *prompt) } /* First of all, flush stderr */ - tmp = _PyObject_CallMethodIdNoArgs(ferr, &PyId_flush); + tmp = PyObject_CallMethodNoArgs(ferr, &_Py_ID(flush)); if (tmp == NULL) PyErr_Clear(); else @@ -2088,7 +2076,7 @@ builtin_input_impl(PyObject *module, PyObject *prompt) /* We should only use (GNU) readline if Python's sys.stdin and sys.stdout are the same as C's stdin and stdout, because we need to pass it those. */ - tmp = _PyObject_CallMethodIdNoArgs(fin, &PyId_fileno); + tmp = PyObject_CallMethodNoArgs(fin, &_Py_ID(fileno)); if (tmp == NULL) { PyErr_Clear(); tty = 0; @@ -2101,7 +2089,7 @@ builtin_input_impl(PyObject *module, PyObject *prompt) tty = fd == fileno(stdin) && isatty(fd); } if (tty) { - tmp = _PyObject_CallMethodIdNoArgs(fout, &PyId_fileno); + tmp = PyObject_CallMethodNoArgs(fout, &_Py_ID(fileno)); if (tmp == NULL) { PyErr_Clear(); tty = 0; @@ -2127,8 +2115,8 @@ builtin_input_impl(PyObject *module, PyObject *prompt) size_t len; /* stdin is a text stream, so it must have an encoding. */ - stdin_encoding = _PyObject_GetAttrId(fin, &PyId_encoding); - stdin_errors = _PyObject_GetAttrId(fin, &PyId_errors); + stdin_encoding = PyObject_GetAttr(fin, &_Py_ID(encoding)); + stdin_errors = PyObject_GetAttr(fin, &_Py_ID(errors)); if (!stdin_encoding || !stdin_errors || !PyUnicode_Check(stdin_encoding) || !PyUnicode_Check(stdin_errors)) { @@ -2139,7 +2127,7 @@ builtin_input_impl(PyObject *module, PyObject *prompt) stdin_errors_str = PyUnicode_AsUTF8(stdin_errors); if (!stdin_encoding_str || !stdin_errors_str) goto _readline_errors; - tmp = _PyObject_CallMethodIdNoArgs(fout, &PyId_flush); + tmp = PyObject_CallMethodNoArgs(fout, &_Py_ID(flush)); if (tmp == NULL) PyErr_Clear(); else @@ -2148,8 +2136,8 @@ builtin_input_impl(PyObject *module, PyObject *prompt) /* We have a prompt, encode it as stdout would */ const char *stdout_encoding_str, *stdout_errors_str; PyObject *stringpo; - stdout_encoding = _PyObject_GetAttrId(fout, &PyId_encoding); - stdout_errors = _PyObject_GetAttrId(fout, &PyId_errors); + stdout_encoding = PyObject_GetAttr(fout, &_Py_ID(encoding)); + stdout_errors = PyObject_GetAttr(fout, &_Py_ID(errors)); if (!stdout_encoding || !stdout_errors || !PyUnicode_Check(stdout_encoding) || !PyUnicode_Check(stdout_errors)) { @@ -2234,7 +2222,7 @@ builtin_input_impl(PyObject *module, PyObject *prompt) if (PyFile_WriteObject(prompt, fout, Py_PRINT_RAW) != 0) return NULL; } - tmp = _PyObject_CallMethodIdNoArgs(fout, &PyId_flush); + tmp = PyObject_CallMethodNoArgs(fout, &_Py_ID(flush)); if (tmp == NULL) PyErr_Clear(); else @@ -2285,7 +2273,7 @@ builtin_round_impl(PyObject *module, PyObject *number, PyObject *ndigits) return NULL; } - round = _PyObject_LookupSpecial(number, &PyId___round__); + round = _PyObject_LookupSpecial(number, &_Py_ID(__round__)); if (round == NULL) { if (!PyErr_Occurred()) PyErr_Format(PyExc_TypeError, @@ -2346,7 +2334,7 @@ builtin_sorted(PyObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject if (newlist == NULL) return NULL; - callable = _PyObject_GetAttrId(newlist, &PyId_sort); + callable = PyObject_GetAttr(newlist, &_Py_ID(sort)); if (callable == NULL) { Py_DECREF(newlist); return NULL; @@ -2378,7 +2366,7 @@ builtin_vars(PyObject *self, PyObject *args) Py_XINCREF(d); } else { - if (_PyObject_LookupAttrId(v, &PyId___dict__, &d) == 0) { + if (_PyObject_LookupAttr(v, &_Py_ID(__dict__), &d) == 0) { PyErr_SetString(PyExc_TypeError, "vars() argument must have __dict__ attribute"); } diff --git a/Python/ceval.c b/Python/ceval.c index dcceee582a4e6..02e4e7b9e4d54 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -48,8 +48,6 @@ # error "ceval.c must be build with Py_BUILD_CORE define for best performance" #endif -_Py_IDENTIFIER(__name__); - /* Forward declarations */ static PyObject *trace_call_function( PyThreadState *tstate, PyObject *callable, PyObject **stack, @@ -864,18 +862,12 @@ match_keys(PyThreadState *tstate, PyObject *map, PyObject *keys) PyObject *seen = NULL; PyObject *dummy = NULL; PyObject *values = NULL; - PyObject *get_name = NULL; PyObject *get = NULL; // We use the two argument form of map.get(key, default) for two reasons: // - Atomically check for a key and get its value without error handling. // - Don't cause key creation or resizing in dict subclasses like // collections.defaultdict that define __missing__ (or similar). - _Py_IDENTIFIER(get); - get_name = _PyUnicode_FromId(&PyId_get); // borrowed - if (get_name == NULL) { - return NULL; - } - int meth_found = _PyObject_GetMethod(map, get_name, &get); + int meth_found = _PyObject_GetMethod(map, &_Py_ID(get), &get); if (get == NULL) { goto fail; } @@ -1692,9 +1684,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr SET_LOCALS_FROM_FRAME(); #ifdef LLTRACE - _Py_IDENTIFIER(__ltrace__); { - int r = _PyDict_ContainsId(GLOBALS(), &PyId___ltrace__); + int r = PyDict_Contains(GLOBALS(), &_Py_ID(__ltrace__)); if (r < 0) { goto exit_unwind; } @@ -2330,9 +2321,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(PRINT_EXPR) { - _Py_IDENTIFIER(displayhook); PyObject *value = POP(); - PyObject *hook = _PySys_GetObjectId(&PyId_displayhook); + PyObject *hook = _PySys_GetAttr(tstate, &_Py_ID(displayhook)); PyObject *res; if (hook == NULL) { _PyErr_SetString(tstate, PyExc_RuntimeError, @@ -2537,12 +2527,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (tstate->c_tracefunc == NULL) { gen_status = PyIter_Send(receiver, v, &retval); } else { - _Py_IDENTIFIER(send); if (Py_IsNone(v) && PyIter_Check(receiver)) { retval = Py_TYPE(receiver)->tp_iternext(receiver); } else { - retval = _PyObject_CallMethodIdOneArg(receiver, &PyId_send, v); + retval = PyObject_CallMethodOneArg(receiver, &_Py_ID(send), v); } if (retval == NULL) { if (tstate->c_tracefunc != NULL @@ -2675,11 +2664,10 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(LOAD_BUILD_CLASS) { - _Py_IDENTIFIER(__build_class__); - PyObject *bc; if (PyDict_CheckExact(BUILTINS())) { - bc = _PyDict_GetItemIdWithError(BUILTINS(), &PyId___build_class__); + bc = _PyDict_GetItemWithError(BUILTINS(), + &_Py_ID(__build_class__)); if (bc == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_SetString(tstate, PyExc_NameError, @@ -2690,10 +2678,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr Py_INCREF(bc); } else { - PyObject *build_class_str = _PyUnicode_FromId(&PyId___build_class__); - if (build_class_str == NULL) - goto error; - bc = PyObject_GetItem(BUILTINS(), build_class_str); + bc = PyObject_GetItem(BUILTINS(), &_Py_ID(__build_class__)); if (bc == NULL) { if (_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) _PyErr_SetString(tstate, PyExc_NameError, @@ -3252,7 +3237,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(SETUP_ANNOTATIONS) { - _Py_IDENTIFIER(__annotations__); int err; PyObject *ann_dict; if (LOCALS() == NULL) { @@ -3262,8 +3246,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } /* check if __annotations__ in locals()... */ if (PyDict_CheckExact(LOCALS())) { - ann_dict = _PyDict_GetItemIdWithError(LOCALS(), - &PyId___annotations__); + ann_dict = _PyDict_GetItemWithError(LOCALS(), + &_Py_ID(__annotations__)); if (ann_dict == NULL) { if (_PyErr_Occurred(tstate)) { goto error; @@ -3273,8 +3257,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (ann_dict == NULL) { goto error; } - err = _PyDict_SetItemId(LOCALS(), - &PyId___annotations__, ann_dict); + err = PyDict_SetItem(LOCALS(), &_Py_ID(__annotations__), + ann_dict); Py_DECREF(ann_dict); if (err != 0) { goto error; @@ -3283,11 +3267,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } else { /* do the same if locals() is not a dict */ - PyObject *ann_str = _PyUnicode_FromId(&PyId___annotations__); - if (ann_str == NULL) { - goto error; - } - ann_dict = PyObject_GetItem(LOCALS(), ann_str); + ann_dict = PyObject_GetItem(LOCALS(), &_Py_ID(__annotations__)); if (ann_dict == NULL) { if (!_PyErr_ExceptionMatches(tstate, PyExc_KeyError)) { goto error; @@ -3297,7 +3277,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (ann_dict == NULL) { goto error; } - err = PyObject_SetItem(LOCALS(), ann_str, ann_dict); + err = PyObject_SetItem(LOCALS(), &_Py_ID(__annotations__), + ann_dict); Py_DECREF(ann_dict); if (err != 0) { goto error; @@ -4203,11 +4184,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(BEFORE_ASYNC_WITH) { - _Py_IDENTIFIER(__aenter__); - _Py_IDENTIFIER(__aexit__); PyObject *mgr = TOP(); PyObject *res; - PyObject *enter = _PyObject_LookupSpecial(mgr, &PyId___aenter__); + PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__aenter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_Format(tstate, PyExc_TypeError, @@ -4217,7 +4196,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } goto error; } - PyObject *exit = _PyObject_LookupSpecial(mgr, &PyId___aexit__); + PyObject *exit = _PyObject_LookupSpecial(mgr, &_Py_ID(__aexit__)); if (exit == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_Format(tstate, PyExc_TypeError, @@ -4241,11 +4220,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(BEFORE_WITH) { - _Py_IDENTIFIER(__enter__); - _Py_IDENTIFIER(__exit__); PyObject *mgr = TOP(); PyObject *res; - PyObject *enter = _PyObject_LookupSpecial(mgr, &PyId___enter__); + PyObject *enter = _PyObject_LookupSpecial(mgr, &_Py_ID(__enter__)); if (enter == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_Format(tstate, PyExc_TypeError, @@ -4255,7 +4232,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } goto error; } - PyObject *exit = _PyObject_LookupSpecial(mgr, &PyId___exit__); + PyObject *exit = _PyObject_LookupSpecial(mgr, &_Py_ID(__exit__)); if (exit == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_Format(tstate, PyExc_TypeError, @@ -6793,19 +6770,25 @@ PyEval_GetBuiltins(void) /* Convenience function to get a builtin from its name */ PyObject * -_PyEval_GetBuiltinId(_Py_Identifier *name) +_PyEval_GetBuiltin(PyObject *name) { PyThreadState *tstate = _PyThreadState_GET(); - PyObject *attr = _PyDict_GetItemIdWithError(PyEval_GetBuiltins(), name); + PyObject *attr = PyDict_GetItemWithError(PyEval_GetBuiltins(), name); if (attr) { Py_INCREF(attr); } else if (!_PyErr_Occurred(tstate)) { - _PyErr_SetObject(tstate, PyExc_AttributeError, _PyUnicode_FromId(name)); + _PyErr_SetObject(tstate, PyExc_AttributeError, name); } return attr; } +PyObject * +_PyEval_GetBuiltinId(_Py_Identifier *name) +{ + return _PyEval_GetBuiltin(_PyUnicode_FromId(name)); +} + PyObject * PyEval_GetLocals(void) { @@ -7047,11 +7030,10 @@ static PyObject * import_name(PyThreadState *tstate, InterpreterFrame *frame, PyObject *name, PyObject *fromlist, PyObject *level) { - _Py_IDENTIFIER(__import__); PyObject *import_func, *res; PyObject* stack[5]; - import_func = _PyDict_GetItemIdWithError(frame->f_builtins, &PyId___import__); + import_func = _PyDict_GetItemWithError(frame->f_builtins, &_Py_ID(__import__)); if (import_func == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_SetString(tstate, PyExc_ImportError, "__import__ not found"); @@ -7098,7 +7080,7 @@ import_from(PyThreadState *tstate, PyObject *v, PyObject *name) /* Issue #17636: in case this failed because of a circular relative import, try to fallback on reading the module directly from sys.modules. */ - pkgname = _PyObject_GetAttrId(v, &PyId___name__); + pkgname = PyObject_GetAttr(v, &_Py_ID(__name__)); if (pkgname == NULL) { goto error; } @@ -7140,8 +7122,7 @@ import_from(PyThreadState *tstate, PyObject *v, PyObject *name) PyErr_SetImportError(errmsg, pkgname, NULL); } else { - _Py_IDENTIFIER(__spec__); - PyObject *spec = _PyObject_GetAttrId(v, &PyId___spec__); + PyObject *spec = PyObject_GetAttr(v, &_Py_ID(__spec__)); const char *fmt = _PyModuleSpec_IsInitializing(spec) ? "cannot import name %R from partially initialized module %R " @@ -7163,17 +7144,15 @@ import_from(PyThreadState *tstate, PyObject *v, PyObject *name) static int import_all_from(PyThreadState *tstate, PyObject *locals, PyObject *v) { - _Py_IDENTIFIER(__all__); - _Py_IDENTIFIER(__dict__); PyObject *all, *dict, *name, *value; int skip_leading_underscores = 0; int pos, err; - if (_PyObject_LookupAttrId(v, &PyId___all__, &all) < 0) { + if (_PyObject_LookupAttr(v, &_Py_ID(__all__), &all) < 0) { return -1; /* Unexpected error */ } if (all == NULL) { - if (_PyObject_LookupAttrId(v, &PyId___dict__, &dict) < 0) { + if (_PyObject_LookupAttr(v, &_Py_ID(__dict__), &dict) < 0) { return -1; } if (dict == NULL) { @@ -7200,7 +7179,7 @@ import_all_from(PyThreadState *tstate, PyObject *locals, PyObject *v) break; } if (!PyUnicode_Check(name)) { - PyObject *modname = _PyObject_GetAttrId(v, &PyId___name__); + PyObject *modname = PyObject_GetAttr(v, &_Py_ID(__name__)); if (modname == NULL) { Py_DECREF(name); err = -1; @@ -7400,14 +7379,13 @@ format_exc_check_arg(PyThreadState *tstate, PyObject *exc, if (exc == PyExc_NameError) { // Include the name in the NameError exceptions to offer suggestions later. - _Py_IDENTIFIER(name); PyObject *type, *value, *traceback; PyErr_Fetch(&type, &value, &traceback); PyErr_NormalizeException(&type, &value, &traceback); if (PyErr_GivenExceptionMatches(value, PyExc_NameError)) { // We do not care if this fails because we are going to restore the // NameError anyway. - (void)_PyObject_SetAttrId(value, &PyId_name, obj); + (void)PyObject_SetAttr(value, &_Py_ID(name), obj); } PyErr_Restore(type, value, traceback); } diff --git a/Python/codecs.c b/Python/codecs.c index 343b6e2d03396..33965f885f706 100644 --- a/Python/codecs.c +++ b/Python/codecs.c @@ -522,7 +522,6 @@ PyObject *PyCodec_Decode(PyObject *object, PyObject * _PyCodec_LookupTextEncoding(const char *encoding, const char *alternate_command) { - _Py_IDENTIFIER(_is_text_encoding); PyObject *codec; PyObject *attr; int is_text_codec; @@ -536,7 +535,7 @@ PyObject * _PyCodec_LookupTextEncoding(const char *encoding, * attribute. */ if (!PyTuple_CheckExact(codec)) { - if (_PyObject_LookupAttrId(codec, &PyId__is_text_encoding, &attr) < 0) { + if (_PyObject_LookupAttr(codec, &_Py_ID(_is_text_encoding), &attr) < 0) { Py_DECREF(codec); return NULL; } diff --git a/Python/compile.c b/Python/compile.c index 3f33f306bc571..5f9bcf1dfd962 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -632,11 +632,9 @@ compiler_unit_free(struct compiler_unit *u) static int compiler_set_qualname(struct compiler *c) { - _Py_static_string(dot, "."); - _Py_static_string(dot_locals, "."); Py_ssize_t stack_size; struct compiler_unit *u = c->u; - PyObject *name, *base, *dot_str, *dot_locals_str; + PyObject *name, *base; base = NULL; stack_size = PyList_GET_SIZE(c->c_stack); @@ -667,11 +665,10 @@ compiler_set_qualname(struct compiler *c) if (!force_global) { if (parent->u_scope_type == COMPILER_SCOPE_FUNCTION || parent->u_scope_type == COMPILER_SCOPE_ASYNC_FUNCTION - || parent->u_scope_type == COMPILER_SCOPE_LAMBDA) { - dot_locals_str = _PyUnicode_FromId(&dot_locals); - if (dot_locals_str == NULL) - return 0; - base = PyUnicode_Concat(parent->u_qualname, dot_locals_str); + || parent->u_scope_type == COMPILER_SCOPE_LAMBDA) + { + base = PyUnicode_Concat(parent->u_qualname, + &_Py_STR(dot_locals)); if (base == NULL) return 0; } @@ -683,12 +680,7 @@ compiler_set_qualname(struct compiler *c) } if (base != NULL) { - dot_str = _PyUnicode_FromId(&dot); - if (dot_str == NULL) { - Py_DECREF(base); - return 0; - } - name = PyUnicode_Concat(base, dot_str); + name = PyUnicode_Concat(base, &_Py_STR(dot)); Py_DECREF(base); if (name == NULL) return 0; @@ -1603,17 +1595,11 @@ compiler_enter_scope(struct compiler *c, identifier name, } if (u->u_ste->ste_needs_class_closure) { /* Cook up an implicit __class__ cell. */ - _Py_IDENTIFIER(__class__); - PyObject *name; int res; assert(u->u_scope_type == COMPILER_SCOPE_CLASS); assert(PyDict_GET_SIZE(u->u_cellvars) == 0); - name = _PyUnicode_FromId(&PyId___class__); - if (!name) { - compiler_unit_free(u); - return 0; - } - res = PyDict_SetItem(u->u_cellvars, name, _PyLong_GetZero()); + res = PyDict_SetItem(u->u_cellvars, &_Py_ID(__class__), + _PyLong_GetZero()); if (res < 0) { compiler_unit_free(u); return 0; @@ -1998,11 +1984,6 @@ compiler_body(struct compiler *c, asdl_stmt_seq *stmts) int i = 0; stmt_ty st; PyObject *docstring; - _Py_IDENTIFIER(__doc__); - PyObject *__doc__ = _PyUnicode_FromId(&PyId___doc__); /* borrowed ref*/ - if (__doc__ == NULL) { - return 0; - } /* Set current line number to the line number of first statement. This way line number for SETUP_ANNOTATIONS will always @@ -2027,7 +2008,7 @@ compiler_body(struct compiler *c, asdl_stmt_seq *stmts) assert(st->kind == Expr_kind); VISIT(c, expr, st->v.Expr.value); UNSET_LOC(c); - if (!compiler_nameop(c, __doc__, Store)) + if (!compiler_nameop(c, &_Py_ID(__doc__), Store)) return 0; } } @@ -2041,12 +2022,8 @@ compiler_mod(struct compiler *c, mod_ty mod) { PyCodeObject *co; int addNone = 1; - _Py_static_string(PyId__module, ""); - PyObject *module = _PyUnicode_FromId(&PyId__module); /* borrowed ref */ - if (module == NULL) { - return 0; - } - if (!compiler_enter_scope(c, module, COMPILER_SCOPE_MODULE, mod, 1)) { + if (!compiler_enter_scope(c, &_Py_STR(anon_module), COMPILER_SCOPE_MODULE, + mod, 1)) { return NULL; } c->u->u_lineno = 1; @@ -2324,7 +2301,6 @@ compiler_visit_annotations(struct compiler *c, arguments_ty args, Return 0 on error, -1 if no annotations pushed, 1 if a annotations is pushed. */ - _Py_IDENTIFIER(return); Py_ssize_t annotations_len = 0; if (!compiler_visit_argannotations(c, args->args, &annotations_len)) @@ -2342,11 +2318,8 @@ compiler_visit_annotations(struct compiler *c, arguments_ty args, args->kwarg->annotation, &annotations_len)) return 0; - identifier return_str = _PyUnicode_FromId(&PyId_return); /* borrowed ref */ - if (return_str == NULL) { - return 0; - } - if (!compiler_visit_argannotation(c, return_str, returns, &annotations_len)) { + if (!compiler_visit_argannotation(c, &_Py_ID(return), returns, + &annotations_len)) { return 0; } @@ -2891,7 +2864,6 @@ compiler_lambda(struct compiler *c, expr_ty e) { PyCodeObject *co; PyObject *qualname; - identifier name; Py_ssize_t funcflags; arguments_ty args = e->v.Lambda.args; assert(e->kind == Lambda_kind); @@ -2899,18 +2871,12 @@ compiler_lambda(struct compiler *c, expr_ty e) if (!compiler_check_debug_args(c, args)) return 0; - _Py_static_string(PyId_lambda, ""); - name = _PyUnicode_FromId(&PyId_lambda); /* borrowed ref */ - if (name == NULL) { - return 0; - } - funcflags = compiler_default_arguments(c, args); if (funcflags == -1) { return 0; } - if (!compiler_enter_scope(c, name, COMPILER_SCOPE_LAMBDA, + if (!compiler_enter_scope(c, &_Py_STR(anon_lambda), COMPILER_SCOPE_LAMBDA, (void *)e, e->lineno)) { return 0; } @@ -3809,12 +3775,6 @@ compiler_from_import(struct compiler *c, stmt_ty s) { Py_ssize_t i, n = asdl_seq_LEN(s->v.ImportFrom.names); PyObject *names; - _Py_static_string(PyId_empty_string, ""); - PyObject *empty_string = _PyUnicode_FromId(&PyId_empty_string); /* borrowed ref */ - - if (empty_string == NULL) { - return 0; - } ADDOP_LOAD_CONST_NEW(c, PyLong_FromLong(s->v.ImportFrom.level)); @@ -3841,7 +3801,7 @@ compiler_from_import(struct compiler *c, stmt_ty s) ADDOP_NAME(c, IMPORT_NAME, s->v.ImportFrom.module, names); } else { - ADDOP_NAME(c, IMPORT_NAME, empty_string, names); + ADDOP_NAME(c, IMPORT_NAME, &_Py_STR(empty), names); } for (i = 0; i < n; i++) { alias_ty alias = (alias_ty)asdl_seq_GET(s->v.ImportFrom.names, i); @@ -5389,13 +5349,8 @@ compiler_comprehension(struct compiler *c, expr_ty e, int type, static int compiler_genexp(struct compiler *c, expr_ty e) { - _Py_static_string(PyId_genexpr, ""); - identifier name = _PyUnicode_FromId(&PyId_genexpr); /* borrowed ref */ - if (name == NULL) { - return 0; - } assert(e->kind == GeneratorExp_kind); - return compiler_comprehension(c, e, COMP_GENEXP, name, + return compiler_comprehension(c, e, COMP_GENEXP, &_Py_STR(anon_genexpr), e->v.GeneratorExp.generators, e->v.GeneratorExp.elt, NULL); } @@ -5403,13 +5358,8 @@ compiler_genexp(struct compiler *c, expr_ty e) static int compiler_listcomp(struct compiler *c, expr_ty e) { - _Py_static_string(PyId_listcomp, ""); - identifier name = _PyUnicode_FromId(&PyId_listcomp); /* borrowed ref */ - if (name == NULL) { - return 0; - } assert(e->kind == ListComp_kind); - return compiler_comprehension(c, e, COMP_LISTCOMP, name, + return compiler_comprehension(c, e, COMP_LISTCOMP, &_Py_STR(anon_listcomp), e->v.ListComp.generators, e->v.ListComp.elt, NULL); } @@ -5417,13 +5367,8 @@ compiler_listcomp(struct compiler *c, expr_ty e) static int compiler_setcomp(struct compiler *c, expr_ty e) { - _Py_static_string(PyId_setcomp, ""); - identifier name = _PyUnicode_FromId(&PyId_setcomp); /* borrowed ref */ - if (name == NULL) { - return 0; - } assert(e->kind == SetComp_kind); - return compiler_comprehension(c, e, COMP_SETCOMP, name, + return compiler_comprehension(c, e, COMP_SETCOMP, &_Py_STR(anon_setcomp), e->v.SetComp.generators, e->v.SetComp.elt, NULL); } @@ -5432,13 +5377,8 @@ compiler_setcomp(struct compiler *c, expr_ty e) static int compiler_dictcomp(struct compiler *c, expr_ty e) { - _Py_static_string(PyId_dictcomp, ""); - identifier name = _PyUnicode_FromId(&PyId_dictcomp); /* borrowed ref */ - if (name == NULL) { - return 0; - } assert(e->kind == DictComp_kind); - return compiler_comprehension(c, e, COMP_DICTCOMP, name, + return compiler_comprehension(c, e, COMP_DICTCOMP, &_Py_STR(anon_dictcomp), e->v.DictComp.generators, e->v.DictComp.key, e->v.DictComp.value); } @@ -5960,12 +5900,6 @@ compiler_annassign(struct compiler *c, stmt_ty s) { expr_ty targ = s->v.AnnAssign.target; PyObject* mangled; - _Py_IDENTIFIER(__annotations__); - /* borrowed ref*/ - PyObject *__annotations__ = _PyUnicode_FromId(&PyId___annotations__); - if (__annotations__ == NULL) { - return 0; - } assert(s->kind == AnnAssign_kind); @@ -5988,7 +5922,7 @@ compiler_annassign(struct compiler *c, stmt_ty s) else { VISIT(c, expr, s->v.AnnAssign.annotation); } - ADDOP_NAME(c, LOAD_NAME, __annotations__, names); + ADDOP_NAME(c, LOAD_NAME, &_Py_ID(__annotations__), names); mangled = _Py_Mangle(c->u->u_private, targ->v.Name.id); ADDOP_LOAD_CONST_NEW(c, mangled); ADDOP(c, STORE_SUBSCR); diff --git a/Python/errors.c b/Python/errors.c index 023234974c47d..10a06a5ae3267 100644 --- a/Python/errors.c +++ b/Python/errors.c @@ -28,12 +28,6 @@ extern char *strerror(int); extern "C" { #endif -_Py_IDENTIFIER(__main__); -_Py_IDENTIFIER(__module__); -_Py_IDENTIFIER(builtins); -_Py_IDENTIFIER(stderr); -_Py_IDENTIFIER(flush); - /* Forward declarations */ static PyObject * _PyErr_FormatV(PyThreadState *tstate, PyObject *exception, @@ -1135,7 +1129,7 @@ PyErr_NewException(const char *name, PyObject *base, PyObject *dict) goto failure; } - int r = _PyDict_ContainsId(dict, &PyId___module__); + int r = PyDict_Contains(dict, &_Py_ID(__module__)); if (r < 0) { goto failure; } @@ -1144,7 +1138,7 @@ PyErr_NewException(const char *name, PyObject *base, PyObject *dict) (Py_ssize_t)(dot-name)); if (modulename == NULL) goto failure; - if (_PyDict_SetItemId(dict, &PyId___module__, modulename) != 0) + if (PyDict_SetItem(dict, &_Py_ID(__module__), modulename) != 0) goto failure; } if (PyTuple_Check(base)) { @@ -1347,7 +1341,7 @@ write_unraisable_exc_file(PyThreadState *tstate, PyObject *exc_type, assert(PyExceptionClass_Check(exc_type)); - PyObject *modulename = _PyObject_GetAttrId(exc_type, &PyId___module__); + PyObject *modulename = PyObject_GetAttr(exc_type, &_Py_ID(__module__)); if (modulename == NULL || !PyUnicode_Check(modulename)) { Py_XDECREF(modulename); _PyErr_Clear(tstate); @@ -1356,8 +1350,8 @@ write_unraisable_exc_file(PyThreadState *tstate, PyObject *exc_type, } } else { - if (!_PyUnicode_EqualToASCIIId(modulename, &PyId_builtins) && - !_PyUnicode_EqualToASCIIId(modulename, &PyId___main__)) { + if (!_PyUnicode_Equal(modulename, &_Py_ID(builtins)) && + !_PyUnicode_Equal(modulename, &_Py_ID(__main__))) { if (PyFile_WriteObject(modulename, file, Py_PRINT_RAW) < 0) { Py_DECREF(modulename); return -1; @@ -1405,7 +1399,7 @@ write_unraisable_exc_file(PyThreadState *tstate, PyObject *exc_type, } /* Explicitly call file.flush() */ - PyObject *res = _PyObject_CallMethodIdNoArgs(file, &PyId_flush); + PyObject *res = _PyObject_CallMethodNoArgs(file, &_Py_ID(flush)); if (!res) { return -1; } @@ -1420,7 +1414,7 @@ write_unraisable_exc(PyThreadState *tstate, PyObject *exc_type, PyObject *exc_value, PyObject *exc_tb, PyObject *err_msg, PyObject *obj) { - PyObject *file = _PySys_GetObjectId(&PyId_stderr); + PyObject *file = _PySys_GetAttr(tstate, &_Py_ID(stderr)); if (file == NULL || file == Py_None) { return 0; } @@ -1524,8 +1518,7 @@ _PyErr_WriteUnraisableMsg(const char *err_msg_str, PyObject *obj) goto error; } - _Py_IDENTIFIER(unraisablehook); - PyObject *hook = _PySys_GetObjectId(&PyId_unraisablehook); + PyObject *hook = _PySys_GetAttr(tstate, &_Py_ID(unraisablehook)); if (hook == NULL) { Py_DECREF(hook_args); goto default_hook; @@ -1600,14 +1593,6 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, int end_lineno, int end_col_offset) { PyObject *exc, *v, *tb, *tmp; - _Py_IDENTIFIER(filename); - _Py_IDENTIFIER(lineno); - _Py_IDENTIFIER(end_lineno); - _Py_IDENTIFIER(msg); - _Py_IDENTIFIER(offset); - _Py_IDENTIFIER(end_offset); - _Py_IDENTIFIER(print_file_and_line); - _Py_IDENTIFIER(text); PyThreadState *tstate = _PyThreadState_GET(); /* add attributes for the line number and filename for the error */ @@ -1619,7 +1604,7 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, if (tmp == NULL) _PyErr_Clear(tstate); else { - if (_PyObject_SetAttrId(v, &PyId_lineno, tmp)) { + if (PyObject_SetAttr(v, &_Py_ID(lineno), tmp)) { _PyErr_Clear(tstate); } Py_DECREF(tmp); @@ -1631,7 +1616,7 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, _PyErr_Clear(tstate); } } - if (_PyObject_SetAttrId(v, &PyId_offset, tmp ? tmp : Py_None)) { + if (PyObject_SetAttr(v, &_Py_ID(offset), tmp ? tmp : Py_None)) { _PyErr_Clear(tstate); } Py_XDECREF(tmp); @@ -1643,7 +1628,7 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, _PyErr_Clear(tstate); } } - if (_PyObject_SetAttrId(v, &PyId_end_lineno, tmp ? tmp : Py_None)) { + if (PyObject_SetAttr(v, &_Py_ID(end_lineno), tmp ? tmp : Py_None)) { _PyErr_Clear(tstate); } Py_XDECREF(tmp); @@ -1655,20 +1640,20 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, _PyErr_Clear(tstate); } } - if (_PyObject_SetAttrId(v, &PyId_end_offset, tmp ? tmp : Py_None)) { + if (PyObject_SetAttr(v, &_Py_ID(end_offset), tmp ? tmp : Py_None)) { _PyErr_Clear(tstate); } Py_XDECREF(tmp); tmp = NULL; if (filename != NULL) { - if (_PyObject_SetAttrId(v, &PyId_filename, filename)) { + if (PyObject_SetAttr(v, &_Py_ID(filename), filename)) { _PyErr_Clear(tstate); } tmp = PyErr_ProgramTextObject(filename, lineno); if (tmp) { - if (_PyObject_SetAttrId(v, &PyId_text, tmp)) { + if (PyObject_SetAttr(v, &_Py_ID(text), tmp)) { _PyErr_Clear(tstate); } Py_DECREF(tmp); @@ -1678,7 +1663,7 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, } } if (exc != PyExc_SyntaxError) { - if (_PyObject_LookupAttrId(v, &PyId_msg, &tmp) < 0) { + if (_PyObject_LookupAttr(v, &_Py_ID(msg), &tmp) < 0) { _PyErr_Clear(tstate); } else if (tmp) { @@ -1687,7 +1672,7 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, else { tmp = PyObject_Str(v); if (tmp) { - if (_PyObject_SetAttrId(v, &PyId_msg, tmp)) { + if (PyObject_SetAttr(v, &_Py_ID(msg), tmp)) { _PyErr_Clear(tstate); } Py_DECREF(tmp); @@ -1696,15 +1681,15 @@ PyErr_SyntaxLocationObjectEx(PyObject *filename, int lineno, int col_offset, _PyErr_Clear(tstate); } } - if (_PyObject_LookupAttrId(v, &PyId_print_file_and_line, &tmp) < 0) { + + if (_PyObject_LookupAttr(v, &_Py_ID(print_file_and_line), &tmp) < 0) { _PyErr_Clear(tstate); } else if (tmp) { Py_DECREF(tmp); } else { - if (_PyObject_SetAttrId(v, &PyId_print_file_and_line, - Py_None)) { + if (PyObject_SetAttr(v, &_Py_ID(print_file_and_line), Py_None)) { _PyErr_Clear(tstate); } } diff --git a/Python/import.c b/Python/import.c index 332db6919f1c2..74f8e1dd4c30d 100644 --- a/Python/import.c +++ b/Python/import.c @@ -36,9 +36,6 @@ extern struct _inittab _PyImport_Inittab[]; struct _inittab *PyImport_Inittab = _PyImport_Inittab; static struct _inittab *inittab_copy = NULL; -_Py_IDENTIFIER(__path__); -_Py_IDENTIFIER(__spec__); - /*[clinic input] module _imp [clinic start generated code]*/ @@ -74,9 +71,7 @@ _PyImportZip_Init(PyThreadState *tstate) } } else { - _Py_IDENTIFIER(zipimporter); - PyObject *zipimporter = _PyObject_GetAttrId(zipimport, - &PyId_zipimporter); + PyObject *zipimporter = PyObject_GetAttr(zipimport, &_Py_ID(zipimporter)); Py_DECREF(zipimport); if (zipimporter == NULL) { _PyErr_Clear(tstate); /* No zipimporter object -- okay */ @@ -345,20 +340,18 @@ import_ensure_initialized(PyInterpreterState *interp, PyObject *mod, PyObject *n { PyObject *spec; - _Py_IDENTIFIER(_lock_unlock_module); - /* Optimization: only call _bootstrap._lock_unlock_module() if __spec__._initializing is true. NOTE: because of this, initializing must be set *before* stuffing the new module in sys.modules. */ - spec = _PyObject_GetAttrId(mod, &PyId___spec__); + spec = PyObject_GetAttr(mod, &_Py_ID(__spec__)); int busy = _PyModuleSpec_IsInitializing(spec); Py_XDECREF(spec); if (busy) { /* Wait until module is done importing. */ - PyObject *value = _PyObject_CallMethodIdOneArg( - interp->importlib, &PyId__lock_unlock_module, name); + PyObject *value = _PyObject_CallMethodOneArg( + interp->importlib, &_Py_ID(_lock_unlock_module), name); if (value == NULL) { return -1; } @@ -710,7 +703,6 @@ PyImport_ExecCodeModuleWithPathnames(const char *name, PyObject *co, } else if (cpathobj != NULL) { PyInterpreterState *interp = _PyInterpreterState_GET(); - _Py_IDENTIFIER(_get_sourcefile); if (interp == NULL) { Py_FatalError("no current interpreter"); @@ -719,8 +711,8 @@ PyImport_ExecCodeModuleWithPathnames(const char *name, PyObject *co, external= PyObject_GetAttrString(interp->importlib, "_bootstrap_external"); if (external != NULL) { - pathobj = _PyObject_CallMethodIdOneArg( - external, &PyId__get_sourcefile, cpathobj); + pathobj = _PyObject_CallMethodOneArg( + external, &_Py_ID(_get_sourcefile), cpathobj); Py_DECREF(external); } if (pathobj == NULL) @@ -740,7 +732,6 @@ PyImport_ExecCodeModuleWithPathnames(const char *name, PyObject *co, static PyObject * module_dict_for_exec(PyThreadState *tstate, PyObject *name) { - _Py_IDENTIFIER(__builtins__); PyObject *m, *d; m = import_add_module(tstate, name); @@ -749,10 +740,9 @@ module_dict_for_exec(PyThreadState *tstate, PyObject *name) /* If the module is being reloaded, we get the old module back and re-use its dict to exec the new code. */ d = PyModule_GetDict(m); - int r = _PyDict_ContainsId(d, &PyId___builtins__); + int r = PyDict_Contains(d, &_Py_ID(__builtins__)); if (r == 0) { - r = _PyDict_SetItemId(d, &PyId___builtins__, - PyEval_GetBuiltins()); + r = PyDict_SetItem(d, &_Py_ID(__builtins__), PyEval_GetBuiltins()); } if (r < 0) { remove_module(tstate, name); @@ -794,7 +784,6 @@ PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, { PyThreadState *tstate = _PyThreadState_GET(); PyObject *d, *external, *res; - _Py_IDENTIFIER(_fix_up_module); d = module_dict_for_exec(tstate, name); if (d == NULL) { @@ -810,9 +799,8 @@ PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, Py_DECREF(d); return NULL; } - res = _PyObject_CallMethodIdObjArgs(external, - &PyId__fix_up_module, - d, name, pathname, cpathname, NULL); + res = PyObject_CallMethodObjArgs(external, &_Py_ID(_fix_up_module), + d, name, pathname, cpathname, NULL); Py_DECREF(external); if (res != NULL) { Py_DECREF(res); @@ -1542,9 +1530,6 @@ remove_importlib_frames(PyThreadState *tstate) static PyObject * resolve_name(PyThreadState *tstate, PyObject *name, PyObject *globals, int level) { - _Py_IDENTIFIER(__package__); - _Py_IDENTIFIER(__name__); - _Py_IDENTIFIER(parent); PyObject *abs_name; PyObject *package = NULL; PyObject *spec; @@ -1560,14 +1545,14 @@ resolve_name(PyThreadState *tstate, PyObject *name, PyObject *globals, int level _PyErr_SetString(tstate, PyExc_TypeError, "globals must be a dict"); goto error; } - package = _PyDict_GetItemIdWithError(globals, &PyId___package__); + package = PyDict_GetItemWithError(globals, &_Py_ID(__package__)); if (package == Py_None) { package = NULL; } else if (package == NULL && _PyErr_Occurred(tstate)) { goto error; } - spec = _PyDict_GetItemIdWithError(globals, &PyId___spec__); + spec = PyDict_GetItemWithError(globals, &_Py_ID(__spec__)); if (spec == NULL && _PyErr_Occurred(tstate)) { goto error; } @@ -1581,7 +1566,7 @@ resolve_name(PyThreadState *tstate, PyObject *name, PyObject *globals, int level } else if (spec != NULL && spec != Py_None) { int equal; - PyObject *parent = _PyObject_GetAttrId(spec, &PyId_parent); + PyObject *parent = PyObject_GetAttr(spec, &_Py_ID(parent)); if (parent == NULL) { goto error; } @@ -1600,7 +1585,7 @@ resolve_name(PyThreadState *tstate, PyObject *name, PyObject *globals, int level } } else if (spec != NULL && spec != Py_None) { - package = _PyObject_GetAttrId(spec, &PyId_parent); + package = PyObject_GetAttr(spec, &_Py_ID(parent)); if (package == NULL) { goto error; } @@ -1617,7 +1602,7 @@ resolve_name(PyThreadState *tstate, PyObject *name, PyObject *globals, int level goto error; } - package = _PyDict_GetItemIdWithError(globals, &PyId___name__); + package = PyDict_GetItemWithError(globals, &_Py_ID(__name__)); if (package == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_SetString(tstate, PyExc_KeyError, @@ -1633,7 +1618,7 @@ resolve_name(PyThreadState *tstate, PyObject *name, PyObject *globals, int level goto error; } - int haspath = _PyDict_ContainsId(globals, &PyId___path__); + int haspath = PyDict_Contains(globals, &_Py_ID(__path__)); if (haspath < 0) { goto error; } @@ -1701,7 +1686,6 @@ resolve_name(PyThreadState *tstate, PyObject *name, PyObject *globals, int level static PyObject * import_find_and_load(PyThreadState *tstate, PyObject *abs_name) { - _Py_IDENTIFIER(_find_and_load); PyObject *mod = NULL; PyInterpreterState *interp = tstate->interp; int import_time = _PyInterpreterState_GetConfig(interp)->import_time; @@ -1742,9 +1726,8 @@ import_find_and_load(PyThreadState *tstate, PyObject *abs_name) if (PyDTrace_IMPORT_FIND_LOAD_START_ENABLED()) PyDTrace_IMPORT_FIND_LOAD_START(PyUnicode_AsUTF8(abs_name)); - mod = _PyObject_CallMethodIdObjArgs(interp->importlib, - &PyId__find_and_load, abs_name, - interp->import_func, NULL); + mod = PyObject_CallMethodObjArgs(interp->importlib, &_Py_ID(_find_and_load), + abs_name, interp->import_func, NULL); if (PyDTrace_IMPORT_FIND_LOAD_DONE_ENABLED()) PyDTrace_IMPORT_FIND_LOAD_DONE(PyUnicode_AsUTF8(abs_name), @@ -1788,7 +1771,6 @@ PyImport_ImportModuleLevelObject(PyObject *name, PyObject *globals, int level) { PyThreadState *tstate = _PyThreadState_GET(); - _Py_IDENTIFIER(_handle_fromlist); PyObject *abs_name = NULL; PyObject *final_mod = NULL; PyObject *mod = NULL; @@ -1909,13 +1891,13 @@ PyImport_ImportModuleLevelObject(PyObject *name, PyObject *globals, } else { PyObject *path; - if (_PyObject_LookupAttrId(mod, &PyId___path__, &path) < 0) { + if (_PyObject_LookupAttr(mod, &_Py_ID(__path__), &path) < 0) { goto error; } if (path) { Py_DECREF(path); - final_mod = _PyObject_CallMethodIdObjArgs( - interp->importlib, &PyId__handle_fromlist, + final_mod = PyObject_CallMethodObjArgs( + interp->importlib, &_Py_ID(_handle_fromlist), mod, fromlist, interp->import_func, NULL); } else { @@ -1955,10 +1937,8 @@ PyImport_ImportModuleLevel(const char *name, PyObject *globals, PyObject *locals PyObject * PyImport_ReloadModule(PyObject *m) { - _Py_IDENTIFIER(importlib); - _Py_IDENTIFIER(reload); PyObject *reloaded_module = NULL; - PyObject *importlib = _PyImport_GetModuleId(&PyId_importlib); + PyObject *importlib = PyImport_GetModule(&_Py_ID(importlib)); if (importlib == NULL) { if (PyErr_Occurred()) { return NULL; @@ -1970,7 +1950,7 @@ PyImport_ReloadModule(PyObject *m) } } - reloaded_module = _PyObject_CallMethodIdOneArg(importlib, &PyId_reload, m); + reloaded_module = PyObject_CallMethodOneArg(importlib, &_Py_ID(reload), m); Py_DECREF(importlib); return reloaded_module; } @@ -1988,26 +1968,12 @@ PyImport_ReloadModule(PyObject *m) PyObject * PyImport_Import(PyObject *module_name) { - _Py_IDENTIFIER(__import__); - _Py_IDENTIFIER(__builtins__); - PyThreadState *tstate = _PyThreadState_GET(); PyObject *globals = NULL; PyObject *import = NULL; PyObject *builtins = NULL; PyObject *r = NULL; - /* Initialize constant string objects */ - PyObject *import_str = _PyUnicode_FromId(&PyId___import__); // borrowed ref - if (import_str == NULL) { - return NULL; - } - - PyObject *builtins_str = _PyUnicode_FromId(&PyId___builtins__); // borrowed ref - if (builtins_str == NULL) { - return NULL; - } - PyObject *from_list = PyList_New(0); if (from_list == NULL) { goto err; @@ -2017,7 +1983,7 @@ PyImport_Import(PyObject *module_name) globals = PyEval_GetGlobals(); if (globals != NULL) { Py_INCREF(globals); - builtins = PyObject_GetItem(globals, builtins_str); + builtins = PyObject_GetItem(globals, &_Py_ID(__builtins__)); if (builtins == NULL) goto err; } @@ -2028,20 +1994,20 @@ PyImport_Import(PyObject *module_name) if (builtins == NULL) { goto err; } - globals = Py_BuildValue("{OO}", builtins_str, builtins); + globals = Py_BuildValue("{OO}", &_Py_ID(__builtins__), builtins); if (globals == NULL) goto err; } /* Get the __import__ function from the builtins */ if (PyDict_Check(builtins)) { - import = PyObject_GetItem(builtins, import_str); + import = PyObject_GetItem(builtins, &_Py_ID(__import__)); if (import == NULL) { - _PyErr_SetObject(tstate, PyExc_KeyError, import_str); + _PyErr_SetObject(tstate, PyExc_KeyError, &_Py_ID(__import__)); } } else - import = PyObject_GetAttr(builtins, import_str); + import = PyObject_GetAttr(builtins, &_Py_ID(__import__)); if (import == NULL) goto err; diff --git a/Python/importdl.c b/Python/importdl.c index 6d2554741f982..f66c6013d2c98 100644 --- a/Python/importdl.c +++ b/Python/importdl.c @@ -2,6 +2,9 @@ /* Support for dynamic loading of extension modules */ #include "Python.h" +#include "pycore_call.h" +#include "pycore_pystate.h" +#include "pycore_runtime.h" /* ./configure sets HAVE_DYNAMIC_LOADING if dynamic loading of modules is supported on this platform. configure will then compile and link in one @@ -38,7 +41,6 @@ get_encoded_name(PyObject *name, const char **hook_prefix) { PyObject *encoded = NULL; PyObject *modname = NULL; Py_ssize_t name_len, lastdot; - _Py_IDENTIFIER(replace); /* Get the short name (substring after last dot) */ name_len = PyUnicode_GetLength(name); @@ -76,7 +78,7 @@ get_encoded_name(PyObject *name, const char **hook_prefix) { } /* Replace '-' by '_' */ - modname = _PyObject_CallMethodId(encoded, &PyId_replace, "cc", '-', '_'); + modname = _PyObject_CallMethod(encoded, &_Py_ID(replace), "cc", '-', '_'); if (modname == NULL) goto error; diff --git a/Python/marshal.c b/Python/marshal.c index 51c77555d9ea9..44e492925cb25 100644 --- a/Python/marshal.c +++ b/Python/marshal.c @@ -703,7 +703,6 @@ r_string(Py_ssize_t n, RFILE *p) read = fread(p->buf, 1, n, p->fp); } else { - _Py_IDENTIFIER(readinto); PyObject *res, *mview; Py_buffer buf; @@ -713,7 +712,7 @@ r_string(Py_ssize_t n, RFILE *p) if (mview == NULL) return NULL; - res = _PyObject_CallMethodId(p->readable, &PyId_readinto, "N", mview); + res = _PyObject_CallMethod(p->readable, &_Py_ID(readinto), "N", mview); if (res != NULL) { read = PyNumber_AsSsize_t(res, PyExc_ValueError); Py_DECREF(res); @@ -1713,12 +1712,11 @@ marshal_dump_impl(PyObject *module, PyObject *value, PyObject *file, /* XXX Quick hack -- need to do this differently */ PyObject *s; PyObject *res; - _Py_IDENTIFIER(write); s = PyMarshal_WriteObjectToString(value, version); if (s == NULL) return NULL; - res = _PyObject_CallMethodIdOneArg(file, &PyId_write, s); + res = _PyObject_CallMethodOneArg(file, &_Py_ID(write), s); Py_DECREF(s); return res; } @@ -1745,7 +1743,6 @@ marshal_load(PyObject *module, PyObject *file) /*[clinic end generated code: output=f8e5c33233566344 input=c85c2b594cd8124a]*/ { PyObject *data, *result; - _Py_IDENTIFIER(read); RFILE rf; /* @@ -1755,7 +1752,7 @@ marshal_load(PyObject *module, PyObject *file) * This can be removed if we guarantee good error handling * for r_string() */ - data = _PyObject_CallMethodId(file, &PyId_read, "i", 0); + data = _PyObject_CallMethod(file, &_Py_ID(read), "i", 0); if (data == NULL) return NULL; if (!PyBytes_Check(data)) { diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index ba7bcd2f8899e..4a3a1abb3a4f0 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -21,9 +21,9 @@ #include "pycore_pylifecycle.h" // _PyErr_Print() #include "pycore_pymem.h" // _PyObject_DebugMallocStats() #include "pycore_pystate.h" // _PyThreadState_GET() +#include "pycore_runtime.h" // _Py_ID() #include "pycore_runtime_init.h" // _PyRuntimeState_INIT #include "pycore_sliceobject.h" // _PySlice_Fini() -#include "pycore_structseq.h" // _PyStructSequence_InitState() #include "pycore_symtable.h" // _PySymtable_Fini() #include "pycore_sysmodule.h" // _PySys_ClearAuditHooks() #include "pycore_traceback.h" // _Py_DumpTracebackThreads() @@ -64,13 +64,6 @@ extern void _PyIO_Fini(void); #define PUTS(fd, str) _Py_write_noraise(fd, str, (int)strlen(str)) -_Py_IDENTIFIER(flush); -_Py_IDENTIFIER(name); -_Py_IDENTIFIER(stdin); -_Py_IDENTIFIER(stdout); -_Py_IDENTIFIER(stderr); -_Py_IDENTIFIER(threading); - #ifdef __cplusplus extern "C" { #endif @@ -704,11 +697,6 @@ pycore_init_types(PyInterpreterState *interp) { PyStatus status; - status = _PyStructSequence_InitState(interp); - if (_PyStatus_EXCEPTION(status)) { - return status; - } - status = _PyTypes_InitState(interp); if (_PyStatus_EXCEPTION(status)) { return status; @@ -1450,8 +1438,7 @@ finalize_clear_modules_dict(PyObject *modules) PyDict_Clear(modules); } else { - _Py_IDENTIFIER(clear); - if (_PyObject_CallMethodIdNoArgs(modules, &PyId_clear) == NULL) { + if (PyObject_CallMethodNoArgs(modules, &_Py_ID(clear)) == NULL) { PyErr_WriteUnraisable(NULL); } } @@ -1622,13 +1609,14 @@ file_is_closed(PyObject *fobj) static int flush_std_files(void) { - PyObject *fout = _PySys_GetObjectId(&PyId_stdout); - PyObject *ferr = _PySys_GetObjectId(&PyId_stderr); + PyThreadState *tstate = _PyThreadState_GET(); + PyObject *fout = _PySys_GetAttr(tstate, &_Py_ID(stdout)); + PyObject *ferr = _PySys_GetAttr(tstate, &_Py_ID(stderr)); PyObject *tmp; int status = 0; if (fout != NULL && fout != Py_None && !file_is_closed(fout)) { - tmp = _PyObject_CallMethodIdNoArgs(fout, &PyId_flush); + tmp = PyObject_CallMethodNoArgs(fout, &_Py_ID(flush)); if (tmp == NULL) { PyErr_WriteUnraisable(fout); status = -1; @@ -1638,7 +1626,7 @@ flush_std_files(void) } if (ferr != NULL && ferr != Py_None && !file_is_closed(ferr)) { - tmp = _PyObject_CallMethodIdNoArgs(ferr, &PyId_flush); + tmp = PyObject_CallMethodNoArgs(ferr, &_Py_ID(flush)); if (tmp == NULL) { PyErr_Clear(); status = -1; @@ -2227,10 +2215,6 @@ create_stdio(const PyConfig *config, PyObject* io, const char* newline; PyObject *line_buffering, *write_through; int buffering, isatty; - _Py_IDENTIFIER(open); - _Py_IDENTIFIER(isatty); - _Py_IDENTIFIER(TextIOWrapper); - _Py_IDENTIFIER(mode); const int buffered_stdio = config->buffered_stdio; if (!is_valid_fd(fd)) @@ -2249,16 +2233,15 @@ create_stdio(const PyConfig *config, PyObject* io, mode = "wb"; else mode = "rb"; - buf = _PyObject_CallMethodId(io, &PyId_open, "isiOOOO", - fd, mode, buffering, - Py_None, Py_None, /* encoding, errors */ - Py_None, Py_False); /* newline, closefd */ + buf = _PyObject_CallMethod(io, &_Py_ID(open), "isiOOOO", + fd, mode, buffering, + Py_None, Py_None, /* encoding, errors */ + Py_None, Py_False); /* newline, closefd */ if (buf == NULL) goto error; if (buffering) { - _Py_IDENTIFIER(raw); - raw = _PyObject_GetAttrId(buf, &PyId_raw); + raw = PyObject_GetAttr(buf, &_Py_ID(raw)); if (raw == NULL) goto error; } @@ -2274,9 +2257,9 @@ create_stdio(const PyConfig *config, PyObject* io, #endif text = PyUnicode_FromString(name); - if (text == NULL || _PyObject_SetAttrId(raw, &PyId_name, text) < 0) + if (text == NULL || PyObject_SetAttr(raw, &_Py_ID(name), text) < 0) goto error; - res = _PyObject_CallMethodIdNoArgs(raw, &PyId_isatty); + res = PyObject_CallMethodNoArgs(raw, &_Py_ID(isatty)); if (res == NULL) goto error; isatty = PyObject_IsTrue(res); @@ -2319,9 +2302,9 @@ create_stdio(const PyConfig *config, PyObject* io, goto error; } - stream = _PyObject_CallMethodId(io, &PyId_TextIOWrapper, "OOOsOO", - buf, encoding_str, errors_str, - newline, line_buffering, write_through); + stream = _PyObject_CallMethod(io, &_Py_ID(TextIOWrapper), "OOOsOO", + buf, encoding_str, errors_str, + newline, line_buffering, write_through); Py_CLEAR(buf); Py_CLEAR(encoding_str); Py_CLEAR(errors_str); @@ -2333,7 +2316,7 @@ create_stdio(const PyConfig *config, PyObject* io, else mode = "r"; text = PyUnicode_FromString(mode); - if (!text || _PyObject_SetAttrId(stream, &PyId_mode, text) < 0) + if (!text || PyObject_SetAttr(stream, &_Py_ID(mode), text) < 0) goto error; Py_CLEAR(text); return stream; @@ -2432,7 +2415,7 @@ init_sys_streams(PyThreadState *tstate) if (std == NULL) goto error; PySys_SetObject("__stdin__", std); - _PySys_SetObjectId(&PyId_stdin, std); + _PySys_SetAttr(&_Py_ID(stdin), std); Py_DECREF(std); /* Set sys.stdout */ @@ -2443,7 +2426,7 @@ init_sys_streams(PyThreadState *tstate) if (std == NULL) goto error; PySys_SetObject("__stdout__", std); - _PySys_SetObjectId(&PyId_stdout, std); + _PySys_SetAttr(&_Py_ID(stdout), std); Py_DECREF(std); #if 1 /* Disable this if you have trouble debugging bootstrap stuff */ @@ -2472,7 +2455,7 @@ init_sys_streams(PyThreadState *tstate) Py_DECREF(std); goto error; } - if (_PySys_SetObjectId(&PyId_stderr, std) < 0) { + if (_PySys_SetAttr(&_Py_ID(stderr), std) < 0) { Py_DECREF(std); goto error; } @@ -2522,7 +2505,7 @@ _Py_FatalError_PrintExc(PyThreadState *tstate) return 0; } - ferr = _PySys_GetObjectId(&PyId_stderr); + ferr = _PySys_GetAttr(tstate, &_Py_ID(stderr)); if (ferr == NULL || ferr == Py_None) { /* sys.stderr is not set yet or set to None, no need to try to display the exception */ @@ -2547,7 +2530,7 @@ _Py_FatalError_PrintExc(PyThreadState *tstate) Py_XDECREF(tb); /* sys.stderr may be buffered: call sys.stderr.flush() */ - res = _PyObject_CallMethodIdNoArgs(ferr, &PyId_flush); + res = PyObject_CallMethodNoArgs(ferr, &_Py_ID(flush)); if (res == NULL) { _PyErr_Clear(tstate); } @@ -2899,9 +2882,8 @@ Py_ExitStatusException(PyStatus status) static void wait_for_thread_shutdown(PyThreadState *tstate) { - _Py_IDENTIFIER(_shutdown); PyObject *result; - PyObject *threading = _PyImport_GetModuleId(&PyId_threading); + PyObject *threading = PyImport_GetModule(&_Py_ID(threading)); if (threading == NULL) { if (_PyErr_Occurred(tstate)) { PyErr_WriteUnraisable(NULL); @@ -2909,7 +2891,7 @@ wait_for_thread_shutdown(PyThreadState *tstate) /* else: threading not imported */ return; } - result = _PyObject_CallMethodIdNoArgs(threading, &PyId__shutdown); + result = PyObject_CallMethodNoArgs(threading, &_Py_ID(_shutdown)); if (result == NULL) { PyErr_WriteUnraisable(threading); } diff --git a/Python/pythonrun.c b/Python/pythonrun.c index 6f170a4dd63eb..b34a22391822b 100644 --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -38,20 +38,6 @@ #endif -_Py_IDENTIFIER(__main__); -_Py_IDENTIFIER(builtins); -_Py_IDENTIFIER(excepthook); -_Py_IDENTIFIER(flush); -_Py_IDENTIFIER(last_traceback); -_Py_IDENTIFIER(last_type); -_Py_IDENTIFIER(last_value); -_Py_IDENTIFIER(ps1); -_Py_IDENTIFIER(ps2); -_Py_IDENTIFIER(stdin); -_Py_IDENTIFIER(stdout); -_Py_IDENTIFIER(stderr); -_Py_static_string(PyId_string, ""); - #ifdef __cplusplus extern "C" { #endif @@ -130,14 +116,15 @@ _PyRun_InteractiveLoopObject(FILE *fp, PyObject *filename, PyCompilerFlags *flag flags = &local_flags; } - PyObject *v = _PySys_GetObjectId(&PyId_ps1); + PyThreadState *tstate = _PyThreadState_GET(); + PyObject *v = _PySys_GetAttr(tstate, &_Py_ID(ps1)); if (v == NULL) { - _PySys_SetObjectId(&PyId_ps1, v = PyUnicode_FromString(">>> ")); + _PySys_SetAttr(&_Py_ID(ps1), v = PyUnicode_FromString(">>> ")); Py_XDECREF(v); } - v = _PySys_GetObjectId(&PyId_ps2); + v = _PySys_GetAttr(tstate, &_Py_ID(ps2)); if (v == NULL) { - _PySys_SetObjectId(&PyId_ps2, v = PyUnicode_FromString("... ")); + _PySys_SetAttr(&_Py_ID(ps2), v = PyUnicode_FromString("... ")); Py_XDECREF(v); } @@ -199,31 +186,25 @@ static int PyRun_InteractiveOneObjectEx(FILE *fp, PyObject *filename, PyCompilerFlags *flags) { - PyObject *m, *d, *v, *w, *oenc = NULL, *mod_name; + PyObject *m, *d, *v, *w, *oenc = NULL; mod_ty mod; PyArena *arena; const char *ps1 = "", *ps2 = "", *enc = NULL; int errcode = 0; - _Py_IDENTIFIER(encoding); - _Py_IDENTIFIER(__main__); - - mod_name = _PyUnicode_FromId(&PyId___main__); /* borrowed */ - if (mod_name == NULL) { - return -1; - } + PyThreadState *tstate = _PyThreadState_GET(); if (fp == stdin) { /* Fetch encoding from sys.stdin if possible. */ - v = _PySys_GetObjectId(&PyId_stdin); + v = _PySys_GetAttr(tstate, &_Py_ID(stdin)); if (v && v != Py_None) { - oenc = _PyObject_GetAttrId(v, &PyId_encoding); + oenc = PyObject_GetAttr(v, &_Py_ID(encoding)); if (oenc) enc = PyUnicode_AsUTF8(oenc); if (!enc) PyErr_Clear(); } } - v = _PySys_GetObjectId(&PyId_ps1); + v = _PySys_GetAttr(tstate, &_Py_ID(ps1)); if (v != NULL) { v = PyObject_Str(v); if (v == NULL) @@ -236,7 +217,7 @@ PyRun_InteractiveOneObjectEx(FILE *fp, PyObject *filename, } } } - w = _PySys_GetObjectId(&PyId_ps2); + w = _PySys_GetAttr(tstate, &_Py_ID(ps2)); if (w != NULL) { w = PyObject_Str(w); if (w == NULL) @@ -271,7 +252,7 @@ PyRun_InteractiveOneObjectEx(FILE *fp, PyObject *filename, } return -1; } - m = PyImport_AddModuleObject(mod_name); + m = PyImport_AddModuleObject(&_Py_ID(__main__)); if (m == NULL) { _PyArena_Free(arena); return -1; @@ -520,37 +501,28 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename, { Py_ssize_t hold; PyObject *v; - _Py_IDENTIFIER(msg); - _Py_IDENTIFIER(filename); - _Py_IDENTIFIER(lineno); - _Py_IDENTIFIER(offset); - _Py_IDENTIFIER(end_lineno); - _Py_IDENTIFIER(end_offset); - _Py_IDENTIFIER(text); *message = NULL; *filename = NULL; /* new style errors. `err' is an instance */ - *message = _PyObject_GetAttrId(err, &PyId_msg); + *message = PyObject_GetAttr(err, &_Py_ID(msg)); if (!*message) goto finally; - v = _PyObject_GetAttrId(err, &PyId_filename); + v = PyObject_GetAttr(err, &_Py_ID(filename)); if (!v) goto finally; if (v == Py_None) { Py_DECREF(v); - *filename = _PyUnicode_FromId(&PyId_string); - if (*filename == NULL) - goto finally; + *filename = &_Py_STR(anon_string); Py_INCREF(*filename); } else { *filename = v; } - v = _PyObject_GetAttrId(err, &PyId_lineno); + v = PyObject_GetAttr(err, &_Py_ID(lineno)); if (!v) goto finally; hold = PyLong_AsSsize_t(v); @@ -559,7 +531,7 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename, goto finally; *lineno = hold; - v = _PyObject_GetAttrId(err, &PyId_offset); + v = PyObject_GetAttr(err, &_Py_ID(offset)); if (!v) goto finally; if (v == Py_None) { @@ -574,7 +546,7 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename, } if (Py_TYPE(err) == (PyTypeObject*)PyExc_SyntaxError) { - v = _PyObject_GetAttrId(err, &PyId_end_lineno); + v = PyObject_GetAttr(err, &_Py_ID(end_lineno)); if (!v) { PyErr_Clear(); *end_lineno = *lineno; @@ -590,7 +562,7 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename, *end_lineno = hold; } - v = _PyObject_GetAttrId(err, &PyId_end_offset); + v = PyObject_GetAttr(err, &_Py_ID(end_offset)); if (!v) { PyErr_Clear(); *end_offset = -1; @@ -611,7 +583,7 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename, *end_offset = -1; } - v = _PyObject_GetAttrId(err, &PyId_text); + v = PyObject_GetAttr(err, &_Py_ID(text)); if (!v) goto finally; if (v == Py_None) { @@ -745,8 +717,7 @@ _Py_HandleSystemExit(int *exitcode_p) if (PyExceptionInstance_Check(value)) { /* The error code should be in the `code' attribute. */ - _Py_IDENTIFIER(code); - PyObject *code = _PyObject_GetAttrId(value, &PyId_code); + PyObject *code = PyObject_GetAttr(value, &_Py_ID(code)); if (code) { Py_DECREF(value); value = code; @@ -761,7 +732,8 @@ _Py_HandleSystemExit(int *exitcode_p) exitcode = (int)PyLong_AsLong(value); } else { - PyObject *sys_stderr = _PySys_GetObjectId(&PyId_stderr); + PyThreadState *tstate = _PyThreadState_GET(); + PyObject *sys_stderr = _PySys_GetAttr(tstate, &_Py_ID(stderr)); /* We clear the exception here to avoid triggering the assertion * in PyObject_Str that ensures it won't silently lose exception * details. @@ -824,17 +796,17 @@ _PyErr_PrintEx(PyThreadState *tstate, int set_sys_last_vars) /* Now we know v != NULL too */ if (set_sys_last_vars) { - if (_PySys_SetObjectId(&PyId_last_type, exception) < 0) { + if (_PySys_SetAttr(&_Py_ID(last_type), exception) < 0) { _PyErr_Clear(tstate); } - if (_PySys_SetObjectId(&PyId_last_value, v) < 0) { + if (_PySys_SetAttr(&_Py_ID(last_value), v) < 0) { _PyErr_Clear(tstate); } - if (_PySys_SetObjectId(&PyId_last_traceback, tb) < 0) { + if (_PySys_SetAttr(&_Py_ID(last_traceback), tb) < 0) { _PyErr_Clear(tstate); } } - hook = _PySys_GetObjectId(&PyId_excepthook); + hook = _PySys_GetAttr(tstate, &_Py_ID(excepthook)); if (_PySys_Audit(tstate, "sys.excepthook", "OOOO", hook ? hook : Py_None, exception, v, tb) < 0) { if (PyErr_ExceptionMatches(PyExc_RuntimeError)) { @@ -979,9 +951,8 @@ print_exception_file_and_line(struct exception_print_context *ctx, { PyObject *f = ctx->file; - _Py_IDENTIFIER(print_file_and_line); PyObject *tmp; - int res = _PyObject_LookupAttrId(*value_p, &PyId_print_file_and_line, &tmp); + int res = _PyObject_LookupAttr(*value_p, &_Py_ID(print_file_and_line), &tmp); if (res <= 0) { if (res < 0) { PyErr_Clear(); @@ -1051,14 +1022,12 @@ print_exception_message(struct exception_print_context *ctx, PyObject *type, { PyObject *f = ctx->file; - _Py_IDENTIFIER(__module__); - assert(PyExceptionClass_Check(type)); if (write_indented_margin(ctx, f) < 0) { return -1; } - PyObject *modulename = _PyObject_GetAttrId(type, &PyId___module__); + PyObject *modulename = PyObject_GetAttr(type, &_Py_ID(__module__)); if (modulename == NULL || !PyUnicode_Check(modulename)) { Py_XDECREF(modulename); PyErr_Clear(); @@ -1067,8 +1036,8 @@ print_exception_message(struct exception_print_context *ctx, PyObject *type, } } else { - if (!_PyUnicode_EqualToASCIIId(modulename, &PyId_builtins) && - !_PyUnicode_EqualToASCIIId(modulename, &PyId___main__)) + if (!_PyUnicode_Equal(modulename, &_Py_ID(builtins)) && + !_PyUnicode_Equal(modulename, &_Py_ID(__main__))) { int res = PyFile_WriteObject(modulename, f, Py_PRINT_RAW); Py_DECREF(modulename); @@ -1168,9 +1137,7 @@ print_exception_note(struct exception_print_context *ctx, PyObject *value) return 0; } - _Py_IDENTIFIER(__note__); - - PyObject *note = _PyObject_GetAttrId(value, &PyId___note__); + PyObject *note = PyObject_GetAttr(value, &_Py_ID(__note__)); if (note == NULL) { return -1; } @@ -1549,11 +1516,13 @@ _PyErr_Display(PyObject *file, PyObject *exception, PyObject *value, PyObject *t } if (print_exception_recursive(&ctx, value) < 0) { PyErr_Clear(); + _PyObject_Dump(value); + fprintf(stderr, "lost sys.stderr\n"); } Py_XDECREF(ctx.seen); /* Call file.flush() */ - PyObject *res = _PyObject_CallMethodIdNoArgs(file, &PyId_flush); + PyObject *res = _PyObject_CallMethodNoArgs(file, &_Py_ID(flush)); if (!res) { /* Silently ignore file.flush() error */ PyErr_Clear(); @@ -1566,7 +1535,8 @@ _PyErr_Display(PyObject *file, PyObject *exception, PyObject *value, PyObject *t void PyErr_Display(PyObject *exception, PyObject *value, PyObject *tb) { - PyObject *file = _PySys_GetObjectId(&PyId_stderr); + PyThreadState *tstate = _PyThreadState_GET(); + PyObject *file = _PySys_GetAttr(tstate, &_Py_ID(stderr)); if (file == NULL) { _PyObject_Dump(value); fprintf(stderr, "lost sys.stderr\n"); @@ -1587,20 +1557,16 @@ PyRun_StringFlags(const char *str, int start, PyObject *globals, PyObject *ret = NULL; mod_ty mod; PyArena *arena; - PyObject *filename; - - filename = _PyUnicode_FromId(&PyId_string); /* borrowed */ - if (filename == NULL) - return NULL; arena = _PyArena_New(); if (arena == NULL) return NULL; - mod = _PyParser_ASTFromString(str, filename, start, flags, arena); + mod = _PyParser_ASTFromString( + str, &_Py_STR(anon_string), start, flags, arena); if (mod != NULL) - ret = run_mod(mod, filename, globals, locals, flags, arena); + ret = run_mod(mod, &_Py_STR(anon_string), globals, locals, flags, arena); _PyArena_Free(arena); return ret; } @@ -1662,17 +1628,18 @@ flush_io(void) /* Save the current exception */ PyErr_Fetch(&type, &value, &traceback); - f = _PySys_GetObjectId(&PyId_stderr); + PyThreadState *tstate = _PyThreadState_GET(); + f = _PySys_GetAttr(tstate, &_Py_ID(stderr)); if (f != NULL) { - r = _PyObject_CallMethodIdNoArgs(f, &PyId_flush); + r = _PyObject_CallMethodNoArgs(f, &_Py_ID(flush)); if (r) Py_DECREF(r); else PyErr_Clear(); } - f = _PySys_GetObjectId(&PyId_stdout); + f = _PySys_GetAttr(tstate, &_Py_ID(stdout)); if (f != NULL) { - r = _PyObject_CallMethodIdNoArgs(f, &PyId_flush); + r = _PyObject_CallMethodNoArgs(f, &_Py_ID(flush)); if (r) Py_DECREF(r); else diff --git a/Python/specialize.c b/Python/specialize.c index b95bdf73e8bf6..21759231d28e6 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -1,6 +1,7 @@ #include "Python.h" #include "pycore_code.h" #include "pycore_dict.h" +#include "pycore_global_strings.h" // _Py_ID() #include "pycore_long.h" #include "pycore_moduleobject.h" #include "pycore_object.h" @@ -596,8 +597,6 @@ specialize_module_load_attr( { PyModuleObject *m = (PyModuleObject *)owner; PyObject *value = NULL; - PyObject *getattr; - _Py_IDENTIFIER(__getattr__); assert((owner->ob_type->tp_flags & Py_TPFLAGS_MANAGED_DICT) == 0); PyDictObject *dict = (PyDictObject *)m->md_dict; if (dict == NULL) { @@ -608,13 +607,8 @@ specialize_module_load_attr( SPECIALIZATION_FAIL(opcode, SPEC_FAIL_NON_STRING_OR_SPLIT); return -1; } - getattr = _PyUnicode_FromId(&PyId___getattr__); /* borrowed */ - if (getattr == NULL) { - SPECIALIZATION_FAIL(opcode, SPEC_FAIL_OVERRIDDEN); - PyErr_Clear(); - return -1; - } - Py_ssize_t index = _PyDict_GetItemHint(dict, getattr, -1, &value); + Py_ssize_t index = _PyDict_GetItemHint(dict, &_Py_ID(__getattr__), -1, + &value); assert(index != DKIX_ERROR); if (index != DKIX_EMPTY) { SPECIALIZATION_FAIL(opcode, SPEC_FAIL_MODULE_ATTR_NOT_FOUND); @@ -1223,7 +1217,6 @@ binary_subscr_fail_kind(PyTypeObject *container_type, PyObject *sub) } #endif -_Py_IDENTIFIER(__getitem__); #define SIMPLE_FUNCTION 0 @@ -1268,7 +1261,7 @@ _Py_Specialize_BinarySubscr( goto success; } PyTypeObject *cls = Py_TYPE(container); - PyObject *descriptor = _PyType_LookupId(cls, &PyId___getitem__); + PyObject *descriptor = _PyType_Lookup(cls, &_Py_ID(__getitem__)); if (descriptor && Py_TYPE(descriptor) == &PyFunction_Type) { PyFunctionObject *func = (PyFunctionObject *)descriptor; PyCodeObject *code = (PyCodeObject *)func->func_code; @@ -1385,8 +1378,7 @@ _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *ins } goto fail; } - _Py_IDENTIFIER(__setitem__); - PyObject *descriptor = _PyType_LookupId(container_type, &PyId___setitem__); + PyObject *descriptor = _PyType_Lookup(container_type, &_Py_ID(__setitem__)); if (descriptor && Py_TYPE(descriptor) == &PyFunction_Type) { PyFunctionObject *func = (PyFunctionObject *)descriptor; PyCodeObject *code = (PyCodeObject *)func->func_code; @@ -1474,7 +1466,6 @@ builtin_call_fail_kind(int ml_flags) #endif static PyMethodDescrObject *_list_append = NULL; -_Py_IDENTIFIER(append); static int specialize_method_descriptor( @@ -1486,7 +1477,8 @@ specialize_method_descriptor( return -1; } if (_list_append == NULL) { - _list_append = (PyMethodDescrObject *)_PyType_LookupId(&PyList_Type, &PyId_append); + _list_append = (PyMethodDescrObject *)_PyType_Lookup(&PyList_Type, + &_Py_ID(append)); } assert(_list_append != NULL); if (nargs == 2 && descr == _list_append) { diff --git a/Python/sysmodule.c b/Python/sysmodule.c index acb03781a4f14..57bf04dd306ae 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -55,17 +55,21 @@ module sys #include "clinic/sysmodule.c.h" -_Py_IDENTIFIER(_); -_Py_IDENTIFIER(__sizeof__); -_Py_IDENTIFIER(_xoptions); -_Py_IDENTIFIER(buffer); -_Py_IDENTIFIER(builtins); -_Py_IDENTIFIER(encoding); -_Py_IDENTIFIER(path); -_Py_IDENTIFIER(stdout); -_Py_IDENTIFIER(stderr); -_Py_IDENTIFIER(warnoptions); -_Py_IDENTIFIER(write); +PyObject * +_PySys_GetAttr(PyThreadState *tstate, PyObject *name) +{ + PyObject *sd = tstate->interp->sysdict; + if (sd == NULL) { + return NULL; + } + PyObject *exc_type, *exc_value, *exc_tb; + _PyErr_Fetch(tstate, &exc_type, &exc_value, &exc_tb); + /* XXX Suppress a new exception if it was raised and restore + * the old one. */ + PyObject *value = _PyDict_GetItemWithError(sd, name); + _PyErr_Restore(tstate, exc_type, exc_value, exc_tb); + return value; +} static PyObject * sys_get_object_id(PyThreadState *tstate, _Py_Identifier *key) @@ -147,6 +151,13 @@ _PySys_SetObjectId(_Py_Identifier *key, PyObject *v) return sys_set_object_id(interp, key, v); } +int +_PySys_SetAttr(PyObject *key, PyObject *v) +{ + PyInterpreterState *interp = _PyInterpreterState_GET(); + return sys_set_object(interp, key, v); +} + static int sys_set_object_str(PyInterpreterState *interp, const char *name, PyObject *v) { @@ -258,9 +269,8 @@ sys_audit_tstate(PyThreadState *ts, const char *event, /* Disallow tracing in hooks unless explicitly enabled */ PyThreadState_EnterTracing(ts); while ((hook = PyIter_Next(hooks)) != NULL) { - _Py_IDENTIFIER(__cantrace__); PyObject *o; - int canTrace = _PyObject_LookupAttrId(hook, &PyId___cantrace__, &o); + int canTrace = _PyObject_LookupAttr(hook, &_Py_ID(__cantrace__), &o); if (o) { canTrace = PyObject_IsTrue(o); Py_DECREF(o); @@ -631,7 +641,7 @@ sys_displayhook_unencodable(PyObject *outf, PyObject *o) const char *stdout_encoding_str; int ret; - stdout_encoding = _PyObject_GetAttrId(outf, &PyId_encoding); + stdout_encoding = PyObject_GetAttr(outf, &_Py_ID(encoding)); if (stdout_encoding == NULL) goto error; stdout_encoding_str = PyUnicode_AsUTF8(stdout_encoding); @@ -648,12 +658,12 @@ sys_displayhook_unencodable(PyObject *outf, PyObject *o) if (encoded == NULL) goto error; - if (_PyObject_LookupAttrId(outf, &PyId_buffer, &buffer) < 0) { + if (_PyObject_LookupAttr(outf, &_Py_ID(buffer), &buffer) < 0) { Py_DECREF(encoded); goto error; } if (buffer) { - result = _PyObject_CallMethodIdOneArg(buffer, &PyId_write, encoded); + result = PyObject_CallMethodOneArg(buffer, &_Py_ID(write), encoded); Py_DECREF(buffer); Py_DECREF(encoded); if (result == NULL) @@ -699,7 +709,7 @@ sys_displayhook(PyObject *module, PyObject *o) static PyObject *newline = NULL; PyThreadState *tstate = _PyThreadState_GET(); - builtins = _PyImport_GetModuleId(&PyId_builtins); + builtins = PyImport_GetModule(&_Py_ID(builtins)); if (builtins == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_SetString(tstate, PyExc_RuntimeError, @@ -715,9 +725,9 @@ sys_displayhook(PyObject *module, PyObject *o) if (o == Py_None) { Py_RETURN_NONE; } - if (_PyObject_SetAttrId(builtins, &PyId__, Py_None) != 0) + if (PyObject_SetAttr(builtins, &_Py_ID(_), Py_None) != 0) return NULL; - outf = sys_get_object_id(tstate, &PyId_stdout); + outf = _PySys_GetAttr(tstate, &_Py_ID(stdout)); if (outf == NULL || outf == Py_None) { _PyErr_SetString(tstate, PyExc_RuntimeError, "lost sys.stdout"); return NULL; @@ -744,7 +754,7 @@ sys_displayhook(PyObject *module, PyObject *o) } if (PyFile_WriteObject(newline, outf, Py_PRINT_RAW) != 0) return NULL; - if (_PyObject_SetAttrId(builtins, &PyId__, o) != 0) + if (PyObject_SetAttr(builtins, &_Py_ID(_), o) != 0) return NULL; Py_RETURN_NONE; } @@ -1676,7 +1686,7 @@ _PySys_GetSizeOf(PyObject *o) return (size_t)-1; } - method = _PyObject_LookupSpecial(o, &PyId___sizeof__); + method = _PyObject_LookupSpecial(o, &_Py_ID(__sizeof__)); if (method == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_Format(tstate, PyExc_TypeError, @@ -2218,7 +2228,7 @@ _PySys_ReadPreinitXOptions(PyConfig *config) static PyObject * get_warnoptions(PyThreadState *tstate) { - PyObject *warnoptions = sys_get_object_id(tstate, &PyId_warnoptions); + PyObject *warnoptions = _PySys_GetAttr(tstate, &_Py_ID(warnoptions)); if (warnoptions == NULL || !PyList_Check(warnoptions)) { /* PEP432 TODO: we can reach this if warnoptions is NULL in the main * interpreter config. When that happens, we need to properly set @@ -2234,7 +2244,7 @@ get_warnoptions(PyThreadState *tstate) if (warnoptions == NULL) { return NULL; } - if (sys_set_object_id(tstate->interp, &PyId_warnoptions, warnoptions)) { + if (sys_set_object(tstate->interp, &_Py_ID(warnoptions), warnoptions)) { Py_DECREF(warnoptions); return NULL; } @@ -2252,7 +2262,7 @@ PySys_ResetWarnOptions(void) return; } - PyObject *warnoptions = sys_get_object_id(tstate, &PyId_warnoptions); + PyObject *warnoptions = _PySys_GetAttr(tstate, &_Py_ID(warnoptions)); if (warnoptions == NULL || !PyList_Check(warnoptions)) return; PyList_SetSlice(warnoptions, 0, PyList_GET_SIZE(warnoptions), NULL); @@ -2306,7 +2316,7 @@ int PySys_HasWarnOptions(void) { PyThreadState *tstate = _PyThreadState_GET(); - PyObject *warnoptions = sys_get_object_id(tstate, &PyId_warnoptions); + PyObject *warnoptions = _PySys_GetAttr(tstate, &_Py_ID(warnoptions)); return (warnoptions != NULL && PyList_Check(warnoptions) && PyList_GET_SIZE(warnoptions) > 0); } @@ -2314,7 +2324,7 @@ PySys_HasWarnOptions(void) static PyObject * get_xoptions(PyThreadState *tstate) { - PyObject *xoptions = sys_get_object_id(tstate, &PyId__xoptions); + PyObject *xoptions = _PySys_GetAttr(tstate, &_Py_ID(_xoptions)); if (xoptions == NULL || !PyDict_Check(xoptions)) { /* PEP432 TODO: we can reach this if xoptions is NULL in the main * interpreter config. When that happens, we need to properly set @@ -2330,7 +2340,7 @@ get_xoptions(PyThreadState *tstate) if (xoptions == NULL) { return NULL; } - if (sys_set_object_id(tstate->interp, &PyId__xoptions, xoptions)) { + if (sys_set_object(tstate->interp, &_Py_ID(_xoptions), xoptions)) { Py_DECREF(xoptions); return NULL; } @@ -3032,7 +3042,7 @@ _PySys_SetPreliminaryStderr(PyObject *sysdict) if (pstderr == NULL) { goto error; } - if (_PyDict_SetItemId(sysdict, &PyId_stderr, pstderr) < 0) { + if (PyDict_SetItem(sysdict, &_Py_ID(stderr), pstderr) < 0) { goto error; } if (PyDict_SetItemString(sysdict, "__stderr__", pstderr) < 0) { @@ -3157,7 +3167,7 @@ PySys_SetPath(const wchar_t *path) if ((v = makepathobject(path, DELIM)) == NULL) Py_FatalError("can't create sys.path"); PyInterpreterState *interp = _PyInterpreterState_GET(); - if (sys_set_object_id(interp, &PyId_path, v) != 0) { + if (sys_set_object(interp, &_Py_ID(path), v) != 0) { Py_FatalError("can't assign sys.path"); } Py_DECREF(v); @@ -3214,7 +3224,7 @@ PySys_SetArgvEx(int argc, wchar_t **argv, int updatepath) Py_FatalError("can't compute path0 from argv"); } - PyObject *sys_path = sys_get_object_id(tstate, &PyId_path); + PyObject *sys_path = _PySys_GetAttr(tstate, &_Py_ID(path)); if (sys_path != NULL) { if (PyList_Insert(sys_path, 0, path0) < 0) { Py_DECREF(path0); @@ -3241,7 +3251,7 @@ sys_pyfile_write_unicode(PyObject *unicode, PyObject *file) if (file == NULL) return -1; assert(unicode != NULL); - PyObject *result = _PyObject_CallMethodIdOneArg(file, &PyId_write, unicode); + PyObject *result = _PyObject_CallMethodOneArg(file, &_Py_ID(write), unicode); if (result == NULL) { return -1; } @@ -3296,7 +3306,7 @@ sys_pyfile_write(const char *text, PyObject *file) */ static void -sys_write(_Py_Identifier *key, FILE *fp, const char *format, va_list va) +sys_write(PyObject *key, FILE *fp, const char *format, va_list va) { PyObject *file; PyObject *error_type, *error_value, *error_traceback; @@ -3305,7 +3315,7 @@ sys_write(_Py_Identifier *key, FILE *fp, const char *format, va_list va) PyThreadState *tstate = _PyThreadState_GET(); _PyErr_Fetch(tstate, &error_type, &error_value, &error_traceback); - file = sys_get_object_id(tstate, key); + file = _PySys_GetAttr(tstate, key); written = PyOS_vsnprintf(buffer, sizeof(buffer), format, va); if (sys_pyfile_write(buffer, file) != 0) { _PyErr_Clear(tstate); @@ -3325,7 +3335,7 @@ PySys_WriteStdout(const char *format, ...) va_list va; va_start(va, format); - sys_write(&PyId_stdout, stdout, format, va); + sys_write(&_Py_ID(stdout), stdout, format, va); va_end(va); } @@ -3335,12 +3345,12 @@ PySys_WriteStderr(const char *format, ...) va_list va; va_start(va, format); - sys_write(&PyId_stderr, stderr, format, va); + sys_write(&_Py_ID(stderr), stderr, format, va); va_end(va); } static void -sys_format(_Py_Identifier *key, FILE *fp, const char *format, va_list va) +sys_format(PyObject *key, FILE *fp, const char *format, va_list va) { PyObject *file, *message; PyObject *error_type, *error_value, *error_traceback; @@ -3348,7 +3358,7 @@ sys_format(_Py_Identifier *key, FILE *fp, const char *format, va_list va) PyThreadState *tstate = _PyThreadState_GET(); _PyErr_Fetch(tstate, &error_type, &error_value, &error_traceback); - file = sys_get_object_id(tstate, key); + file = _PySys_GetAttr(tstate, key); message = PyUnicode_FromFormatV(format, va); if (message != NULL) { if (sys_pyfile_write_unicode(message, file) != 0) { @@ -3368,7 +3378,7 @@ PySys_FormatStdout(const char *format, ...) va_list va; va_start(va, format); - sys_format(&PyId_stdout, stdout, format, va); + sys_format(&_Py_ID(stdout), stdout, format, va); va_end(va); } @@ -3378,6 +3388,6 @@ PySys_FormatStderr(const char *format, ...) va_list va; va_start(va, format); - sys_format(&PyId_stderr, stderr, format, va); + sys_format(&_Py_ID(stderr), stderr, format, va); va_end(va); } diff --git a/Python/traceback.c b/Python/traceback.c index 4d6cbaae8da6c..f2ae38e56c330 100644 --- a/Python/traceback.c +++ b/Python/traceback.c @@ -6,6 +6,7 @@ #include "code.h" // PyCode_Addr2Line etc #include "frameobject.h" // PyFrame_GetBack() #include "pycore_ast.h" // asdl_seq_* +#include "pycore_call.h" // _PyObject_CallMethodFormat() #include "pycore_compile.h" // _PyAST_Optimize #include "pycore_fileutils.h" // _Py_BEGIN_SUPPRESS_IPH #include "pycore_frame.h" // _PyFrame_GetCode() @@ -32,11 +33,6 @@ /* Function from Parser/tokenizer.c */ extern char* _PyTokenizer_FindEncodingFilename(int, PyObject *); -_Py_IDENTIFIER(TextIOWrapper); -_Py_IDENTIFIER(close); -_Py_IDENTIFIER(open); -_Py_IDENTIFIER(path); - /*[clinic input] class TracebackType "PyTracebackObject *" "&PyTraceback_Type" [clinic start generated code]*/ @@ -317,6 +313,7 @@ _Py_FindSourceFile(PyObject *filename, char* namebuf, size_t namelen, PyObject * const char* filepath; Py_ssize_t len; PyObject* result; + PyObject *open = NULL; filebytes = PyUnicode_EncodeFSDefault(filename); if (filebytes == NULL) { @@ -333,11 +330,13 @@ _Py_FindSourceFile(PyObject *filename, char* namebuf, size_t namelen, PyObject * tail++; taillen = strlen(tail); - syspath = _PySys_GetObjectId(&PyId_path); + PyThreadState *tstate = _PyThreadState_GET(); + syspath = _PySys_GetAttr(tstate, &_Py_ID(path)); if (syspath == NULL || !PyList_Check(syspath)) goto error; npath = PyList_Size(syspath); + open = PyObject_GetAttr(io, &_Py_ID(open)); for (i = 0; i < npath; i++) { v = PyList_GetItem(syspath, i); if (v == NULL) { @@ -364,7 +363,7 @@ _Py_FindSourceFile(PyObject *filename, char* namebuf, size_t namelen, PyObject * namebuf[len++] = SEP; strcpy(namebuf+len, tail); - binary = _PyObject_CallMethodId(io, &PyId_open, "ss", namebuf, "rb"); + binary = _PyObject_CallMethodFormat(tstate, open, "ss", namebuf, "rb"); if (binary != NULL) { result = binary; goto finally; @@ -376,6 +375,7 @@ _Py_FindSourceFile(PyObject *filename, char* namebuf, size_t namelen, PyObject * error: result = NULL; finally: + Py_XDECREF(open); Py_DECREF(filebytes); return result; } @@ -448,10 +448,11 @@ display_source_line_with_margin(PyObject *f, PyObject *filename, int lineno, int } io = PyImport_ImportModule("io"); - if (io == NULL) + if (io == NULL) { return -1; - binary = _PyObject_CallMethodId(io, &PyId_open, "Os", filename, "rb"); + } + binary = _PyObject_CallMethod(io, &_Py_ID(open), "Os", filename, "rb"); if (binary == NULL) { PyErr_Clear(); @@ -480,14 +481,15 @@ display_source_line_with_margin(PyObject *f, PyObject *filename, int lineno, int PyMem_Free(found_encoding); return 0; } - fob = _PyObject_CallMethodId(io, &PyId_TextIOWrapper, "Os", binary, encoding); + fob = _PyObject_CallMethod(io, &_Py_ID(TextIOWrapper), + "Os", binary, encoding); Py_DECREF(io); PyMem_Free(found_encoding); if (fob == NULL) { PyErr_Clear(); - res = _PyObject_CallMethodIdNoArgs(binary, &PyId_close); + res = PyObject_CallMethodNoArgs(binary, &_Py_ID(close)); Py_DECREF(binary); if (res) Py_DECREF(res); @@ -506,7 +508,7 @@ display_source_line_with_margin(PyObject *f, PyObject *filename, int lineno, int break; } } - res = _PyObject_CallMethodIdNoArgs(fob, &PyId_close); + res = PyObject_CallMethodNoArgs(fob, &_Py_ID(close)); if (res) { Py_DECREF(res); } diff --git a/Tools/c-analyzer/TODO b/Tools/c-analyzer/TODO index 4b9b2857e1d1e..55338ebc855d0 100644 --- a/Tools/c-analyzer/TODO +++ b/Tools/c-analyzer/TODO @@ -156,8 +156,6 @@ Objects/codeobject.c:PyCode_NewEmpty():emptystring static PyObject Objects/exceptions.c:_check_for_legacy_statements():exec_prefix static PyObject *exec_prefix Objects/exceptions.c:_check_for_legacy_statements():print_prefix static PyObject *print_prefix Objects/funcobject.c:PyFunction_NewWithQualName():__name__ static PyObject *__name__ -Objects/typeobject.c:object___reduce_ex___impl():objreduce static PyObject *objreduce -Objects/typeobject.c:resolve_slotdups():pname static PyObject *pname Objects/unicodeobject.c:unicode_empty static PyObject *unicode_empty Objects/unicodeobject.c:unicode_latin1 static PyObject *unicode_latin1[256] Python/_warnings.c:is_internal_frame():bootstrap_string static PyObject *bootstrap_string diff --git a/Tools/scripts/generate_global_objects.py b/Tools/scripts/generate_global_objects.py index a06d201fd6144..b184e74898940 100644 --- a/Tools/scripts/generate_global_objects.py +++ b/Tools/scripts/generate_global_objects.py @@ -1,9 +1,7 @@ -import argparse -import ast -import builtins -import collections import contextlib +import glob import os.path +import re import sys @@ -12,6 +10,298 @@ INTERNAL = os.path.join(ROOT, 'Include', 'internal') +STRING_LITERALS = { + 'empty': '', + 'dot': '.', + 'comma_sep': ', ', + 'percent': '%', + 'dbl_percent': '%%', + + '"anonymous" labels': None, + 'anon_dictcomp': '', + 'anon_genexpr': '', + 'anon_lambda': '', + 'anon_listcomp': '', + 'anon_module': '', + 'anon_setcomp': '', + 'anon_string': '', + 'dot_locals': '.', +} +IDENTIFIERS = [ + 'Py_Repr', + 'TextIOWrapper', + 'WarningMessage', + '_', + '__IOBase_closed', + '__abc_tpflags__', + '__abs__', + '__abstractmethods__', + '__add__', + '__aenter__', + '__aexit__', + '__aiter__', + '__all__', + '__and__', + '__anext__', + '__annotations__', + '__args__', + '__await__', + '__bases__', + '__bool__', + '__build_class__', + '__builtins__', + '__bytes__', + '__call__', + '__cantrace__', + '__class__', + '__class_getitem__', + '__classcell__', + '__complex__', + '__contains__', + '__copy__', + '__del__', + '__delattr__', + '__delete__', + '__delitem__', + '__dict__', + '__dir__', + '__divmod__', + '__doc__', + '__enter__', + '__eq__', + '__exit__', + '__file__', + '__float__', + '__floordiv__', + '__format__', + '__fspath__', + '__ge__', + '__get__', + '__getattr__', + '__getattribute__', + '__getinitargs__', + '__getitem__', + '__getnewargs__', + '__getnewargs_ex__', + '__getstate__', + '__gt__', + '__hash__', + '__iadd__', + '__iand__', + '__ifloordiv__', + '__ilshift__', + '__imatmul__', + '__imod__', + '__import__', + '__imul__', + '__index__', + '__init__', + '__init_subclass__', + '__instancecheck__', + '__int__', + '__invert__', + '__ior__', + '__ipow__', + '__irshift__', + '__isabstractmethod__', + '__isub__', + '__iter__', + '__itruediv__', + '__ixor__', + '__le__', + '__len__', + '__length_hint__', + '__loader__', + '__lshift__', + '__lt__', + '__ltrace__', + '__main__', + '__matmul__', + '__missing__', + '__mod__', + '__module__', + '__mro_entries__', + '__mul__', + '__name__', + '__ne__', + '__neg__', + '__new__', + '__newobj__', + '__newobj_ex__', + '__next__', + '__note__', + '__or__', + '__origin__', + '__package__', + '__parameters__', + '__path__', + '__pos__', + '__pow__', + '__prepare__', + '__qualname__', + '__radd__', + '__rand__', + '__rdivmod__', + '__reduce__', + '__reduce_ex__', + '__repr__', + '__reversed__', + '__rfloordiv__', + '__rlshift__', + '__rmatmul__', + '__rmod__', + '__rmul__', + '__ror__', + '__round__', + '__rpow__', + '__rrshift__', + '__rshift__', + '__rsub__', + '__rtruediv__', + '__rxor__', + '__set__', + '__set_name__', + '__setattr__', + '__setitem__', + '__setstate__', + '__sizeof__', + '__slotnames__', + '__slots__', + '__spec__', + '__str__', + '__sub__', + '__subclasscheck__', + '__subclasshook__', + '__truediv__', + '__trunc__', + '__warningregistry__', + '__weakref__', + '__xor__', + '_abc_impl', + '_blksize', + '_dealloc_warn', + '_finalizing', + '_find_and_load', + '_fix_up_module', + '_get_sourcefile', + '_handle_fromlist', + '_initializing', + '_is_text_encoding', + '_lock_unlock_module', + '_showwarnmsg', + '_shutdown', + '_slotnames', + '_strptime_time', + '_uninitialized_submodules', + '_warn_unawaited_coroutine', + '_xoptions', + 'add', + 'append', + 'big', + 'buffer', + 'builtins', + 'clear', + 'close', + 'code', + 'copy', + 'copyreg', + 'decode', + 'default', + 'defaultaction', + 'difference_update', + 'dispatch_table', + 'displayhook', + 'enable', + 'encoding', + 'end_lineno', + 'end_offset', + 'errors', + 'excepthook', + 'extend', + 'filename', + 'fileno', + 'fillvalue', + 'filters', + 'find_class', + 'flush', + 'get', + 'get_source', + 'getattr', + 'ignore', + 'importlib', + 'intersection', + 'isatty', + 'items', + 'iter', + 'keys', + 'last_traceback', + 'last_type', + 'last_value', + 'latin1', + 'lineno', + 'little', + 'match', + 'metaclass', + 'mode', + 'modules', + 'mro', + 'msg', + 'n_fields', + 'n_sequence_fields', + 'n_unnamed_fields', + 'name', + 'obj', + 'offset', + 'onceregistry', + 'open', + 'parent', + 'partial', + 'path', + 'peek', + 'persistent_id', + 'persistent_load', + 'print_file_and_line', + 'ps1', + 'ps2', + 'raw', + 'read', + 'read1', + 'readable', + 'readall', + 'readinto', + 'readinto1', + 'readline', + 'reducer_override', + 'reload', + 'replace', + 'reset', + 'return', + 'reversed', + 'seek', + 'seekable', + 'send', + 'setstate', + 'sort', + 'stderr', + 'stdin', + 'stdout', + 'strict', + 'symmetric_difference_update', + 'tell', + 'text', + 'threading', + 'throw', + 'unraisablehook', + 'values', + 'version', + 'warnings', + 'warnoptions', + 'writable', + 'write', + 'zipimporter', +] + + ####################################### # helpers @@ -64,6 +354,38 @@ def block(self, prefix, suffix="", *, continuation=None): END = '/* End auto-generated code */' +def generate_global_strings(): + filename = os.path.join(INTERNAL, 'pycore_global_strings.h') + + # Read the non-generated part of the file. + with open(filename) as infile: + before = ''.join(iter_to_marker(infile, START))[:-1] + for _ in iter_to_marker(infile, END): + pass + after = infile.read()[:-1] + + # Generate the file. + with open(filename, 'w', encoding='utf-8') as outfile: + printer = Printer(outfile) + printer.write(before) + printer.write(START) + with printer.block('struct _Py_global_strings', ';'): + with printer.block('struct', ' literals;'): + for name, literal in STRING_LITERALS.items(): + if literal is None: + outfile.write('\n') + printer.write(f'// {name}') + else: + printer.write(f'STRUCT_FOR_STR({name}, "{literal}")') + outfile.write('\n') + with printer.block('struct', ' identifiers;'): + for name in sorted(IDENTIFIERS): + assert name.isidentifier(), name + printer.write(f'STRUCT_FOR_ID({name})') + printer.write(END) + printer.write(after) + + def generate_runtime_init(): # First get some info from the declarations. nsmallposints = None @@ -106,19 +428,174 @@ def generate_runtime_init(): with printer.block('.bytes_characters =', ','): for i in range(256): printer.write(f'_PyBytes_CHAR_INIT({i}),') + printer.write('') + # Global strings. + with printer.block('.strings =', ','): + with printer.block('.literals =', ','): + for name, literal in STRING_LITERALS.items(): + if literal is None: + printer.write('') + else: + printer.write(f'INIT_STR({name}, "{literal}"),') + with printer.block('.identifiers =', ','): + for name in sorted(IDENTIFIERS): + assert name.isidentifier(), name + printer.write(f'INIT_ID({name}),') printer.write(END) printer.write(after) +####################################### +# checks + +def err(msg): + print(msg, file=sys.stderr) + + +GETTER_RE = re.compile(r''' + ^ + .*? + (?: + (?: + _Py_ID + [(] + ( \w+ ) # + [)] + ) + | + (?: + _Py_STR + [(] + ( \w+ ) # + [)] + ) + ) +''', re.VERBOSE) +TYPESLOTS_RE = re.compile(r''' + ^ + .*? + (?: + (?: + SLOT0 [(] .*?, \s* + ( \w+ ) # + [)] + ) + | + (?: + SLOT1 [(] .*?, \s* + ( \w+ ) # + , .* [)] + ) + | + (?: + SLOT1BIN [(] .*?, .*?, \s* + ( \w+ ) # + , \s* + ( \w+ ) # + [)] + ) + | + (?: + SLOT1BINFULL [(] .*?, .*?, .*?, \s* + ( \w+ ) # + , \s* + ( \w+ ) # + [)] + ) + | + ( SLOT \d .* [^)] $ ) # + ) +''', re.VERBOSE) + +def check_orphan_strings(): + literals = set(n for n, s in STRING_LITERALS.items() if s) + identifiers = set(IDENTIFIERS) + files = glob.iglob(os.path.join(ROOT, '**', '*.[ch]'), recursive=True) + for i, filename in enumerate(files, start=1): + print('.', end='') + if i % 5 == 0: + print(' ', end='') + if i % 20 == 0: + print() + if i % 100 == 0: + print() + with open(filename) as infile: + wrapped = None + for line in infile: + identifier = literal = reverse = None + + line = line.splitlines()[0] + if wrapped: + line = f'{wrapped.rstrip()} {line}' + wrapped = None + + if os.path.basename(filename) == '_warnings.c': + m = re.match(r'^.* = GET_WARNINGS_ATTR[(][^,]*, (\w+),', line) + if m: + identifier, = m.groups() + elif os.path.basename(filename) == 'typeobject.c': + m = TYPESLOTS_RE.match(line) + if m: + (slot0, + slot1, + slot1bin, reverse, + slot1binfull, fullreverse, + wrapped, + ) = m.groups() + identifier = slot0 or slot1 or slot1bin or slot1binfull + reverse = reverse or fullreverse + + if not identifier and not literal: + m = GETTER_RE.match(line) + if not m: + continue + identifier, literal = m.groups() + + if literal: + if literals and literal in literals: + literals.remove(literal) + if identifier: + if identifiers and identifier in identifiers: + identifiers.remove(identifier) + if reverse: + if identifiers and reverse in identifiers: + identifiers.remove(reverse) + if not literals and not identifiers: + break + else: + continue + break + if i % 20: + print() + if not literals and not identifiers: + return + print('ERROR:', file=sys.stderr) + if literals: + err(' unused global string literals:') + for name in sorted(literals): + err(f' {name}') + if identifiers: + if literals: + print() + err(' unused global identifiers:') + for name in sorted(identifiers): + err(f' {name}') + + ####################################### # the script -def main() -> None: +def main(*, check=False) -> None: + generate_global_strings() generate_runtime_init() + if check: + check_orphan_strings() + if __name__ == '__main__': - argv = sys.argv[1:] - if argv: - sys.exit(f'ERROR: got unexpected args {argv}') - main() + import argparse + parser = argparse.ArgumentParser() + parser.add_argument('--check', action='store_true') + args = parser.parse_args() + main(**vars(args)) From webhook-mailer at python.org Tue Feb 8 16:01:46 2022 From: webhook-mailer at python.org (brettcannon) Date: Tue, 08 Feb 2022 21:01:46 -0000 Subject: [Python-checkins] bpo-46556: emit `DeprecationWarning` from `pathlib.Path.__enter__()` (GH-30971) Message-ID: https://github.com/python/cpython/commit/06e1701ad3956352bc0f42b8f51c2f8cc85bf378 commit: 06e1701ad3956352bc0f42b8f51c2f8cc85bf378 branch: main author: Barney Gale committer: brettcannon date: 2022-02-08T13:01:37-08:00 summary: bpo-46556: emit `DeprecationWarning` from `pathlib.Path.__enter__()` (GH-30971) In Python 3.9, Path.__exit__() was made a no-op and has never been documented. Co-authored-by: Brett Cannon files: A Misc/NEWS.d/next/Library/2022-01-27-23-20-30.bpo-46556.tlpAgS.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Lib/pathlib.py b/Lib/pathlib.py index 7f4210e2b80c9..4763ab54f6ba8 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -877,17 +877,20 @@ def _make_child_relpath(self, part): return self._from_parsed_parts(self._drv, self._root, parts) def __enter__(self): + # In previous versions of pathlib, __exit__() marked this path as + # closed; subsequent attempts to perform I/O would raise an IOError. + # This functionality was never documented, and had the effect of + # making Path objects mutable, contrary to PEP 428. + # In Python 3.9 __exit__() was made a no-op. + # In Python 3.11 __enter__() began emitting DeprecationWarning. + # In Python 3.13 __enter__() and __exit__() should be removed. + warnings.warn("pathlib.Path.__enter__() is deprecated and scheduled " + "for removal in Python 3.13; Path objects as a context " + "manager is a no-op", + DeprecationWarning, stacklevel=2) return self def __exit__(self, t, v, tb): - # https://bugs.python.org/issue39682 - # In previous versions of pathlib, this method marked this path as - # closed; subsequent attempts to perform I/O would raise an IOError. - # This functionality was never documented, and had the effect of - # making Path objects mutable, contrary to PEP 428. In Python 3.9 the - # _closed attribute was removed, and this method made a no-op. - # This method and __enter__()/__exit__() should be deprecated and - # removed in the future. pass # Public API diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index ec2baca18fd81..f03fcbef1b0a0 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -1850,8 +1850,10 @@ def test_with(self): it = p.iterdir() it2 = p.iterdir() next(it2) - with p: - pass + # bpo-46556: path context managers are deprecated in Python 3.11. + with self.assertWarns(DeprecationWarning): + with p: + pass # Using a path as a context manager is a no-op, thus the following # operations should still succeed after the context manage exits. next(it) @@ -1859,8 +1861,9 @@ def test_with(self): p.exists() p.resolve() p.absolute() - with p: - pass + with self.assertWarns(DeprecationWarning): + with p: + pass def test_chmod(self): p = self.cls(BASE) / 'fileA' diff --git a/Misc/NEWS.d/next/Library/2022-01-27-23-20-30.bpo-46556.tlpAgS.rst b/Misc/NEWS.d/next/Library/2022-01-27-23-20-30.bpo-46556.tlpAgS.rst new file mode 100644 index 0000000000000..1209e0e2bd8b0 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-27-23-20-30.bpo-46556.tlpAgS.rst @@ -0,0 +1,2 @@ +Deprecate undocumented support for using a :class:`pathlib.Path` object as a +context manager. From webhook-mailer at python.org Tue Feb 8 16:04:13 2022 From: webhook-mailer at python.org (brettcannon) Date: Tue, 08 Feb 2022 21:04:13 -0000 Subject: [Python-checkins] bpo-46678: Fix Invalid cross device link in Lib/test/support/import_helper.py (GH-31204) (GH-31207) Message-ID: https://github.com/python/cpython/commit/c2735b75afd530631efde4ddd3cb24bbdc285559 commit: c2735b75afd530631efde4ddd3cb24bbdc285559 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: brettcannon date: 2022-02-08T13:04:05-08:00 summary: bpo-46678: Fix Invalid cross device link in Lib/test/support/import_helper.py (GH-31204) (GH-31207) In `Lib/test/support/import_helper.py`, the function `make_legacy_pyc` makes a call to `os.rename` which can fail when the source and target live on different devices. This happens (for example) when `PYTHONPYCACHEPREFIX` is set to a directory anywhere on disk, while a ramdisk is mounted on `/tmp` (the latter of which is the default on various Linux distros). Replacing `os.rename` with `shutil.move` fixes this. Automerge-Triggered-By: GH:brettcannon (cherry picked from commit da576e08296490e94924421af71001bcfbccb317) Co-authored-by: Jason Wilkes files: A Misc/NEWS.d/next/Tests/2022-02-07-12-40-45.bpo-46678.zfOrgL.rst M Lib/test/support/import_helper.py diff --git a/Lib/test/support/import_helper.py b/Lib/test/support/import_helper.py index 43ae31483420d..efa8ffad6a8e0 100644 --- a/Lib/test/support/import_helper.py +++ b/Lib/test/support/import_helper.py @@ -2,6 +2,7 @@ import importlib import importlib.util import os +import shutil import sys import unittest import warnings @@ -58,7 +59,7 @@ def make_legacy_pyc(source): pyc_file = importlib.util.cache_from_source(source) up_one = os.path.dirname(os.path.abspath(source)) legacy_pyc = os.path.join(up_one, source + 'c') - os.rename(pyc_file, legacy_pyc) + shutil.move(pyc_file, legacy_pyc) return legacy_pyc diff --git a/Misc/NEWS.d/next/Tests/2022-02-07-12-40-45.bpo-46678.zfOrgL.rst b/Misc/NEWS.d/next/Tests/2022-02-07-12-40-45.bpo-46678.zfOrgL.rst new file mode 100644 index 0000000000000..e369cb1f67baf --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-07-12-40-45.bpo-46678.zfOrgL.rst @@ -0,0 +1,3 @@ +The function ``make_legacy_pyc`` in ``Lib/test/support/import_helper.py`` no +longer fails when ``PYTHONPYCACHEPREFIX`` is set to a directory on a +different device from where tempfiles are stored. From webhook-mailer at python.org Tue Feb 8 20:38:06 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Wed, 09 Feb 2022 01:38:06 -0000 Subject: [Python-checkins] bpo-36876: Update the c-analyzer whitelist. (gh-31225) Message-ID: https://github.com/python/cpython/commit/77bab59c8a1f04922bb975cc4f11e5323d1d379d commit: 77bab59c8a1f04922bb975cc4f11e5323d1d379d branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-08T18:37:57-07:00 summary: bpo-36876: Update the c-analyzer whitelist. (gh-31225) This change adds variables that had been added since the last time the whitelist was updated. It also cleans up the list a little. https://bugs.python.org/issue36876 files: A Tools/c-analyzer/cpython/globals-to-fix.tsv A Tools/c-analyzer/table-file.py M Tools/c-analyzer/cpython/_analyzer.py M Tools/c-analyzer/cpython/ignored.tsv diff --git a/Tools/c-analyzer/cpython/_analyzer.py b/Tools/c-analyzer/cpython/_analyzer.py index 09904236cd623..49de9d887284a 100644 --- a/Tools/c-analyzer/cpython/_analyzer.py +++ b/Tools/c-analyzer/cpython/_analyzer.py @@ -32,6 +32,7 @@ _DATA_DIR = os.path.dirname(__file__) KNOWN_FILE = os.path.join(_DATA_DIR, 'known.tsv') IGNORED_FILE = os.path.join(_DATA_DIR, 'ignored.tsv') +NEED_FIX_FILE = os.path.join(_DATA_DIR, 'globals-to-fix.tsv') KNOWN_IN_DOT_C = { 'struct _odictobject': False, 'PyTupleObject': False, @@ -85,6 +86,7 @@ def write_known(): def read_ignored(): if not _IGNORED: _IGNORED.update(_datafiles.read_ignored(IGNORED_FILE, relroot=REPO_ROOT)) + _IGNORED.update(_datafiles.read_ignored(NEED_FIX_FILE, relroot=REPO_ROOT)) return dict(_IGNORED) diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv new file mode 100644 index 0000000000000..544216ca0b68b --- /dev/null +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -0,0 +1,1646 @@ +filename funcname name reason +#??? - somevar ??? + +# These are all variables that we will be making non-global. + +################################## +# global objects to fix in core code + +#----------------------- +# static types + +Objects/boolobject.c - PyBool_Type - +Objects/bytearrayobject.c - PyByteArrayIter_Type - +Objects/bytearrayobject.c - PyByteArray_Type - +Objects/bytesobject.c - PyBytesIter_Type - +Objects/bytesobject.c - PyBytes_Type - +Objects/capsule.c - PyCapsule_Type - +Objects/cellobject.c - PyCell_Type - +Objects/classobject.c - PyInstanceMethod_Type - +Objects/classobject.c - PyMethod_Type - +Objects/codeobject.c - LineIterator - +Objects/codeobject.c - PositionsIterator - +Objects/codeobject.c - PyCode_Type - +Objects/complexobject.c - PyComplex_Type - +Objects/descrobject.c - PyClassMethodDescr_Type - +Objects/descrobject.c - PyDictProxy_Type - +Objects/descrobject.c - PyGetSetDescr_Type - +Objects/descrobject.c - PyMemberDescr_Type - +Objects/descrobject.c - PyMethodDescr_Type - +Objects/descrobject.c - PyProperty_Type - +Objects/descrobject.c - PyWrapperDescr_Type - +Objects/descrobject.c - _PyMethodWrapper_Type - +Objects/dictobject.c - PyDictItems_Type - +Objects/dictobject.c - PyDictIterItem_Type - +Objects/dictobject.c - PyDictIterKey_Type - +Objects/dictobject.c - PyDictIterValue_Type - +Objects/dictobject.c - PyDictKeys_Type - +Objects/dictobject.c - PyDictRevIterItem_Type - +Objects/dictobject.c - PyDictRevIterKey_Type - +Objects/dictobject.c - PyDictRevIterValue_Type - +Objects/dictobject.c - PyDictValues_Type - +Objects/dictobject.c - PyDict_Type - +Objects/enumobject.c - PyEnum_Type - +Objects/enumobject.c - PyReversed_Type - +Objects/exceptions.c - _PyExc_BaseExceptionGroup - +Objects/exceptions.c - _PyExc_EncodingWarning - +Objects/fileobject.c - PyStdPrinter_Type - +Objects/floatobject.c - FloatInfoType - +Objects/floatobject.c - PyFloat_Type - +Objects/frameobject.c - PyFrame_Type - +Objects/funcobject.c - PyClassMethod_Type - +Objects/funcobject.c - PyFunction_Type - +Objects/funcobject.c - PyStaticMethod_Type - +Objects/genericaliasobject.c - Py_GenericAliasType - +Objects/genobject.c - PyAsyncGen_Type - +Objects/genobject.c - PyCoro_Type - +Objects/genobject.c - PyGen_Type - +Objects/genobject.c - _PyAsyncGenASend_Type - +Objects/genobject.c - _PyAsyncGenAThrow_Type - +Objects/genobject.c - _PyAsyncGenWrappedValue_Type - +Objects/genobject.c - _PyCoroWrapper_Type - +Objects/interpreteridobject.c - _PyInterpreterID_Type - +Objects/iterobject.c - PyCallIter_Type - +Objects/iterobject.c - PySeqIter_Type - +Objects/iterobject.c - _PyAnextAwaitable_Type - +Objects/listobject.c - PyListIter_Type - +Objects/listobject.c - PyListRevIter_Type - +Objects/listobject.c - PyList_Type - +Objects/longobject.c - Int_InfoType - +Objects/longobject.c - PyLong_Type - +Objects/memoryobject.c - PyMemoryIter_Type - +Objects/memoryobject.c - PyMemoryView_Type - +Objects/memoryobject.c - _PyManagedBuffer_Type - +Objects/methodobject.c - PyCFunction_Type - +Objects/methodobject.c - PyCMethod_Type - +Objects/moduleobject.c - PyModuleDef_Type - +Objects/moduleobject.c - PyModule_Type - +Objects/namespaceobject.c - _PyNamespace_Type - +Objects/object.c - _PyNone_Type - +Objects/object.c - _PyNotImplemented_Type - +Objects/odictobject.c - PyODictItems_Type - +Objects/odictobject.c - PyODictIter_Type - +Objects/odictobject.c - PyODictKeys_Type - +Objects/odictobject.c - PyODictValues_Type - +Objects/odictobject.c - PyODict_Type - +Objects/picklebufobject.c - PyPickleBuffer_Type - +Objects/rangeobject.c - PyLongRangeIter_Type - +Objects/rangeobject.c - PyRangeIter_Type - +Objects/rangeobject.c - PyRange_Type - +Objects/setobject.c - PyFrozenSet_Type - +Objects/setobject.c - PySetIter_Type - +Objects/setobject.c - PySet_Type - +Objects/setobject.c - _PySetDummy_Type - +Objects/sliceobject.c - PyEllipsis_Type - +Objects/sliceobject.c - PySlice_Type - +Objects/tupleobject.c - PyTupleIter_Type - +Objects/tupleobject.c - PyTuple_Type - +Objects/typeobject.c - PyBaseObject_Type - +Objects/typeobject.c - PySuper_Type - +Objects/typeobject.c - PyType_Type - +Objects/unicodeobject.c - EncodingMapType - +Objects/unicodeobject.c - PyUnicodeIter_Type - +Objects/unicodeobject.c - PyUnicode_Type - +Objects/unionobject.c - _PyUnion_Type - +Objects/unionobject.c - _Py_UnionType - +Objects/weakrefobject.c - _PyWeakref_CallableProxyType - +Objects/weakrefobject.c - _PyWeakref_ProxyType - +Objects/weakrefobject.c - _PyWeakref_RefType - + +#----------------------- +# builtin exception types + +Objects/exceptions.c - _PyExc_BaseException - +Objects/exceptions.c - _PyExc_UnicodeEncodeError - +Objects/exceptions.c - _PyExc_UnicodeDecodeError - +Objects/exceptions.c - _PyExc_UnicodeTranslateError - +Objects/exceptions.c - _PyExc_MemoryError - +Objects/exceptions.c - _PyExc_Exception - +Objects/exceptions.c - _PyExc_TypeError - +Objects/exceptions.c - _PyExc_StopAsyncIteration - +Objects/exceptions.c - _PyExc_StopIteration - +Objects/exceptions.c - _PyExc_GeneratorExit - +Objects/exceptions.c - _PyExc_SystemExit - +Objects/exceptions.c - _PyExc_KeyboardInterrupt - +Objects/exceptions.c - _PyExc_ImportError - +Objects/exceptions.c - _PyExc_ModuleNotFoundError - +Objects/exceptions.c - _PyExc_OSError - +Objects/exceptions.c - _PyExc_BlockingIOError - +Objects/exceptions.c - _PyExc_ConnectionError - +Objects/exceptions.c - _PyExc_ChildProcessError - +Objects/exceptions.c - _PyExc_BrokenPipeError - +Objects/exceptions.c - _PyExc_ConnectionAbortedError - +Objects/exceptions.c - _PyExc_ConnectionRefusedError - +Objects/exceptions.c - _PyExc_ConnectionResetError - +Objects/exceptions.c - _PyExc_FileExistsError - +Objects/exceptions.c - _PyExc_FileNotFoundError - +Objects/exceptions.c - _PyExc_IsADirectoryError - +Objects/exceptions.c - _PyExc_NotADirectoryError - +Objects/exceptions.c - _PyExc_InterruptedError - +Objects/exceptions.c - _PyExc_PermissionError - +Objects/exceptions.c - _PyExc_ProcessLookupError - +Objects/exceptions.c - _PyExc_TimeoutError - +Objects/exceptions.c - _PyExc_EOFError - +Objects/exceptions.c - _PyExc_RuntimeError - +Objects/exceptions.c - _PyExc_RecursionError - +Objects/exceptions.c - _PyExc_NotImplementedError - +Objects/exceptions.c - _PyExc_NameError - +Objects/exceptions.c - _PyExc_UnboundLocalError - +Objects/exceptions.c - _PyExc_AttributeError - +Objects/exceptions.c - _PyExc_SyntaxError - +Objects/exceptions.c - _PyExc_IndentationError - +Objects/exceptions.c - _PyExc_TabError - +Objects/exceptions.c - _PyExc_LookupError - +Objects/exceptions.c - _PyExc_IndexError - +Objects/exceptions.c - _PyExc_KeyError - +Objects/exceptions.c - _PyExc_ValueError - +Objects/exceptions.c - _PyExc_UnicodeError - +Objects/exceptions.c - _PyExc_AssertionError - +Objects/exceptions.c - _PyExc_ArithmeticError - +Objects/exceptions.c - _PyExc_FloatingPointError - +Objects/exceptions.c - _PyExc_OverflowError - +Objects/exceptions.c - _PyExc_ZeroDivisionError - +Objects/exceptions.c - _PyExc_SystemError - +Objects/exceptions.c - _PyExc_ReferenceError - +Objects/exceptions.c - _PyExc_BufferError - +Objects/exceptions.c - _PyExc_Warning - +Objects/exceptions.c - _PyExc_UserWarning - +Objects/exceptions.c - _PyExc_DeprecationWarning - +Objects/exceptions.c - _PyExc_PendingDeprecationWarning - +Objects/exceptions.c - _PyExc_SyntaxWarning - +Objects/exceptions.c - _PyExc_RuntimeWarning - +Objects/exceptions.c - _PyExc_FutureWarning - +Objects/exceptions.c - _PyExc_ImportWarning - +Objects/exceptions.c - _PyExc_UnicodeWarning - +Objects/exceptions.c - _PyExc_BytesWarning - +Objects/exceptions.c - _PyExc_ResourceWarning - +Objects/exceptions.c - PyExc_EnvironmentError - +Objects/exceptions.c - PyExc_IOError - +Objects/exceptions.c - PyExc_BaseException - +Objects/exceptions.c - PyExc_Exception - +Objects/exceptions.c - PyExc_TypeError - +Objects/exceptions.c - PyExc_StopAsyncIteration - +Objects/exceptions.c - PyExc_StopIteration - +Objects/exceptions.c - PyExc_GeneratorExit - +Objects/exceptions.c - PyExc_SystemExit - +Objects/exceptions.c - PyExc_KeyboardInterrupt - +Objects/exceptions.c - PyExc_ImportError - +Objects/exceptions.c - PyExc_ModuleNotFoundError - +Objects/exceptions.c - PyExc_OSError - +Objects/exceptions.c - PyExc_BlockingIOError - +Objects/exceptions.c - PyExc_ConnectionError - +Objects/exceptions.c - PyExc_ChildProcessError - +Objects/exceptions.c - PyExc_BrokenPipeError - +Objects/exceptions.c - PyExc_ConnectionAbortedError - +Objects/exceptions.c - PyExc_ConnectionRefusedError - +Objects/exceptions.c - PyExc_ConnectionResetError - +Objects/exceptions.c - PyExc_FileExistsError - +Objects/exceptions.c - PyExc_FileNotFoundError - +Objects/exceptions.c - PyExc_IsADirectoryError - +Objects/exceptions.c - PyExc_NotADirectoryError - +Objects/exceptions.c - PyExc_InterruptedError - +Objects/exceptions.c - PyExc_PermissionError - +Objects/exceptions.c - PyExc_ProcessLookupError - +Objects/exceptions.c - PyExc_TimeoutError - +Objects/exceptions.c - PyExc_EOFError - +Objects/exceptions.c - PyExc_RuntimeError - +Objects/exceptions.c - PyExc_RecursionError - +Objects/exceptions.c - PyExc_NotImplementedError - +Objects/exceptions.c - PyExc_NameError - +Objects/exceptions.c - PyExc_UnboundLocalError - +Objects/exceptions.c - PyExc_AttributeError - +Objects/exceptions.c - PyExc_SyntaxError - +Objects/exceptions.c - PyExc_IndentationError - +Objects/exceptions.c - PyExc_TabError - +Objects/exceptions.c - PyExc_LookupError - +Objects/exceptions.c - PyExc_IndexError - +Objects/exceptions.c - PyExc_KeyError - +Objects/exceptions.c - PyExc_ValueError - +Objects/exceptions.c - PyExc_UnicodeError - +Objects/exceptions.c - PyExc_UnicodeEncodeError - +Objects/exceptions.c - PyExc_UnicodeDecodeError - +Objects/exceptions.c - PyExc_UnicodeTranslateError - +Objects/exceptions.c - PyExc_AssertionError - +Objects/exceptions.c - PyExc_ArithmeticError - +Objects/exceptions.c - PyExc_FloatingPointError - +Objects/exceptions.c - PyExc_OverflowError - +Objects/exceptions.c - PyExc_ZeroDivisionError - +Objects/exceptions.c - PyExc_SystemError - +Objects/exceptions.c - PyExc_ReferenceError - +Objects/exceptions.c - PyExc_MemoryError - +Objects/exceptions.c - PyExc_BufferError - +Objects/exceptions.c - PyExc_Warning - +Objects/exceptions.c - PyExc_UserWarning - +Objects/exceptions.c - PyExc_DeprecationWarning - +Objects/exceptions.c - PyExc_PendingDeprecationWarning - +Objects/exceptions.c - PyExc_SyntaxWarning - +Objects/exceptions.c - PyExc_RuntimeWarning - +Objects/exceptions.c - PyExc_FutureWarning - +Objects/exceptions.c - PyExc_ImportWarning - +Objects/exceptions.c - PyExc_UnicodeWarning - +Objects/exceptions.c - PyExc_BytesWarning - +Objects/exceptions.c - PyExc_ResourceWarning - + +#----------------------- +# singletons + +Objects/boolobject.c - _Py_FalseStruct - +Objects/boolobject.c - _Py_TrueStruct - +Objects/dictobject.c - empty_keys_struct - +Objects/dictobject.c - empty_values_struct - +Objects/object.c - _Py_NoneStruct - +Objects/object.c - _Py_NotImplementedStruct - +Objects/setobject.c - _dummy_struct - +Objects/setobject.c - _PySet_Dummy - +Objects/sliceobject.c - _Py_EllipsisObject - + +#----------------------- +# cached PyUnicodeObject - initialized once + +# _Py_IDENTIFIER (global) +Objects/bytesobject.c - PyId___bytes__ - +Objects/classobject.c - PyId___name__ - +Objects/classobject.c - PyId___qualname__ - +Objects/descrobject.c - PyId_getattr - +Objects/fileobject.c - PyId_open - +Objects/frameobject.c - PyId___builtins__ - +Objects/iterobject.c - PyId_iter - +Objects/longobject.c - PyId_big - +Objects/longobject.c - PyId_little - +Objects/moduleobject.c - PyId___annotations__ - +Objects/moduleobject.c - PyId___dict__ - +Objects/moduleobject.c - PyId___dir__ - +Objects/moduleobject.c - PyId___doc__ - +Objects/moduleobject.c - PyId___name__ - +Objects/moduleobject.c - PyId___spec__ - +Objects/object.c - PyId_Py_Repr - +Objects/object.c - PyId___bytes__ - +Objects/object.c - PyId___dir__ - +Objects/object.c - PyId___isabstractmethod__ - +Objects/odictobject.c - PyId_items - +Objects/rangeobject.c - PyId_iter - +Objects/structseq.c - PyId_n_fields - +Objects/structseq.c - PyId_n_sequence_fields - +Objects/structseq.c - PyId_n_unnamed_fields - +Objects/typeobject.c - PyId___abstractmethods__ - +Objects/typeobject.c - PyId___annotations__ - +Objects/typeobject.c - PyId___class__ - +Objects/typeobject.c - PyId___class_getitem__ - +Objects/typeobject.c - PyId___classcell__ - +Objects/typeobject.c - PyId___delitem__ - +Objects/typeobject.c - PyId___dict__ - +Objects/typeobject.c - PyId___doc__ - +Objects/typeobject.c - PyId___getattribute__ - +Objects/typeobject.c - PyId___getitem__ - +Objects/typeobject.c - PyId___hash__ - +Objects/typeobject.c - PyId___init_subclass__ - +Objects/typeobject.c - PyId___len__ - +Objects/typeobject.c - PyId___module__ - +Objects/typeobject.c - PyId___name__ - +Objects/typeobject.c - PyId___new__ - +Objects/typeobject.c - PyId___qualname__ - +Objects/typeobject.c - PyId___set_name__ - +Objects/typeobject.c - PyId___setitem__ - +Objects/typeobject.c - PyId___weakref__ - +Objects/typeobject.c - PyId_builtins - +Objects/typeobject.c - PyId_mro - + +# _Py_IDENTIFIER (local) +Objects/abstract.c PyIter_Send PyId_send - +Objects/abstract.c PyMapping_Items PyId_items - +Objects/abstract.c PyMapping_Keys PyId_keys - +Objects/abstract.c PyMapping_Values PyId_values - +Objects/abstract.c PyNumber_Long PyId___trunc__ - +Objects/abstract.c PyObject_Format PyId___format__ - +Objects/abstract.c PyObject_GetItem PyId___class_getitem__ - +Objects/abstract.c PyObject_LengthHint PyId___length_hint__ - +Objects/abstract.c abstract_get_bases PyId___bases__ - +Objects/abstract.c object_isinstance PyId___class__ - +Objects/abstract.c object_issubclass PyId___subclasscheck__ - +Objects/abstract.c object_recursive_isinstance PyId___instancecheck__ - +Objects/bytearrayobject.c _common_reduce PyId___dict__ - +Objects/bytearrayobject.c bytearrayiter_reduce PyId_iter - +Objects/bytesobject.c striter_reduce PyId_iter - +Objects/classobject.c method_reduce PyId_getattr - +Objects/complexobject.c try_complex_special_method PyId___complex__ - +Objects/descrobject.c calculate_qualname PyId___qualname__ - +Objects/descrobject.c mappingproxy_copy PyId_copy - +Objects/descrobject.c mappingproxy_get PyId_get - +Objects/descrobject.c mappingproxy_items PyId_items - +Objects/descrobject.c mappingproxy_keys PyId_keys - +Objects/descrobject.c mappingproxy_reversed PyId___reversed__ - +Objects/descrobject.c mappingproxy_values PyId_values - +Objects/descrobject.c property_init_impl PyId___doc__ - +Objects/dictobject.c _PyDictView_Intersect PyId_intersection - +Objects/dictobject.c dict_subscript PyId___missing__ - +Objects/dictobject.c dict_update_arg PyId_keys - +Objects/dictobject.c dictitems_xor PyId_items - +Objects/dictobject.c dictiter_reduce PyId_iter - +Objects/dictobject.c dictviews_sub PyId_difference_update - +Objects/dictobject.c dictviews_xor PyId_symmetric_difference_update - +Objects/enumobject.c reversed_new_impl PyId___reversed__ - +Objects/exceptions.c ImportError_getstate PyId_name - +Objects/exceptions.c ImportError_getstate PyId_path - +Objects/fileobject.c PyFile_GetLine PyId_readline - +Objects/fileobject.c PyFile_WriteObject PyId_write - +Objects/fileobject.c PyObject_AsFileDescriptor PyId_fileno - +Objects/funcobject.c PyFunction_NewWithQualName PyId___name__ - +Objects/funcobject.c functools_wraps PyId___annotations__ - +Objects/funcobject.c functools_wraps PyId___doc__ - +Objects/funcobject.c functools_wraps PyId___module__ - +Objects/funcobject.c functools_wraps PyId___name__ - +Objects/funcobject.c functools_wraps PyId___qualname__ - +Objects/genericaliasobject.c _Py_make_parameters PyId___parameters__ - +Objects/genericaliasobject.c ga_repr_item PyId___args__ - +Objects/genericaliasobject.c ga_repr_item PyId___module__ - +Objects/genericaliasobject.c ga_repr_item PyId___origin__ - +Objects/genericaliasobject.c ga_repr_item PyId___qualname__ - +Objects/genericaliasobject.c make_parameters PyId___parameters__ - +Objects/genericaliasobject.c subs_tvars PyId___parameters__ - +Objects/genobject.c PyIter_Send PyId_send - +Objects/genobject.c _gen_throw PyId_throw - +Objects/genobject.c gen_close_iter PyId_close - +Objects/listobject.c listiter_reduce_general PyId_iter - +Objects/listobject.c listiter_reduce_general PyId_reversed - +Objects/methodobject.c meth_get__qualname__ PyId___qualname__ - +Objects/methodobject.c meth_reduce PyId_getattr - +Objects/moduleobject.c PyModule_GetFilenameObject PyId___file__ - +Objects/moduleobject.c _PyModuleSpec_IsInitializing PyId__initializing - +Objects/moduleobject.c _PyModuleSpec_IsUninitializedSubmodule PyId__uninitialized_submodules - +Objects/moduleobject.c module_dir PyId___dict__ - +Objects/moduleobject.c module_dir PyId___dir__ - +Objects/moduleobject.c module_getattro PyId___getattr__ - +Objects/moduleobject.c module_init_dict PyId___loader__ - +Objects/moduleobject.c module_init_dict PyId___package__ - +Objects/object.c _PyObject_FunctionStr PyId___module__ - +Objects/object.c _PyObject_FunctionStr PyId___qualname__ - +Objects/object.c _PyObject_FunctionStr PyId_builtins - +Objects/object.c set_attribute_error_context PyId_name - +Objects/object.c set_attribute_error_context PyId_obj - +Objects/odictobject.c mutablemapping_update_arg PyId_keys - +Objects/odictobject.c odict_reduce PyId___dict__ - +Objects/odictobject.c odictiter_reduce PyId_iter - +Objects/setobject.c set_reduce PyId___dict__ - +Objects/setobject.c setiter_reduce PyId_iter - +Objects/tupleobject.c tupleiter_reduce PyId_iter - +Objects/typeobject.c _PyObject_GetItemsIter PyId_items - +Objects/typeobject.c _PyObject_GetNewArguments PyId___getnewargs__ - +Objects/typeobject.c _PyObject_GetNewArguments PyId___getnewargs_ex__ - +Objects/typeobject.c _PyObject_GetState PyId___getstate__ - +Objects/typeobject.c _PyType_GetSlotNames PyId___slotnames__ - +Objects/typeobject.c _PyType_GetSlotNames PyId__slotnames - +Objects/typeobject.c import_copyreg PyId_copyreg - +Objects/typeobject.c merge_class_dict PyId___bases__ - +Objects/typeobject.c object___reduce_ex___impl PyId___reduce__ - +Objects/typeobject.c overrides_hash PyId___eq__ - +Objects/typeobject.c reduce_newobj PyId___newobj__ - +Objects/typeobject.c reduce_newobj PyId___newobj_ex__ - +Objects/typeobject.c slot_am_aiter PyId___aiter__ - +Objects/typeobject.c slot_am_anext PyId___anext__ - +Objects/typeobject.c slot_am_await PyId___await__ - +Objects/typeobject.c slot_nb_bool PyId___bool__ - +Objects/typeobject.c slot_nb_index PyId___index__ - +Objects/typeobject.c slot_nb_inplace_power PyId___ipow__ - +Objects/typeobject.c slot_nb_power PyId___pow__ - +Objects/typeobject.c slot_sq_contains PyId___contains__ - +Objects/typeobject.c slot_tp_call PyId___call__ - +Objects/typeobject.c slot_tp_descr_get PyId___get__ - +Objects/typeobject.c slot_tp_descr_set PyId___delete__ - +Objects/typeobject.c slot_tp_descr_set PyId___set__ - +Objects/typeobject.c slot_tp_finalize PyId___del__ - +Objects/typeobject.c slot_tp_getattr_hook PyId___getattr__ - +Objects/typeobject.c slot_tp_init PyId___init__ - +Objects/typeobject.c slot_tp_iter PyId___iter__ - +Objects/typeobject.c slot_tp_iternext PyId___next__ - +Objects/typeobject.c slot_tp_repr PyId___repr__ - +Objects/typeobject.c slot_tp_setattro PyId___delattr__ - +Objects/typeobject.c slot_tp_setattro PyId___setattr__ - +Objects/typeobject.c type_new PyId___classcell__ - +Objects/typeobject.c type_new PyId___mro_entries__ - +Objects/typeobject.c type_new PyId___qualname__ - +Objects/typeobject.c type_new PyId___slots__ - +Objects/typeobject.c type_new_get_bases PyId___mro_entries__ - +Objects/typeobject.c type_new_get_slots PyId___slots__ - +Objects/unicodeobject.c unicodeiter_reduce PyId_iter - +Objects/unionobject.c union_repr_item PyId___args__ - +Objects/unionobject.c union_repr_item PyId___module__ - +Objects/unionobject.c union_repr_item PyId___origin__ - +Objects/unionobject.c union_repr_item PyId___qualname__ - +Objects/weakrefobject.c proxy_bytes PyId___bytes__ - +Objects/weakrefobject.c proxy_reversed PyId___reversed__ - +Objects/weakrefobject.c weakref_repr PyId___name__ - +Parser/tokenizer.c fp_setreadl PyId_open - +Parser/tokenizer.c fp_setreadl PyId_readline - + +# _Py_static_string +Objects/typeobject.c - name_op - +Objects/typeobject.c object_new comma_id - +Objects/typeobject.c slot_mp_subscript id - +Objects/typeobject.c slot_nb_add op_id - +Objects/typeobject.c slot_nb_add rop_id - +Objects/typeobject.c slot_nb_subtract op_id - +Objects/typeobject.c slot_nb_subtract rop_id - +Objects/typeobject.c slot_nb_multiply op_id - +Objects/typeobject.c slot_nb_multiply rop_id - +Objects/typeobject.c slot_nb_matrix_multiply op_id - +Objects/typeobject.c slot_nb_matrix_multiply rop_id - +Objects/typeobject.c slot_nb_remainder op_id - +Objects/typeobject.c slot_nb_remainder rop_id - +Objects/typeobject.c slot_nb_divmod op_id - +Objects/typeobject.c slot_nb_divmod rop_id - +Objects/typeobject.c slot_nb_power_binary op_id - +Objects/typeobject.c slot_nb_power_binary rop_id - +Objects/typeobject.c slot_nb_negative id - +Objects/typeobject.c slot_nb_positive id - +Objects/typeobject.c slot_nb_absolute id - +Objects/typeobject.c slot_nb_invert id - +Objects/typeobject.c slot_nb_lshift op_id - +Objects/typeobject.c slot_nb_lshift rop_id - +Objects/typeobject.c slot_nb_rshift op_id - +Objects/typeobject.c slot_nb_rshift rop_id - +Objects/typeobject.c slot_nb_and op_id - +Objects/typeobject.c slot_nb_and rop_id - +Objects/typeobject.c slot_nb_xor op_id - +Objects/typeobject.c slot_nb_xor rop_id - +Objects/typeobject.c slot_nb_or op_id - +Objects/typeobject.c slot_nb_or rop_id - +Objects/typeobject.c slot_nb_int id - +Objects/typeobject.c slot_nb_float id - +Objects/typeobject.c slot_nb_inplace_add id - +Objects/typeobject.c slot_nb_inplace_subtract id - +Objects/typeobject.c slot_nb_inplace_multiply id - +Objects/typeobject.c slot_nb_inplace_matrix_multiply id - +Objects/typeobject.c slot_nb_inplace_remainder id - +Objects/typeobject.c slot_nb_inplace_lshift id - +Objects/typeobject.c slot_nb_inplace_rshift id - +Objects/typeobject.c slot_nb_inplace_and id - +Objects/typeobject.c slot_nb_inplace_xor id - +Objects/typeobject.c slot_nb_inplace_or id - +Objects/typeobject.c slot_nb_floor_divide op_id - +Objects/typeobject.c slot_nb_floor_divide rop_id - +Objects/typeobject.c slot_nb_true_divide op_id - +Objects/typeobject.c slot_nb_true_divide rop_id - +Objects/typeobject.c slot_nb_inplace_floor_divide id - +Objects/typeobject.c slot_nb_inplace_true_divide id - +Objects/typeobject.c slot_tp_str id - +Python/compile.c compiler_set_qualname dot - +Python/compile.c compiler_set_qualname dot_locals - + +# manually cached PyUnicodeObject +Objects/boolobject.c - false_str - +Objects/boolobject.c - true_str - +Objects/classobject.c method_get_doc docstr - +Objects/classobject.c instancemethod_get_doc docstr - +Objects/codeobject.c PyCode_NewEmpty emptystring - +Objects/exceptions.c _check_for_legacy_statements print_prefix - +Objects/exceptions.c _check_for_legacy_statements exec_prefix - +Objects/funcobject.c PyFunction_NewWithQualName __name__ - +Objects/listobject.c - indexerr - +Objects/typeobject.c object___reduce_ex___impl objreduce - +# XXX This should have been found by the analyzer but wasn't: +Python/_warnings.c is_internal_frame bootstrap_string - +# XXX This should have been found by the analyzer but wasn't: +Python/_warnings.c is_internal_frame importlib_string - +# XXX This should have been found by the analyzer but wasn't: +Python/ast_unparse.c - _str_close_br - +# XXX This should have been found by the analyzer but wasn't: +Python/ast_unparse.c - _str_dbl_close_br - +# XXX This should have been found by the analyzer but wasn't: +Python/ast_unparse.c - _str_dbl_open_br - +# XXX This should have been found by the analyzer but wasn't: +Python/ast_unparse.c - _str_inf - +# XXX This should have been found by the analyzer but wasn't: +Python/ast_unparse.c - _str_open_br - +# XXX This should have been found by the analyzer but wasn't: +Python/ast_unparse.c - _str_replace_inf - +# XXX This should have been found by the analyzer but wasn't: +Python/compile.c - __annotations__ - +# XXX This should have been found by the analyzer but wasn't: +Python/compile.c - __doc__ - +# XXX This should have been found by the analyzer but wasn't: +Python/compile.c compiler_dictcomp name - +# XXX This should have been found by the analyzer but wasn't: +Python/compile.c compiler_from_import empty_string - +# XXX This should have been found by the analyzer but wasn't: +Python/compile.c compiler_genexp name - +# XXX This should have been found by the analyzer but wasn't: +Python/compile.c compiler_lambda name - +# XXX This should have been found by the analyzer but wasn't: +Python/compile.c compiler_listcomp name - +# XXX This should have been found by the analyzer but wasn't: +Python/compile.c compiler_setcomp name - +# XXX This should have been found by the analyzer but wasn't: +Python/compile.c compiler_visit_annotations return_str - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c PyImport_Import builtins_str - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c PyImport_Import import_str - +# XXX This should have been found by the analyzer but wasn't: +Python/sysmodule.c - whatstrings - +# XXX This should have been found by the analyzer but wasn't: +Python/sysmodule.c sys_displayhook newline - + +#----------------------- +# _PyArg_Parser + +Objects/clinic/bytearrayobject.c.h bytearray___init__ _parser - +Objects/clinic/bytearrayobject.c.h bytearray_decode _parser - +Objects/clinic/bytearrayobject.c.h bytearray_hex _parser - +Objects/clinic/bytearrayobject.c.h bytearray_rsplit _parser - +Objects/clinic/bytearrayobject.c.h bytearray_split _parser - +Objects/clinic/bytearrayobject.c.h bytearray_splitlines _parser - +Objects/clinic/bytearrayobject.c.h bytearray_translate _parser - +Objects/clinic/bytesobject.c.h bytes_decode _parser - +Objects/clinic/bytesobject.c.h bytes_hex _parser - +Objects/clinic/bytesobject.c.h bytes_new _parser - +Objects/clinic/bytesobject.c.h bytes_rsplit _parser - +Objects/clinic/bytesobject.c.h bytes_split _parser - +Objects/clinic/bytesobject.c.h bytes_splitlines _parser - +Objects/clinic/bytesobject.c.h bytes_translate _parser - +Objects/clinic/codeobject.c.h code__varname_from_oparg _parser - +Objects/clinic/codeobject.c.h code_replace _parser - +Objects/clinic/complexobject.c.h complex_new _parser - +Objects/clinic/descrobject.c.h mappingproxy_new _parser - +Objects/clinic/descrobject.c.h property_init _parser - +Objects/clinic/enumobject.c.h enum_new _parser - +Objects/clinic/funcobject.c.h func_new _parser - +Objects/clinic/listobject.c.h list_sort _parser - +Objects/clinic/longobject.c.h int_from_bytes _parser - +Objects/clinic/longobject.c.h int_to_bytes _parser - +Objects/clinic/longobject.c.h long_new _parser - +Objects/clinic/memoryobject.c.h memoryview _parser - +Objects/clinic/memoryobject.c.h memoryview_cast _parser - +Objects/clinic/memoryobject.c.h memoryview_hex _parser - +Objects/clinic/memoryobject.c.h memoryview_tobytes _parser - +Objects/clinic/moduleobject.c.h module___init__ _parser - +Objects/clinic/odictobject.c.h OrderedDict_fromkeys _parser - +Objects/clinic/odictobject.c.h OrderedDict_move_to_end _parser - +Objects/clinic/odictobject.c.h OrderedDict_pop _parser - +Objects/clinic/odictobject.c.h OrderedDict_popitem _parser - +Objects/clinic/odictobject.c.h OrderedDict_setdefault _parser - +Objects/clinic/structseq.c.h structseq_new _parser - +Objects/clinic/unicodeobject.c.h unicode_encode _parser - +Objects/clinic/unicodeobject.c.h unicode_expandtabs _parser - +Objects/clinic/unicodeobject.c.h unicode_new _parser - +Objects/clinic/unicodeobject.c.h unicode_rsplit _parser - +Objects/clinic/unicodeobject.c.h unicode_split _parser - +Objects/clinic/unicodeobject.c.h unicode_splitlines _parser - +Python/clinic/Python-tokenize.c.h tokenizeriter_new _parser - +Python/clinic/_warnings.c.h warnings_warn _parser - +Python/clinic/bltinmodule.c.h builtin_compile _parser - +Python/clinic/bltinmodule.c.h builtin_pow _parser - +Python/clinic/bltinmodule.c.h builtin_print _parser - +Python/clinic/bltinmodule.c.h builtin_round _parser - +Python/clinic/bltinmodule.c.h builtin_sum _parser - +Python/clinic/import.c.h _imp_find_frozen _parser - +Python/clinic/import.c.h _imp_source_hash _parser - +Python/clinic/sysmodule.c.h sys_addaudithook _parser - +Python/clinic/sysmodule.c.h sys_set_coroutine_origin_tracking_depth _parser - +Python/clinic/traceback.c.h tb_new _parser - + +#----------------------- +# other + +# cache - initialized once +Objects/unicodeobject.c - interned - +Objects/unicodeobject.c - static_strings - +Objects/typeobject.c - method_cache - + +# non-cache - initialized once +Objects/exceptions.c - PyExc_BaseExceptionGroup - +Objects/exceptions.c - PyExc_EncodingWarning - +# XXX This should have been found by the analyzer but wasn't: +Python/context.c - _token_missing - +# XXX This should have been found by the analyzer but wasn't: +Python/fileutils.c - _Py_open_cloexec_works - +# XXX This should have been found by the analyzer but wasn't: +Python/hamt.c - _empty_bitmap_node - +# XXX This should have been found by the analyzer but wasn't: +Python/hamt.c - _empty_hamt - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c PyImport_Import silly_list - + +# state +Objects/typeobject.c resolve_slotdups pname - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c - extensions - + + +################################## +# global non-objects to fix in core code + +#----------------------- +# initialized once + +# during init +Parser/parser.c - Py_DebugFlag - + +# other +Objects/codeobject.c PyCode_NewEmpty nulltuple - +Objects/longobject.c PyLong_FromString log_base_BASE - +Objects/longobject.c PyLong_FromString convwidth_base - +Objects/longobject.c PyLong_FromString convmultmax_base - +Objects/typeobject.c - slotdefs - +Objects/typeobject.c - slotdefs_initialized - +Objects/unicodeobject.c - bloom_linebreak - +Objects/unicodeobject.c - ucnhash_capi - +Parser/action_helpers.c _PyPegen_dummy_name cache - +Parser/pegen.c _PyPegen_dummy_name cache - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c - import_lock - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c import_find_and_load header - + +#----------------------- +# state + +# allocator +Objects/obmalloc.c - _PyObject_Arena - +Objects/obmalloc.c - _Py_tracemalloc_config - +Objects/obmalloc.c - arena_map_bot_count - +Objects/obmalloc.c - arena_map_mid_count - +Objects/obmalloc.c - arena_map_root - +Objects/obmalloc.c - arenas - +Objects/obmalloc.c - maxarenas - +Objects/obmalloc.c - narenas_currently_allocated - +Objects/obmalloc.c - narenas_highwater - +Objects/obmalloc.c - nfp2lasta - +Objects/obmalloc.c - ntimes_arena_allocated - +Objects/obmalloc.c - raw_allocated_blocks - +Objects/obmalloc.c - unused_arena_objects - +Objects/obmalloc.c - usable_arenas - +Objects/obmalloc.c new_arena debug_stats - + +# REPL +Parser/myreadline.c - _PyOS_ReadlineLock - +Parser/myreadline.c - _PyOS_ReadlineTState - +Parser/myreadline.c - PyOS_InputHook - +Parser/myreadline.c - PyOS_ReadlineFunctionPointer - + +# other +Modules/signalmodule.c - signal_global_state - +Objects/dictobject.c - _pydict_global_version - +Objects/dictobject.c - next_dict_keys_version - +Objects/dictobject.c - pydict_global_version - +Objects/floatobject.c - double_format - +Objects/floatobject.c - float_format - +Objects/floatobject.c - detected_double_format - +Objects/floatobject.c - detected_float_format - +Objects/funcobject.c - next_func_version - +Objects/moduleobject.c - max_module_number - +Objects/object.c - _Py_RefTotal - +Objects/typeobject.c - next_version_tag - +Objects/typeobject.c resolve_slotdups ptrs - +Parser/pegen.c - memo_statistics - +# XXX This should have been found by the analyzer but wasn't: +Python/bootstrap_hash.c - urandom_cache - +# XXX This should have been found by the analyzer but wasn't: +Python/ceval.c - lltrace - +# XXX This should have been found by the analyzer but wasn't: +Python/ceval.c make_pending_calls busy - +Python/dynload_shlib.c - handles - +Python/dynload_shlib.c - nhandles - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c - import_lock_level - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c - import_lock_thread - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c import_find_and_load accumulated - +# XXX This should have been found by the analyzer but wasn't: +Python/import.c import_find_and_load import_level - +# XXX This should have been found by the analyzer but wasn't: +Python/pylifecycle.c - _Py_UnhandledKeyboardInterrupt - +# XXX This should have been found by the analyzer but wasn't: +Python/pylifecycle.c fatal_error reentrant - + + +################################## +# global objects to fix in extension modules + +#----------------------- +# modules + +Modules/_abc.c - _abcmodule - +Modules/_asynciomodule.c - _asynciomodule - +Modules/_bisectmodule.c - _bisectmodule - +Modules/_blake2/blake2module.c - blake2_module - +Modules/_bz2module.c - _bz2module - +Modules/_codecsmodule.c - codecsmodule - +Modules/_collectionsmodule.c - _collectionsmodule - +Modules/_contextvarsmodule.c - _contextvarsmodule - +Modules/_cryptmodule.c - cryptmodule - +Modules/_csv.c - _csvmodule - +Modules/_ctypes/_ctypes.c - _ctypesmodule - +Modules/_curses_panel.c - _curses_panelmodule - +Modules/_cursesmodule.c - _cursesmodule - +Modules/_datetimemodule.c - datetimemodule - +Modules/_decimal/_decimal.c - _decimal_module - +Modules/_elementtree.c - elementtreemodule - +Modules/_functoolsmodule.c - _functools_module - +Modules/_gdbmmodule.c - _gdbmmodule - +Modules/_hashopenssl.c - _hashlibmodule - +Modules/_heapqmodule.c - _heapqmodule - +Modules/_io/_iomodule.c - _PyIO_Module - +Modules/_io/_iomodule.h - _PyIO_Module - +Modules/_json.c - jsonmodule - +Modules/_localemodule.c - _localemodule - +Modules/_lsprof.c - _lsprofmodule - +Modules/_lzmamodule.c - _lzmamodule - +Modules/_multiprocessing/multiprocessing.c - multiprocessing_module - +Modules/_multiprocessing/posixshmem.c - this_module - +Modules/_opcode.c - opcodemodule - +Modules/_operator.c - operatormodule - +Modules/_pickle.c - _picklemodule - +Modules/_posixsubprocess.c - _posixsubprocessmodule - +Modules/_queuemodule.c - queuemodule - +Modules/_randommodule.c - _randommodule - +Modules/_sha3/sha3module.c - _sha3module - +Modules/_sqlite/module.c - _sqlite3module - +Modules/_sre.c - sremodule - +Modules/_ssl.c - PySocketModule - +Modules/_ssl.c - _sslmodule - +Modules/_stat.c - statmodule - +Modules/_statisticsmodule.c - statisticsmodule - +Modules/_struct.c - _structmodule - +Modules/_threadmodule.c - threadmodule - +Modules/_tkinter.c - _tkintermodule - +Modules/_tracemalloc.c - module_def - +Modules/_uuidmodule.c - uuidmodule - +Modules/_weakref.c - weakrefmodule - +Modules/_xxsubinterpretersmodule.c - interpretersmodule - +Modules/_zoneinfo.c - zoneinfomodule - +Modules/arraymodule.c - arraymodule - +Modules/atexitmodule.c - atexitmodule - +Modules/audioop.c - audioopmodule - +Modules/binascii.c - binasciimodule - +Modules/cjkcodecs/multibytecodec.c - _multibytecodecmodule - +Modules/cmathmodule.c - cmathmodule - +Modules/errnomodule.c - errnomodule - +Modules/faulthandler.c - module_def - +Modules/fcntlmodule.c - fcntlmodule - +Modules/gcmodule.c - gcmodule - +Modules/grpmodule.c - grpmodule - +Modules/itertoolsmodule.c - itertoolsmodule - +Modules/mathmodule.c - mathmodule - +Modules/md5module.c - _md5module - +Modules/mmapmodule.c - mmapmodule - +Modules/nismodule.c - nismodule - +Modules/ossaudiodev.c - ossaudiodevmodule - +Modules/posixmodule.c - posixmodule - +Modules/pwdmodule.c - pwdmodule - +Modules/pyexpat.c - pyexpatmodule - +Modules/readline.c - readlinemodule - +Modules/resource.c - resourcemodule - +Modules/selectmodule.c - selectmodule - +Modules/sha1module.c - _sha1module - +Modules/sha256module.c - _sha256module - +Modules/sha512module.c - _sha512module - +Modules/signalmodule.c - signalmodule - +Modules/socketmodule.c - socketmodule - +Modules/spwdmodule.c - spwdmodule - +Modules/symtablemodule.c - symtablemodule - +Modules/syslogmodule.c - syslogmodule - +Modules/termios.c - termiosmodule - +Modules/timemodule.c - timemodule - +Modules/unicodedata.c - unicodedata_module - +Modules/xxlimited.c - xxmodule - +Modules/xxmodule.c - xxmodule - +Modules/xxsubtype.c - xxsubtypemodule - +Modules/zlibmodule.c - zlibmodule - +Objects/unicodeobject.c - _string_module - + +#----------------------- +# static types + +Modules/_asynciomodule.c - FutureIterType - +Modules/_asynciomodule.c - FutureType - +Modules/_asynciomodule.c - PyRunningLoopHolder_Type - +Modules/_asynciomodule.c - TaskStepMethWrapper_Type - +Modules/_asynciomodule.c - TaskType - +Modules/_collectionsmodule.c - defdict_type - +Modules/_collectionsmodule.c - deque_type - +Modules/_collectionsmodule.c - dequeiter_type - +Modules/_collectionsmodule.c - dequereviter_type - +Modules/_collectionsmodule.c - tuplegetter_type - +Modules/_csv.c - Dialect_Type - +Modules/_csv.c - Reader_Type - +Modules/_csv.c - Writer_Type - +Modules/_ctypes/_ctypes.c - DictRemover_Type - +Modules/_ctypes/_ctypes.c - PyCArrayType_Type - +Modules/_ctypes/_ctypes.c - PyCArray_Type - +Modules/_ctypes/_ctypes.c - PyCData_Type - +Modules/_ctypes/_ctypes.c - PyCFuncPtrType_Type - +Modules/_ctypes/_ctypes.c - PyCFuncPtr_Type - +Modules/_ctypes/_ctypes.c - PyCPointerType_Type - +Modules/_ctypes/_ctypes.c - PyCPointer_Type - +Modules/_ctypes/_ctypes.c - PyCSimpleType_Type - +Modules/_ctypes/_ctypes.c - PyCStructType_Type - +Modules/_ctypes/_ctypes.c - PyComError_Type - +Modules/_ctypes/_ctypes.c - Simple_Type - +Modules/_ctypes/_ctypes.c - StructParam_Type - +Modules/_ctypes/_ctypes.c - Struct_Type - +Modules/_ctypes/_ctypes.c - UnionType_Type - +Modules/_ctypes/_ctypes.c - Union_Type - +Modules/_ctypes/callbacks.c - PyCThunk_Type - +Modules/_ctypes/callproc.c - PyCArg_Type - +Modules/_ctypes/cfield.c - PyCField_Type - +Modules/_ctypes/stgdict.c - PyCStgDict_Type - +Modules/_cursesmodule.c - PyCursesWindow_Type - +Modules/_datetimemodule.c - PyDateTime_DateTimeType - +Modules/_datetimemodule.c - PyDateTime_DateType - +Modules/_datetimemodule.c - PyDateTime_DeltaType - +Modules/_datetimemodule.c - PyDateTime_IsoCalendarDateType - +Modules/_datetimemodule.c - PyDateTime_TZInfoType - +Modules/_datetimemodule.c - PyDateTime_TimeType - +Modules/_datetimemodule.c - PyDateTime_TimeZoneType - +Modules/_decimal/_decimal.c - PyDecContextManager_Type - +Modules/_decimal/_decimal.c - PyDecContext_Type - +Modules/_decimal/_decimal.c - PyDecSignalDictMixin_Type - +Modules/_decimal/_decimal.c - PyDec_Type - +Modules/_elementtree.c - ElementIter_Type - +Modules/_elementtree.c - Element_Type - +Modules/_elementtree.c - TreeBuilder_Type - +Modules/_elementtree.c - XMLParser_Type - +Modules/_functoolsmodule.c - keyobject_type - +Modules/_functoolsmodule.c - lru_cache_type - +Modules/_functoolsmodule.c - lru_list_elem_type - +Modules/_functoolsmodule.c - partial_type - +Modules/_io/bufferedio.c - PyBufferedIOBase_Type - +Modules/_io/bufferedio.c - PyBufferedRWPair_Type - +Modules/_io/bufferedio.c - PyBufferedRandom_Type - +Modules/_io/bufferedio.c - PyBufferedReader_Type - +Modules/_io/bufferedio.c - PyBufferedWriter_Type - +Modules/_io/bytesio.c - PyBytesIO_Type - +Modules/_io/bytesio.c - _PyBytesIOBuffer_Type - +Modules/_io/fileio.c - PyFileIO_Type - +Modules/_io/iobase.c - PyIOBase_Type - +Modules/_io/iobase.c - PyRawIOBase_Type - +Modules/_io/stringio.c - PyStringIO_Type - +Modules/_io/textio.c - PyIncrementalNewlineDecoder_Type - +Modules/_io/textio.c - PyTextIOBase_Type - +Modules/_io/textio.c - PyTextIOWrapper_Type - +Modules/_io/winconsoleio.c - PyWindowsConsoleIO_Type - +Modules/_multiprocessing/semaphore.c - _PyMp_SemLockType - +Modules/_pickle.c - Pdata_Type - +Modules/_pickle.c - PicklerMemoProxyType - +Modules/_pickle.c - Pickler_Type - +Modules/_pickle.c - UnpicklerMemoProxyType - +Modules/_pickle.c - Unpickler_Type - +Modules/_queuemodule.c - PySimpleQueueType - +Modules/_sre.c - Match_Type - +Modules/_sre.c - Pattern_Type - +Modules/_sre.c - Scanner_Type - +Modules/_ssl.c - PySSLContext_Type - +Modules/_ssl.c - PySSLMemoryBIO_Type - +Modules/_ssl.c - PySSLSession_Type - +Modules/_ssl.c - PySSLSocket_Type - +Modules/_threadmodule.c - Locktype - +Modules/_threadmodule.c - RLocktype - +Modules/_threadmodule.c - localdummytype - +Modules/_threadmodule.c - localtype - +Modules/_xxsubinterpretersmodule.c - ChannelIDtype - +Modules/_zoneinfo.c - PyZoneInfo_ZoneInfoType - +Modules/arraymodule.c - Arraytype - +Modules/arraymodule.c - PyArrayIter_Type - +Modules/cjkcodecs/multibytecodec.c - MultibyteCodec_Type - +Modules/cjkcodecs/multibytecodec.c - MultibyteIncrementalDecoder_Type - +Modules/cjkcodecs/multibytecodec.c - MultibyteIncrementalEncoder_Type - +Modules/cjkcodecs/multibytecodec.c - MultibyteStreamReader_Type - +Modules/cjkcodecs/multibytecodec.c - MultibyteStreamWriter_Type - +Modules/itertoolsmodule.c - _grouper_type - +Modules/itertoolsmodule.c - accumulate_type - +Modules/itertoolsmodule.c - chain_type - +Modules/itertoolsmodule.c - combinations_type - +Modules/itertoolsmodule.c - compress_type - +Modules/itertoolsmodule.c - count_type - +Modules/itertoolsmodule.c - cwr_type - +Modules/itertoolsmodule.c - cycle_type - +Modules/itertoolsmodule.c - dropwhile_type - +Modules/itertoolsmodule.c - filterfalse_type - +Modules/itertoolsmodule.c - groupby_type - +Modules/itertoolsmodule.c - islice_type - +Modules/itertoolsmodule.c - pairwise_type - +Modules/itertoolsmodule.c - permutations_type - +Modules/itertoolsmodule.c - product_type - +Modules/itertoolsmodule.c - repeat_type - +Modules/itertoolsmodule.c - starmap_type - +Modules/itertoolsmodule.c - takewhile_type - +Modules/itertoolsmodule.c - tee_type - +Modules/itertoolsmodule.c - teedataobject_type - +Modules/itertoolsmodule.c - ziplongest_type - +Modules/mmapmodule.c - mmap_object_type - +Modules/ossaudiodev.c - OSSAudioType - +Modules/ossaudiodev.c - OSSMixerType - +Modules/pyexpat.c - Xmlparsetype - +Modules/socketmodule.c - sock_type - +Modules/xxlimited_35.c - Xxo_Type - +Modules/xxmodule.c - Null_Type - +Modules/xxmodule.c - Str_Type - +Modules/xxmodule.c - Xxo_Type - +Modules/xxsubtype.c - spamdict_type - +Modules/xxsubtype.c - spamlist_type - + +#----------------------- +# non-static types - initialized once + +# structseq types +Modules/_cursesmodule.c - NcursesVersionType - +Modules/_threadmodule.c - ExceptHookArgsType - +Modules/resource.c - StructRUsageType - +Modules/signalmodule.c - SiginfoType - +Modules/spwdmodule.c - StructSpwdType - +Modules/timemodule.c - StructTimeType - + +# heap types +Modules/_decimal/_decimal.c - DecimalTuple - +Modules/_decimal/_decimal.c - PyDecSignalDict_Type - +Modules/_tkinter.c - PyTclObject_Type - +Modules/_tkinter.c - Tkapp_Type - +Modules/_tkinter.c - Tktt_Type - +Modules/xxlimited.c - Xxo_Type - + +# exception types +Modules/_ctypes/_ctypes.c - PyExc_ArgError - +Modules/_cursesmodule.c - PyCursesError - +Modules/_decimal/_decimal.c - DecimalException - +Modules/_queuemodule.c - EmptyError - +Modules/_ssl.c - PySSLErrorObject - +Modules/_ssl.c - PySSLCertVerificationErrorObject - +Modules/_ssl.c - PySSLZeroReturnErrorObject - +Modules/_ssl.c - PySSLWantReadErrorObject - +Modules/_ssl.c - PySSLWantWriteErrorObject - +Modules/_ssl.c - PySSLSyscallErrorObject - +Modules/_ssl.c - PySSLEOFErrorObject - +Modules/_threadmodule.c - ThreadError - +Modules/_tkinter.c - Tkinter_TclError - +Modules/_xxsubinterpretersmodule.c - ChannelError - +Modules/_xxsubinterpretersmodule.c - ChannelNotFoundError - +Modules/_xxsubinterpretersmodule.c - ChannelClosedError - +Modules/_xxsubinterpretersmodule.c - ChannelEmptyError - +Modules/_xxsubinterpretersmodule.c - ChannelNotEmptyError - +Modules/_xxsubinterpretersmodule.c - RunFailedError - +Modules/ossaudiodev.c - OSSAudioError - +Modules/pyexpat.c - ErrorObject - +Modules/signalmodule.c - ItimerError - +Modules/socketmodule.c - socket_herror - +Modules/socketmodule.c - socket_gaierror - +Modules/socketmodule.c - socket_timeout - +Modules/xxlimited.c - ErrorObject - +Modules/xxmodule.c - ErrorObject - + +#----------------------- +# cached PyUnicodeObject - initialized once + +# _Py_IDENTIFIER (global) +Modules/_abc.c - PyId___abc_tpflags__ - +Modules/_abc.c - PyId___abstractmethods__ - +Modules/_abc.c - PyId___bases__ - +Modules/_abc.c - PyId___class__ - +Modules/_abc.c - PyId___dict__ - +Modules/_abc.c - PyId___subclasscheck__ - +Modules/_abc.c - PyId___subclasshook__ - +Modules/_abc.c - PyId__abc_impl - +Modules/_asynciomodule.c - PyId___asyncio_running_event_loop__ - +Modules/_asynciomodule.c - PyId__asyncio_future_blocking - +Modules/_asynciomodule.c - PyId_add_done_callback - +Modules/_asynciomodule.c - PyId_call_soon - +Modules/_asynciomodule.c - PyId_cancel - +Modules/_asynciomodule.c - PyId_get_event_loop - +Modules/_asynciomodule.c - PyId_throw - +Modules/_bisectmodule.c - PyId_insert - +Modules/_datetimemodule.c - PyId_as_integer_ratio - +Modules/_datetimemodule.c - PyId_fromutc - +Modules/_datetimemodule.c - PyId_isoformat - +Modules/_datetimemodule.c - PyId_strftime - +Modules/_io/bufferedio.c - PyId__dealloc_warn - +Modules/_io/bufferedio.c - PyId_close - +Modules/_io/bufferedio.c - PyId_flush - +Modules/_io/bufferedio.c - PyId_isatty - +Modules/_io/bufferedio.c - PyId_mode - +Modules/_io/bufferedio.c - PyId_name - +Modules/_io/bufferedio.c - PyId_peek - +Modules/_io/bufferedio.c - PyId_read - +Modules/_io/bufferedio.c - PyId_read1 - +Modules/_io/bufferedio.c - PyId_readable - +Modules/_io/bufferedio.c - PyId_readinto - +Modules/_io/bufferedio.c - PyId_readinto1 - +Modules/_io/bufferedio.c - PyId_writable - +Modules/_io/bufferedio.c - PyId_write - +Modules/_io/fileio.c - PyId_name - +Modules/_io/iobase.c - PyId___IOBase_closed - +Modules/_io/iobase.c - PyId_read - +Modules/_io/textio.c - PyId__dealloc_warn - +Modules/_io/textio.c - PyId_close - +Modules/_io/textio.c - PyId_decode - +Modules/_io/textio.c - PyId_fileno - +Modules/_io/textio.c - PyId_flush - +Modules/_io/textio.c - PyId_getpreferredencoding - +Modules/_io/textio.c - PyId_isatty - +Modules/_io/textio.c - PyId_mode - +Modules/_io/textio.c - PyId_name - +Modules/_io/textio.c - PyId_raw - +Modules/_io/textio.c - PyId_read - +Modules/_io/textio.c - PyId_readable - +Modules/_io/textio.c - PyId_replace - +Modules/_io/textio.c - PyId_reset - +Modules/_io/textio.c - PyId_seek - +Modules/_io/textio.c - PyId_seekable - +Modules/_io/textio.c - PyId_setstate - +Modules/_io/textio.c - PyId_strict - +Modules/_io/textio.c - PyId_tell - +Modules/_io/textio.c - PyId_writable - +Modules/_sqlite/connection.c - PyId_cursor - +Modules/_threadmodule.c - PyId___dict__ - +Modules/_threadmodule.c - PyId_flush - +Modules/_threadmodule.c - PyId_stderr - +Modules/cjkcodecs/multibytecodec.c - PyId_write - +Modules/faulthandler.c - PyId_enable - +Modules/faulthandler.c - PyId_fileno - +Modules/faulthandler.c - PyId_flush - +Modules/faulthandler.c - PyId_stderr - +Modules/posixmodule.c - PyId___fspath__ - +Modules/unicodedata.c - PyId_NFC - +Modules/unicodedata.c - PyId_NFD - +Modules/unicodedata.c - PyId_NFKC - +Modules/unicodedata.c - PyId_NFKD - + +# _Py_IDENTIFIER (local) +Modules/_json.c _encoded_const PyId_false - +Modules/_json.c _encoded_const PyId_null - +Modules/_json.c _encoded_const PyId_true - +Modules/_json.c encoder_listencode_dict PyId_close_dict - +Modules/_json.c encoder_listencode_dict PyId_empty_dict - +Modules/_json.c encoder_listencode_dict PyId_open_dict - +Modules/_json.c encoder_listencode_list PyId_close_array - +Modules/_json.c encoder_listencode_list PyId_empty_array - +Modules/_json.c encoder_listencode_list PyId_open_array - +Modules/_json.c raise_errmsg PyId_JSONDecodeError - +Modules/_json.c raise_errmsg PyId_decoder - +Modules/_sqlite/connection.c final_callback PyId_finalize - +Modules/_sqlite/connection.c pysqlite_connection_execute_impl PyId_execute - +Modules/_sqlite/connection.c pysqlite_connection_executemany_impl PyId_executemany - +Modules/_sqlite/connection.c pysqlite_connection_executescript PyId_executescript - +Modules/_sqlite/connection.c pysqlite_connection_iterdump_impl PyId__iterdump - +Modules/_sqlite/module.c pysqlite_register_converter_impl PyId_upper - +Modules/pyexpat.c pyexpat_xmlparser_ParseFile_impl PyId_read - +Modules/_asynciomodule.c FutureObj_finalize PyId_call_exception_handler - +Modules/_asynciomodule.c FutureObj_finalize PyId_exception - +Modules/_asynciomodule.c FutureObj_finalize PyId_future - +Modules/_asynciomodule.c FutureObj_finalize PyId_message - +Modules/_asynciomodule.c FutureObj_finalize PyId_source_traceback - +Modules/_asynciomodule.c FutureObj_get_state PyId_CANCELLED - +Modules/_asynciomodule.c FutureObj_get_state PyId_FINISHED - +Modules/_asynciomodule.c FutureObj_get_state PyId_PENDING - +Modules/_asynciomodule.c FutureObj_repr PyId__repr_info - +Modules/_asynciomodule.c TaskObj_finalize PyId_call_exception_handler - +Modules/_asynciomodule.c TaskObj_finalize PyId_message - +Modules/_asynciomodule.c TaskObj_finalize PyId_source_traceback - +Modules/_asynciomodule.c TaskObj_finalize PyId_task - +Modules/_asynciomodule.c future_init PyId_get_debug - +Modules/_asynciomodule.c get_future_loop PyId__loop - +Modules/_asynciomodule.c get_future_loop PyId_get_loop - +Modules/_asynciomodule.c register_task PyId_add - +Modules/_asynciomodule.c unregister_task PyId_discard - +Modules/_collectionsmodule.c _collections__count_elements_impl PyId___setitem__ - +Modules/_collectionsmodule.c _collections__count_elements_impl PyId_get - +Modules/_collectionsmodule.c defdict_reduce PyId_items - +Modules/_collectionsmodule.c deque_reduce PyId___dict__ - +Modules/_csv.c csv_writer PyId_write - +Modules/_ctypes/_ctypes.c CDataType_from_param PyId__as_parameter_ - +Modules/_ctypes/_ctypes.c PyCArrayType_new PyId__length_ - +Modules/_ctypes/_ctypes.c PyCArrayType_new PyId__type_ - +Modules/_ctypes/_ctypes.c PyCFuncPtr_set_restype PyId__check_retval_ - +Modules/_ctypes/_ctypes.c PyCPointerType_new PyId__type_ - +Modules/_ctypes/_ctypes.c PyCPointerType_set_type PyId__type_ - +Modules/_ctypes/_ctypes.c PyCSimpleType_from_param PyId__as_parameter_ - +Modules/_ctypes/_ctypes.c PyCSimpleType_new PyId__type_ - +Modules/_ctypes/_ctypes.c StructUnionType_new PyId__abstract_ - +Modules/_ctypes/_ctypes.c StructUnionType_new PyId__fields_ - +Modules/_ctypes/_ctypes.c _build_result PyId___ctypes_from_outparam__ - +Modules/_ctypes/_ctypes.c _init_pos_args PyId__fields_ - +Modules/_ctypes/_ctypes.c c_char_p_from_param PyId__as_parameter_ - +Modules/_ctypes/_ctypes.c c_void_p_from_param PyId__as_parameter_ - +Modules/_ctypes/_ctypes.c c_wchar_p_from_param PyId__as_parameter_ - +Modules/_ctypes/_ctypes.c converters_from_argtypes PyId_from_param - +Modules/_ctypes/_ctypes.c make_funcptrtype_dict PyId__argtypes_ - +Modules/_ctypes/_ctypes.c make_funcptrtype_dict PyId__check_retval_ - +Modules/_ctypes/_ctypes.c make_funcptrtype_dict PyId__flags_ - +Modules/_ctypes/_ctypes.c make_funcptrtype_dict PyId__restype_ - +Modules/_ctypes/callproc.c ConvParam PyId__as_parameter_ - +Modules/_ctypes/callproc.c unpickle PyId___new__ - +Modules/_ctypes/callproc.c unpickle PyId___setstate__ - +Modules/_ctypes/stgdict.c MakeAnonFields PyId__anonymous_ - +Modules/_ctypes/stgdict.c PyCStructUnionType_update_stgdict PyId__pack_ - +Modules/_ctypes/stgdict.c PyCStructUnionType_update_stgdict PyId__swappedbytes_ - +Modules/_ctypes/stgdict.c PyCStructUnionType_update_stgdict PyId__use_broken_old_ctypes_structure_semantics_ - +Modules/_cursesmodule.c _curses_getwin PyId_read - +Modules/_cursesmodule.c _curses_window_putwin PyId_write - +Modules/_cursesmodule.c update_lines_cols PyId_COLS - +Modules/_cursesmodule.c update_lines_cols PyId_LINES - +Modules/_datetimemodule.c build_struct_time PyId_struct_time - +Modules/_datetimemodule.c call_tzname PyId_tzname - +Modules/_datetimemodule.c date_strftime PyId_timetuple - +Modules/_datetimemodule.c date_today PyId_fromtimestamp - +Modules/_datetimemodule.c datetime_strptime PyId__strptime_datetime - +Modules/_datetimemodule.c make_Zreplacement PyId_replace - +Modules/_datetimemodule.c time_time PyId_time - +Modules/_datetimemodule.c tzinfo_reduce PyId___getinitargs__ - +Modules/_datetimemodule.c tzinfo_reduce PyId___getstate__ - +Modules/_elementtree.c _elementtree_Element_find_impl PyId_find - +Modules/_elementtree.c _elementtree_Element_findall_impl PyId_findall - +Modules/_elementtree.c _elementtree_Element_findtext_impl PyId_findtext - +Modules/_elementtree.c _elementtree_Element_iterfind_impl PyId_iterfind - +Modules/_elementtree.c expat_start_doctype_handler PyId_doctype - +Modules/_elementtree.c treebuilder_add_subelement PyId_append - +Modules/_elementtree.c treebuilder_flush_data PyId_tail - +Modules/_elementtree.c treebuilder_flush_data PyId_text - +Modules/_gdbmmodule.c gdbm__exit__ PyId_close - +Modules/_io/_iomodule.c _io_open_impl PyId__blksize - +Modules/_io/_iomodule.c _io_open_impl PyId_close - +Modules/_io/_iomodule.c _io_open_impl PyId_isatty - +Modules/_io/_iomodule.c _io_open_impl PyId_mode - +Modules/_io/fileio.c _io_FileIO_close_impl PyId_close - +Modules/_io/iobase.c _io__IOBase_readlines_impl PyId_extend - +Modules/_io/iobase.c _io__IOBase_tell_impl PyId_seek - +Modules/_io/iobase.c _io__RawIOBase_read_impl PyId_readall - +Modules/_io/iobase.c iobase_finalize PyId__finalizing - +Modules/_lzmamodule.c build_filter_spec PyId_dict_size - +Modules/_lzmamodule.c build_filter_spec PyId_dist - +Modules/_lzmamodule.c build_filter_spec PyId_id - +Modules/_lzmamodule.c build_filter_spec PyId_lc - +Modules/_lzmamodule.c build_filter_spec PyId_lp - +Modules/_lzmamodule.c build_filter_spec PyId_pb - +Modules/_lzmamodule.c build_filter_spec PyId_start_offset - +Modules/_operator.c methodcaller_reduce PyId_partial - +Modules/_pickle.c _Pickle_InitState PyId_getattr - +Modules/_pickle.c _Pickler_SetOutputStream PyId_write - +Modules/_pickle.c _Unpickler_SetInputStream PyId_peek - +Modules/_pickle.c _Unpickler_SetInputStream PyId_read - +Modules/_pickle.c _Unpickler_SetInputStream PyId_readinto - +Modules/_pickle.c _Unpickler_SetInputStream PyId_readline - +Modules/_pickle.c _pickle_Pickler___init___impl PyId_dispatch_table - +Modules/_pickle.c _pickle_Pickler___init___impl PyId_persistent_id - +Modules/_pickle.c _pickle_Unpickler___init___impl PyId_persistent_load - +Modules/_pickle.c do_append PyId_append - +Modules/_pickle.c do_append PyId_extend - +Modules/_pickle.c dump PyId_reducer_override - +Modules/_pickle.c find_class PyId_find_class - +Modules/_pickle.c get_class PyId___class__ - +Modules/_pickle.c instantiate PyId___getinitargs__ - +Modules/_pickle.c instantiate PyId___new__ - +Modules/_pickle.c load_additems PyId_add - +Modules/_pickle.c load_build PyId___dict__ - +Modules/_pickle.c load_build PyId___setstate__ - +Modules/_pickle.c save PyId___reduce__ - +Modules/_pickle.c save PyId___reduce_ex__ - +Modules/_pickle.c save_bytes PyId_latin1 - +Modules/_pickle.c save_dict PyId_items - +Modules/_pickle.c save_global PyId___name__ - +Modules/_pickle.c save_global PyId___qualname__ - +Modules/_pickle.c save_reduce PyId___name__ - +Modules/_pickle.c save_reduce PyId___new__ - +Modules/_pickle.c save_reduce PyId___newobj__ - +Modules/_pickle.c save_reduce PyId___newobj_ex__ - +Modules/_pickle.c whichmodule PyId___main__ - +Modules/_pickle.c whichmodule PyId___module__ - +Modules/_pickle.c whichmodule PyId_modules - +Modules/_sqlite/connection.c _pysqlite_final_callback PyId_finalize - +Modules/_sqlite/connection.c pysqlite_connection_create_collation PyId_upper - +Modules/_sqlite/connection.c pysqlite_connection_iterdump PyId__iterdump - +Modules/_sqlite/connection.c pysqlite_connection_set_isolation_level PyId_upper - +Modules/_sqlite/cursor.c _pysqlite_get_converter PyId_upper - +Modules/_sqlite/microprotocols.c pysqlite_microprotocols_adapt PyId___adapt__ - +Modules/_sqlite/microprotocols.c pysqlite_microprotocols_adapt PyId___conform__ - +Modules/_sqlite/module.c module_register_converter PyId_upper - +Modules/_ssl.c fill_and_set_sslerror PyId_library - +Modules/_ssl.c fill_and_set_sslerror PyId_reason - +Modules/_ssl.c fill_and_set_sslerror PyId_verify_code - +Modules/_ssl.c fill_and_set_sslerror PyId_verify_message - +Modules/_threadmodule.c thread_excepthook_file PyId_name - +Modules/arraymodule.c array_array___reduce_ex__ PyId___dict__ - +Modules/arraymodule.c array_array___reduce_ex__ PyId__array_reconstructor - +Modules/arraymodule.c array_array_fromfile_impl PyId_read - +Modules/arraymodule.c array_array_tofile PyId_write - +Modules/arraymodule.c array_arrayiterator___reduce___impl PyId_iter - +Modules/itertoolsmodule.c _grouper_reduce PyId_iter - +Modules/itertoolsmodule.c cycle_reduce PyId___setstate__ - +Modules/itertoolsmodule.c itertools_tee_impl PyId___copy__ - +Modules/itertoolsmodule.c zip_longest_new PyId_fillvalue - +Modules/main.c pymain_sys_path_add_path0 PyId_path - +Modules/mathmodule.c math_ceil PyId___ceil__ - +Modules/mathmodule.c math_floor PyId___floor__ - +Modules/mathmodule.c math_trunc PyId___trunc__ - +Modules/mmapmodule.c mmap__exit__method PyId_close - +Modules/ossaudiodev.c oss_exit PyId_close - +Modules/pyexpat.c pyexpat_xmlparser_ParseFile PyId_read - +Modules/timemodule.c time_strptime PyId__strptime_time - + +# _Py_static_string +Modules/_pickle.c get_dotted_path PyId_dot - + +# manually cached PyUnicodeOjbect +Modules/_asynciomodule.c - context_kwname - +Modules/_ctypes/callproc.c _ctypes_get_errobj error_object_name - +Modules/_ctypes/_ctypes.c CreateSwappedType suffix - +Modules/_io/_iomodule.c - _PyIO_str_close - +Modules/_io/_iomodule.c - _PyIO_str_closed - +Modules/_io/_iomodule.c - _PyIO_str_decode - +Modules/_io/_iomodule.c - _PyIO_str_encode - +Modules/_io/_iomodule.c - _PyIO_str_fileno - +Modules/_io/_iomodule.c - _PyIO_str_flush - +Modules/_io/_iomodule.c - _PyIO_str_getstate - +Modules/_io/_iomodule.c - _PyIO_str_isatty - +Modules/_io/_iomodule.c - _PyIO_str_locale - +Modules/_io/_iomodule.c - _PyIO_str_newlines - +Modules/_io/_iomodule.c - _PyIO_str_nl - +Modules/_io/_iomodule.c - _PyIO_str_peek - +Modules/_io/_iomodule.c - _PyIO_str_read - +Modules/_io/_iomodule.c - _PyIO_str_read1 - +Modules/_io/_iomodule.c - _PyIO_str_readable - +Modules/_io/_iomodule.c - _PyIO_str_readall - +Modules/_io/_iomodule.c - _PyIO_str_readinto - +Modules/_io/_iomodule.c - _PyIO_str_readline - +Modules/_io/_iomodule.c - _PyIO_str_reset - +Modules/_io/_iomodule.c - _PyIO_str_seek - +Modules/_io/_iomodule.c - _PyIO_str_seekable - +Modules/_io/_iomodule.c - _PyIO_str_setstate - +Modules/_io/_iomodule.c - _PyIO_str_tell - +Modules/_io/_iomodule.c - _PyIO_str_truncate - +Modules/_io/_iomodule.c - _PyIO_str_writable - +Modules/_io/_iomodule.c - _PyIO_str_write - +Modules/_io/_iomodule.c - _PyIO_empty_str - +Modules/_json.c _encoded_const s_null - +Modules/_json.c _encoded_const s_true - +Modules/_json.c _encoded_const s_false - +Modules/_json.c encoder_listencode_dict open_dict - +Modules/_json.c encoder_listencode_dict close_dict - +Modules/_json.c encoder_listencode_dict empty_dict - +Modules/_json.c encoder_listencode_list open_array - +Modules/_json.c encoder_listencode_list close_array - +Modules/_json.c encoder_listencode_list empty_array - +Modules/_threadmodule.c - str_dict - +Modules/_tracemalloc.c - unknown_filename - + +#----------------------- +# _PyArg_Parser + +Modules/clinic/_asynciomodule.c.h _asyncio_Future___init__ _parser - +Modules/clinic/_asynciomodule.c.h _asyncio_Future_add_done_callback _parser - +Modules/clinic/_asynciomodule.c.h _asyncio_Future_cancel _parser - +Modules/clinic/_asynciomodule.c.h _asyncio_Task___init__ _parser - +Modules/clinic/_asynciomodule.c.h _asyncio_Task_cancel _parser - +Modules/clinic/_asynciomodule.c.h _asyncio_Task_get_stack _parser - +Modules/clinic/_asynciomodule.c.h _asyncio_Task_print_stack _parser - +Modules/clinic/_asynciomodule.c.h _asyncio__enter_task _parser - +Modules/clinic/_asynciomodule.c.h _asyncio__get_event_loop _parser - +Modules/clinic/_asynciomodule.c.h _asyncio__leave_task _parser - +Modules/clinic/_asynciomodule.c.h _asyncio__register_task _parser - +Modules/clinic/_asynciomodule.c.h _asyncio__unregister_task _parser - +Modules/clinic/_bisectmodule.c.h _bisect_bisect_left _parser - +Modules/clinic/_bisectmodule.c.h _bisect_bisect_right _parser - +Modules/clinic/_bisectmodule.c.h _bisect_insort_left _parser - +Modules/clinic/_bisectmodule.c.h _bisect_insort_right _parser - +Modules/clinic/_bz2module.c.h _bz2_BZ2Decompressor_decompress _parser - +Modules/clinic/_codecsmodule.c.h _codecs_decode _parser - +Modules/clinic/_codecsmodule.c.h _codecs_encode _parser - +Modules/clinic/_curses_panel.c.h _curses_panel_panel_bottom _parser - +Modules/clinic/_curses_panel.c.h _curses_panel_panel_hide _parser - +Modules/clinic/_curses_panel.c.h _curses_panel_panel_move _parser - +Modules/clinic/_curses_panel.c.h _curses_panel_panel_replace _parser - +Modules/clinic/_curses_panel.c.h _curses_panel_panel_set_userptr _parser - +Modules/clinic/_curses_panel.c.h _curses_panel_panel_show _parser - +Modules/clinic/_curses_panel.c.h _curses_panel_panel_top _parser - +Modules/clinic/_curses_panel.c.h _curses_panel_panel_userptr _parser - +Modules/clinic/_cursesmodule.c.h _curses_setupterm _parser - +Modules/clinic/_datetimemodule.c.h datetime_datetime_now _parser - +Modules/clinic/_datetimemodule.c.h iso_calendar_date_new _parser - +Modules/clinic/_dbmmodule.c.h _dbm_dbm_get _parser - +Modules/clinic/_dbmmodule.c.h _dbm_dbm_keys _parser - +Modules/clinic/_dbmmodule.c.h _dbm_dbm_setdefault _parser - +Modules/clinic/_elementtree.c.h _elementtree_Element_find _parser - +Modules/clinic/_elementtree.c.h _elementtree_Element_findall _parser - +Modules/clinic/_elementtree.c.h _elementtree_Element_findtext _parser - +Modules/clinic/_elementtree.c.h _elementtree_Element_get _parser - +Modules/clinic/_elementtree.c.h _elementtree_Element_iter _parser - +Modules/clinic/_elementtree.c.h _elementtree_Element_iterfind _parser - +Modules/clinic/_elementtree.c.h _elementtree_TreeBuilder___init__ _parser - +Modules/clinic/_elementtree.c.h _elementtree_XMLParser___init__ _parser - +Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_firstkey _parser - +Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_keys _parser - +Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_nextkey _parser - +Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_reorganize _parser - +Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_sync _parser - +Modules/clinic/_hashopenssl.c.h EVP_new _parser - +Modules/clinic/_hashopenssl.c.h _hashlib_HMAC_update _parser - +Modules/clinic/_hashopenssl.c.h _hashlib_hmac_new _parser - +Modules/clinic/_hashopenssl.c.h _hashlib_hmac_singleshot _parser - +Modules/clinic/_hashopenssl.c.h _hashlib_openssl_md5 _parser - +Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha1 _parser - +Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha224 _parser - +Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha256 _parser - +Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha384 _parser - +Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha512 _parser - +Modules/clinic/_hashopenssl.c.h pbkdf2_hmac _parser - +Modules/clinic/_lsprof.c.h _lsprof_Profiler_getstats _parser - +Modules/clinic/_lzmamodule.c.h _lzma_LZMADecompressor___init__ _parser - +Modules/clinic/_lzmamodule.c.h _lzma_LZMADecompressor_decompress _parser - +Modules/clinic/_opcode.c.h _opcode_stack_effect _parser - +Modules/clinic/_pickle.c.h _pickle_Pickler___init__ _parser - +Modules/clinic/_pickle.c.h _pickle_Unpickler___init__ _parser - +Modules/clinic/_pickle.c.h _pickle_dump _parser - +Modules/clinic/_pickle.c.h _pickle_dumps _parser - +Modules/clinic/_pickle.c.h _pickle_load _parser - +Modules/clinic/_pickle.c.h _pickle_loads _parser - +Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_get _parser - +Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_get_nowait _parser - +Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_put _parser - +Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_put_nowait _parser - +Modules/clinic/_sre.c.h _sre_SRE_Match_expand _parser - +Modules/clinic/_sre.c.h _sre_SRE_Match_groupdict _parser - +Modules/clinic/_sre.c.h _sre_SRE_Match_groups _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_findall _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_finditer _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_fullmatch _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_match _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_scanner _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_search _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_split _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_sub _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_subn _parser - +Modules/clinic/_sre.c.h _sre_SRE_Scanner_match _parser - +Modules/clinic/_sre.c.h _sre_SRE_Scanner_search _parser - +Modules/clinic/_sre.c.h _sre_compile _parser - +Modules/clinic/_ssl.c.h _ssl__SSLContext__wrap_bio _parser - +Modules/clinic/_ssl.c.h _ssl__SSLContext__wrap_socket _parser - +Modules/clinic/_ssl.c.h _ssl__SSLContext_get_ca_certs _parser - +Modules/clinic/_ssl.c.h _ssl__SSLContext_load_cert_chain _parser - +Modules/clinic/_ssl.c.h _ssl__SSLContext_load_verify_locations _parser - +Modules/clinic/_ssl.c.h _ssl__SSLSocket_get_channel_binding _parser - +Modules/clinic/_ssl.c.h _ssl_txt2obj _parser - +Modules/clinic/_struct.c.h Struct___init__ _parser - +Modules/clinic/_struct.c.h Struct_unpack_from _parser - +Modules/clinic/_struct.c.h unpack_from _parser - +Modules/clinic/_testmultiphase.c.h _testmultiphase_StateAccessType_get_count _parser - +Modules/clinic/_testmultiphase.c.h _testmultiphase_StateAccessType_get_defining_module _parser - +Modules/clinic/_testmultiphase.c.h _testmultiphase_StateAccessType_getmodulebydef_bad_def _parser - +Modules/clinic/_testmultiphase.c.h _testmultiphase_StateAccessType_increment_count_clinic _parser - +Modules/clinic/_winapi.c.h _winapi_ConnectNamedPipe _parser - +Modules/clinic/_winapi.c.h _winapi_GetFileType _parser - +Modules/clinic/_winapi.c.h _winapi_ReadFile _parser - +Modules/clinic/_winapi.c.h _winapi_WriteFile _parser - +Modules/clinic/_winapi.c.h _winapi__mimetypes_read_windows_registry _parser - +Modules/clinic/arraymodule.c.h array_array_extend _parser - +Modules/clinic/binascii.c.h binascii_a2b_base64 _parser - +Modules/clinic/binascii.c.h binascii_a2b_qp _parser - +Modules/clinic/binascii.c.h binascii_b2a_base64 _parser - +Modules/clinic/binascii.c.h binascii_b2a_hex _parser - +Modules/clinic/binascii.c.h binascii_b2a_qp _parser - +Modules/clinic/binascii.c.h binascii_b2a_uu _parser - +Modules/clinic/binascii.c.h binascii_hexlify _parser - +Modules/clinic/cmathmodule.c.h cmath_isclose _parser - +Modules/clinic/gcmodule.c.h gc_collect _parser - +Modules/clinic/gcmodule.c.h gc_get_objects _parser - +Modules/clinic/grpmodule.c.h grp_getgrgid _parser - +Modules/clinic/grpmodule.c.h grp_getgrnam _parser - +Modules/clinic/itertoolsmodule.c.h itertools_accumulate _parser - +Modules/clinic/itertoolsmodule.c.h itertools_combinations _parser - +Modules/clinic/itertoolsmodule.c.h itertools_combinations_with_replacement _parser - +Modules/clinic/itertoolsmodule.c.h itertools_compress _parser - +Modules/clinic/itertoolsmodule.c.h itertools_count _parser - +Modules/clinic/itertoolsmodule.c.h itertools_groupby _parser - +Modules/clinic/itertoolsmodule.c.h itertools_permutations _parser - +Modules/clinic/mathmodule.c.h math_isclose _parser - +Modules/clinic/mathmodule.c.h math_prod _parser - +Modules/clinic/md5module.c.h MD5Type_copy _parser - +Modules/clinic/md5module.c.h _md5_md5 _parser - +Modules/clinic/overlapped.c.h _overlapped_Overlapped _parser - +Modules/clinic/posixmodule.c.h os_DirEntry_is_dir _parser - +Modules/clinic/posixmodule.c.h os_DirEntry_is_file _parser - +Modules/clinic/posixmodule.c.h os_DirEntry_is_symlink _parser - +Modules/clinic/posixmodule.c.h os_DirEntry_stat _parser - +Modules/clinic/posixmodule.c.h os__exit _parser - +Modules/clinic/posixmodule.c.h os__path_normpath _parser - +Modules/clinic/posixmodule.c.h os_access _parser - +Modules/clinic/posixmodule.c.h os_chdir _parser - +Modules/clinic/posixmodule.c.h os_chmod _parser - +Modules/clinic/posixmodule.c.h os_close _parser - +Modules/clinic/posixmodule.c.h os_device_encoding _parser - +Modules/clinic/posixmodule.c.h os_dup2 _parser - +Modules/clinic/posixmodule.c.h os_fspath _parser - +Modules/clinic/posixmodule.c.h os_fstat _parser - +Modules/clinic/posixmodule.c.h os_listdir _parser - +Modules/clinic/posixmodule.c.h os_lstat _parser - +Modules/clinic/posixmodule.c.h os_mkdir _parser - +Modules/clinic/posixmodule.c.h os_open _parser - +Modules/clinic/posixmodule.c.h os_remove _parser - +Modules/clinic/posixmodule.c.h os_rename _parser - +Modules/clinic/posixmodule.c.h os_replace _parser - +Modules/clinic/posixmodule.c.h os_rmdir _parser - +Modules/clinic/posixmodule.c.h os_scandir _parser - +Modules/clinic/posixmodule.c.h os_stat _parser - +Modules/clinic/posixmodule.c.h os_unlink _parser - +Modules/clinic/posixmodule.c.h os_utime _parser - +Modules/clinic/pyexpat.c.h pyexpat_ParserCreate _parser - +Modules/clinic/pyexpat.c.h pyexpat_xmlparser_ExternalEntityParserCreate _parser - +Modules/clinic/pyexpat.c.h pyexpat_xmlparser_Parse _parser - +Modules/clinic/pyexpat.c.h pyexpat_xmlparser_ParseFile _parser - +Modules/clinic/sha1module.c.h SHA1Type_copy _parser - +Modules/clinic/sha1module.c.h _sha1_sha1 _parser - +Modules/clinic/sha256module.c.h SHA256Type_copy _parser - +Modules/clinic/sha256module.c.h _sha256_sha224 _parser - +Modules/clinic/sha256module.c.h _sha256_sha256 _parser - +Modules/clinic/sha512module.c.h SHA512Type_copy _parser - +Modules/clinic/sha512module.c.h _sha512_sha384 _parser - +Modules/clinic/sha512module.c.h _sha512_sha512 _parser - +Modules/clinic/zlibmodule.c.h zlib_Compress_compress _parser - +Modules/clinic/zlibmodule.c.h zlib_Compress_flush _parser - +Modules/clinic/zlibmodule.c.h zlib_Decompress_decompress _parser - +Modules/clinic/zlibmodule.c.h zlib_Decompress_flush _parser - +Modules/clinic/zlibmodule.c.h zlib_compress _parser - +Modules/clinic/zlibmodule.c.h zlib_compressobj _parser - +Modules/clinic/zlibmodule.c.h zlib_decompress _parser - +Modules/clinic/zlibmodule.c.h zlib_decompressobj _parser - + +#----------------------- +# other + +# cached during module init - initialized once +Modules/_asynciomodule.c - asyncio_mod - +Modules/_asynciomodule.c - traceback_extract_stack - +Modules/_asynciomodule.c - asyncio_get_event_loop_policy - +Modules/_asynciomodule.c - asyncio_future_repr_info_func - +Modules/_asynciomodule.c - asyncio_iscoroutine_func - +Modules/_asynciomodule.c - asyncio_task_get_stack_func - +Modules/_asynciomodule.c - asyncio_task_print_stack_func - +Modules/_asynciomodule.c - asyncio_task_repr_info_func - +Modules/_asynciomodule.c - asyncio_InvalidStateError - +Modules/_asynciomodule.c - asyncio_CancelledError - +Modules/_zoneinfo.c - io_open - +Modules/_zoneinfo.c - _tzpath_find_tzfile - +Modules/_zoneinfo.c - _common_mod - + +# other - initialized once +Modules/_ctypes/_ctypes.c - _unpickle - +Modules/_ctypes/_ctypes.c PyCArrayType_from_ctype cache - +Modules/_cursesmodule.c - ModDict - +Modules/_datetimemodule.c datetime_strptime module - +Modules/_datetimemodule.c - PyDateTime_TimeZone_UTC - +Modules/_datetimemodule.c - PyDateTime_Epoch - +Modules/_datetimemodule.c - us_per_ms - +Modules/_datetimemodule.c - us_per_second - +Modules/_datetimemodule.c - us_per_minute - +Modules/_datetimemodule.c - us_per_hour - +Modules/_datetimemodule.c - us_per_day - +Modules/_datetimemodule.c - us_per_week - +Modules/_datetimemodule.c - seconds_per_day - +Modules/_decimal/_decimal.c PyInit__decimal capsule - +Modules/_decimal/_decimal.c - basic_context_template - +Modules/_decimal/_decimal.c - current_context_var - +Modules/_decimal/_decimal.c - default_context_template - +Modules/_decimal/_decimal.c - extended_context_template - +Modules/_decimal/_decimal.c - round_map - +Modules/_decimal/_decimal.c - Rational - +Modules/_decimal/_decimal.c - SignalTuple - +Modules/_functoolsmodule.c - kwd_mark - +Modules/_io/_iomodule.c - _PyIO_empty_bytes - +Modules/_json.c raise_errmsg JSONDecodeError - +Modules/_sqlite/microprotocols.c - psyco_adapters - +Modules/_sqlite/module.h - _pysqlite_converters - +Modules/_ssl.c - err_codes_to_names - +Modules/_ssl.c - err_names_to_codes - +Modules/_ssl.c - lib_codes_to_names - +# XXX This should have been found by the analyzer but wasn't: +Modules/_ssl.c - _ssl_locks - +Modules/_struct.c - cache - +Modules/_tracemalloc.c - tracemalloc_empty_traceback - +Modules/arraymodule.c array_array___reduce_ex__ array_reconstructor - +Modules/cjkcodecs/cjkcodecs.h getmultibytecodec cofunc - +Modules/signalmodule.c - DefaultHandler - +Modules/signalmodule.c - IgnoreHandler - +Modules/signalmodule.c - IntHandler - + +# state +Modules/_asynciomodule.c - cached_running_holder - +Modules/_asynciomodule.c - fi_freelist - +Modules/_asynciomodule.c - fi_freelist_len - +Modules/_asynciomodule.c - all_tasks - +Modules/_asynciomodule.c - current_tasks - +Modules/_asynciomodule.c - iscoroutine_typecache - +Modules/_ctypes/_ctypes.c - _ctypes_ptrtype_cache - +Modules/_tkinter.c - tcl_lock - +Modules/_tkinter.c - excInCmd - +Modules/_tkinter.c - valInCmd - +Modules/_tkinter.c - trbInCmd - +Modules/_zoneinfo.c - TIMEDELTA_CACHE - +Modules/_zoneinfo.c - ZONEINFO_WEAK_CACHE - +Modules/faulthandler.c - fatal_error - +Modules/faulthandler.c - thread - +Modules/faulthandler.c - user_signals - +Modules/faulthandler.c - stack - +Modules/faulthandler.c - old_stack - +Modules/signalmodule.c - Handlers - +Modules/syslogmodule.c - S_ident_o - +Modules/xxlimited_35.c - ErrorObject - + + +################################## +# global non-objects to fix in extension modules + +#----------------------- +# initialized once + +# pre-allocated buffer +Modules/getbuildinfo.c Py_GetBuildInfo buildinfo - +Modules/nismodule.c nisproc_maplist_2 res - +Modules/pyexpat.c PyUnknownEncodingHandler template_buffer - + +# other +Include/datetime.h - PyDateTimeAPI - +Modules/_asynciomodule.c - module_initialized - +Modules/_ctypes/cfield.c _ctypes_get_fielddesc initialized - +Modules/_ctypes/malloc_closure.c - _pagesize - +Modules/_cursesmodule.c - initialised - +Modules/_cursesmodule.c - initialised_setupterm - +Modules/_cursesmodule.c - initialisedcolors - +Modules/_cursesmodule.c - screen_encoding - +Modules/_cursesmodule.c PyInit__curses PyCurses_API - +Modules/_datetimemodule.c - CAPI - +Modules/_decimal/_decimal.c PyInit__decimal initialized - +Modules/_decimal/_decimal.c - _py_long_multiply - +Modules/_decimal/_decimal.c - _py_long_floor_divide - +Modules/_decimal/_decimal.c - _py_long_power - +Modules/_decimal/_decimal.c - _py_float_abs - +Modules/_decimal/_decimal.c - _py_long_bit_length - +Modules/_decimal/_decimal.c - _py_float_as_integer_ratio - +Modules/_decimal/_decimal.c - _decimal_api - +Modules/_elementtree.c - expat_capi - +Modules/_io/bufferedio.c _PyIO_trap_eintr eintr_int - +Modules/_sqlite/module.h - _pysqlite_enable_callback_tracebacks - +Modules/_sqlite/module.h - pysqlite_BaseTypeAdapted - +Modules/_ssl.c - _ssl_locks_count - +Modules/cjkcodecs/cjkcodecs.h - codec_list - +Modules/cjkcodecs/cjkcodecs.h - mapping_list - +Modules/getaddrinfo.c - gai_afdl - +Modules/posixmodule.c os_dup2_impl dup3_works - +Modules/posixmodule.c - structseq_new - +Modules/posixmodule.c - ticks_per_second - +Modules/pyexpat.c PyInit_pyexpat capi - +Modules/readline.c - libedit_append_replace_history_offset - +Modules/readline.c - using_libedit_emulation - +Modules/readline.c - libedit_history_start - +Modules/resource.c - initialized - +Modules/signalmodule.c - initialized - +Modules/socketmodule.c - accept4_works - +Modules/socketmodule.c - sock_cloexec_works - +Modules/socketmodule.c - PySocketModuleAPI - +Modules/spwdmodule.c - initialized - +Modules/timemodule.c - initialized - +Modules/timemodule.c _PyTime_GetClockWithInfo initialized - +Modules/timemodule.c _PyTime_GetProcessTimeWithInfo ticks_per_second - + +#----------------------- +# state + +Modules/_asynciomodule.c - cached_running_holder_tsid - +Modules/_asynciomodule.c - task_name_counter - +Modules/_ctypes/cfield.c - formattable - +Modules/_ctypes/malloc_closure.c - free_list - +Modules/_curses_panel.c - lop - +Modules/_ssl/debughelpers.c _PySSL_keylog_callback lock - +Modules/_tkinter.c - quitMainLoop - +Modules/_tkinter.c - errorInCmd - +Modules/_tkinter.c - Tkinter_busywaitinterval - +Modules/_tkinter.c - call_mutex - +Modules/_tkinter.c - var_mutex - +Modules/_tkinter.c - command_mutex - +Modules/_tkinter.c - HeadFHCD - +Modules/_tkinter.c - stdin_ready - +Modules/_tkinter.c - event_tstate - +Modules/_tracemalloc.c - allocators - +Modules/_tracemalloc.c - tables_lock - +Modules/_tracemalloc.c - tracemalloc_traced_memory - +Modules/_tracemalloc.c - tracemalloc_peak_traced_memory - +Modules/_tracemalloc.c - tracemalloc_filenames - +Modules/_tracemalloc.c - tracemalloc_traceback - +Modules/_tracemalloc.c - tracemalloc_tracebacks - +Modules/_tracemalloc.c - tracemalloc_traces - +Modules/_tracemalloc.c - tracemalloc_domains - +Modules/_tracemalloc.c - tracemalloc_reentrant_key - +Modules/_xxsubinterpretersmodule.c - _globals - +Modules/_zoneinfo.c - ZONEINFO_STRONG_CACHE - +Modules/_zoneinfo.c - ZONEINFO_STRONG_CACHE_MAX_SIZE - +Modules/_zoneinfo.c - NO_TTINFO - +Modules/faulthandler.c faulthandler_dump_traceback reentrant - +Modules/posixmodule.c - environ - +Modules/readline.c - completer_word_break_characters - +Modules/readline.c - _history_length - +Modules/readline.c - should_auto_add_history - +Modules/readline.c - sigwinch_received - +Modules/readline.c - sigwinch_ohandler - +Modules/readline.c - completed_input_string - +Modules/rotatingtree.c - random_stream - +Modules/rotatingtree.c - random_value - +Modules/signalmodule.c - is_tripped - +Modules/signalmodule.c - wakeup - +Modules/socketmodule.c - defaulttimeout - +Modules/syslogmodule.c - S_log_open - diff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsv index f3fdf3b0d05d9..2eda605913ea3 100644 --- a/Tools/c-analyzer/cpython/ignored.tsv +++ b/Tools/c-analyzer/cpython/ignored.tsv @@ -1,7 +1,10 @@ filename funcname name reason #??? - somevar ??? -# XXX The analyzer should have ignored these (forward/extern references): +################################## +# forward/extern references +# XXX The analyzer should have ignored these. + Include/py_curses.h - PyCurses_API - Include/pydecimal.h - _decimal_api - Modules/_blake2/blake2module.c - blake2b_type_spec - @@ -52,6 +55,7 @@ Modules/_multiprocessing/multiprocessing.h - _PyMp_SemLockType - Modules/_sqlite/module.c - _pysqlite_converters - Modules/_sqlite/module.c - _pysqlite_enable_callback_tracebacks - Modules/_sqlite/module.c - pysqlite_BaseTypeAdapted - +Modules/_sqlite/module.h - pysqlite_global_state - Modules/_testcapimodule.c - _PyBytesIOBuffer_Type - Modules/posixmodule.c - _Py_open_cloexec_works - Python/importdl.h - _PyImport_DynLoadFiletab - @@ -59,206 +63,208 @@ Python/importdl.h - _PyImport_DynLoadFiletab - ################################## # test code -# [] Modules/_ctypes/_ctypes_test.c - _ctypes_test_slots - -Modules/_ctypes/_ctypes_test.c - module_methods - -Modules/_ctypes/_ctypes_test.c - my_spams - -Modules/_ctypes/_ctypes_test.c - my_eggs - -Modules/_ctypes/_ctypes_test.c - an_integer - +Modules/_ctypes/_ctypes_test.c - _ctypes_testmodule - Modules/_ctypes/_ctypes_test.c - _xxx_lib - -Modules/_ctypes/_ctypes_test.c - left - -Modules/_ctypes/_ctypes_test.c - top - -Modules/_ctypes/_ctypes_test.c - right - +Modules/_ctypes/_ctypes_test.c - an_integer - Modules/_ctypes/_ctypes_test.c - bottom - -Modules/_ctypes/_ctypes_test.c - _ctypes_testmodule - -Modules/_ctypes/_ctypes_test.c - last_tfrsuv_arg - Modules/_ctypes/_ctypes_test.c - last_tf_arg_s - Modules/_ctypes/_ctypes_test.c - last_tf_arg_u - -Modules/_testbuffer.c - simple_format - -Modules/_testbuffer.c - static_mem - -Modules/_testbuffer.c - static_shape - -Modules/_testbuffer.c - static_strides - +Modules/_ctypes/_ctypes_test.c - last_tfrsuv_arg - +Modules/_ctypes/_ctypes_test.c - left - +Modules/_ctypes/_ctypes_test.c - module_methods - +Modules/_ctypes/_ctypes_test.c - my_eggs - +Modules/_ctypes/_ctypes_test.c - my_spams - +Modules/_ctypes/_ctypes_test.c - right - +Modules/_ctypes/_ctypes_test.c - top - Modules/_testbuffer.c - NDArray_Type - Modules/_testbuffer.c - StaticArray_Type - -Modules/_testbuffer.c ndarray_init kwlist - -Modules/_testbuffer.c ndarray_push kwlist - -Modules/_testbuffer.c staticarray_init kwlist - -Modules/_testbuffer.c - ndarray_methods - +Modules/_testbuffer.c - Struct - Modules/_testbuffer.c - _testbuffer_functions - -Modules/_testbuffer.c - ndarray_getset - -Modules/_testbuffer.c - ndarray_as_buffer - -Modules/_testbuffer.c - staticarray_as_buffer - -Modules/_testbuffer.c - ndarray_as_sequence - -Modules/_testbuffer.c - ndarray_as_mapping - -Modules/_testbuffer.c - structmodule - Modules/_testbuffer.c - _testbuffermodule - -Modules/_testbuffer.c - Struct - Modules/_testbuffer.c - calcsize - +Modules/_testbuffer.c - infobuf - +Modules/_testbuffer.c - ndarray_as_buffer - +Modules/_testbuffer.c - ndarray_as_mapping - +Modules/_testbuffer.c - ndarray_as_sequence - +Modules/_testbuffer.c - ndarray_getset - +Modules/_testbuffer.c - ndarray_methods - Modules/_testbuffer.c - simple_fmt - +Modules/_testbuffer.c - simple_format - Modules/_testbuffer.c - static_buffer - +Modules/_testbuffer.c - static_mem - +Modules/_testbuffer.c - static_shape - +Modules/_testbuffer.c - static_strides - +Modules/_testbuffer.c - staticarray_as_buffer - +Modules/_testbuffer.c - structmodule - +Modules/_testbuffer.c ndarray_init kwlist - Modules/_testbuffer.c ndarray_memoryview_from_buffer format - +Modules/_testbuffer.c ndarray_memoryview_from_buffer info - Modules/_testbuffer.c ndarray_memoryview_from_buffer shape - Modules/_testbuffer.c ndarray_memoryview_from_buffer strides - Modules/_testbuffer.c ndarray_memoryview_from_buffer suboffsets - -Modules/_testbuffer.c ndarray_memoryview_from_buffer info - -Modules/_testbuffer.c - infobuf - -Modules/_testcapimodule.c - TestError - -Modules/_testcapimodule.c test_capsule buffer - -Modules/_testcapimodule.c - decimal_initialized - -Modules/_testcapimodule.c - thread_done - -Modules/_testcapimodule.c - capsule_error - -Modules/_testcapimodule.c - capsule_destructor_call_count - -Modules/_testcapimodule.c - str1 - -Modules/_testcapimodule.c - str2 - -Modules/_testcapimodule.c - test_run_counter - -Modules/_testcapimodule.c - FmHook - +Modules/_testbuffer.c ndarray_push kwlist - +Modules/_testbuffer.c staticarray_init kwlist - +Modules/_testcapimodule.c - ContainerNoGC_members - +Modules/_testcapimodule.c - ContainerNoGC_type - Modules/_testcapimodule.c - FmData - -Modules/_testcapimodule.c - _testcapimodule - -Modules/_testcapimodule.c - _HashInheritanceTester_Type - -Modules/_testcapimodule.c - test_structmembersType - -Modules/_testcapimodule.c - matmulType - -Modules/_testcapimodule.c - ipowType - -Modules/_testcapimodule.c - awaitType - -Modules/_testcapimodule.c - PyRecursingInfinitelyError_Type - -Modules/_testcapimodule.c - MyList_Type - +Modules/_testcapimodule.c - FmHook - Modules/_testcapimodule.c - GenericAlias_Type - Modules/_testcapimodule.c - Generic_Type - -Modules/_testcapimodule.c - MethodDescriptorBase_Type - -Modules/_testcapimodule.c - MethodDescriptorDerived_Type - -Modules/_testcapimodule.c - MethodDescriptorNopGet_Type - -Modules/_testcapimodule.c - MethodDescriptor2_Type - -Modules/_testcapimodule.c - MethInstance_Type - -Modules/_testcapimodule.c - MethClass_Type - -Modules/_testcapimodule.c - MethStatic_Type - -Modules/_testcapimodule.c - ContainerNoGC_type - -Modules/_testcapimodule.c slot_tp_del PyId___tp_del__ - -Modules/_testcapimodule.c raise_SIGINT_then_send_None PyId_send - -Modules/_testcapimodule.c - HeapDocCType_spec - -Modules/_testcapimodule.c - HeapGcCType_spec - -Modules/_testcapimodule.c - HeapCType_spec - +Modules/_testcapimodule.c - HeapCTypeSetattr_slots - +Modules/_testcapimodule.c - HeapCTypeSetattr_spec - +Modules/_testcapimodule.c - HeapCTypeSubclassWithFinalizer_slots - +Modules/_testcapimodule.c - HeapCTypeSubclassWithFinalizer_spec - +Modules/_testcapimodule.c - HeapCTypeSubclass_slots - Modules/_testcapimodule.c - HeapCTypeSubclass_spec - +Modules/_testcapimodule.c - HeapCTypeWithBuffer_slots - Modules/_testcapimodule.c - HeapCTypeWithBuffer_spec - -Modules/_testcapimodule.c - HeapCTypeSubclassWithFinalizer_spec - +Modules/_testcapimodule.c - HeapCTypeWithDict_slots - Modules/_testcapimodule.c - HeapCTypeWithDict_spec - +Modules/_testcapimodule.c - HeapCTypeWithNegativeDict_slots - Modules/_testcapimodule.c - HeapCTypeWithNegativeDict_spec - +Modules/_testcapimodule.c - HeapCTypeWithWeakref_slots - Modules/_testcapimodule.c - HeapCTypeWithWeakref_spec - -Modules/_testcapimodule.c - HeapCTypeSetattr_spec - +Modules/_testcapimodule.c - HeapCType_slots - +Modules/_testcapimodule.c - HeapCType_spec - +Modules/_testcapimodule.c - HeapDocCType_slots - +Modules/_testcapimodule.c - HeapDocCType_spec - +Modules/_testcapimodule.c - HeapGcCType_slots - +Modules/_testcapimodule.c - HeapGcCType_spec - +Modules/_testcapimodule.c - MethClass_Type - +Modules/_testcapimodule.c - MethInstance_Type - +Modules/_testcapimodule.c - MethStatic_Type - +Modules/_testcapimodule.c - MethodDescriptor2_Type - +Modules/_testcapimodule.c - MethodDescriptorBase_Type - +Modules/_testcapimodule.c - MethodDescriptorDerived_Type - +Modules/_testcapimodule.c - MethodDescriptorNopGet_Type - +Modules/_testcapimodule.c - MyList_Type - +Modules/_testcapimodule.c - PyRecursingInfinitelyError_Type - +Modules/_testcapimodule.c - TestError - +Modules/_testcapimodule.c - TestMethods - +Modules/_testcapimodule.c - _HashInheritanceTester_Type - +Modules/_testcapimodule.c - _testcapimodule - +Modules/_testcapimodule.c - awaitType - +Modules/_testcapimodule.c - awaitType_as_async - +Modules/_testcapimodule.c - capsule_context - +Modules/_testcapimodule.c - capsule_destructor_call_count - +Modules/_testcapimodule.c - capsule_error - Modules/_testcapimodule.c - capsule_name - Modules/_testcapimodule.c - capsule_pointer - -Modules/_testcapimodule.c - capsule_context - -Modules/_testcapimodule.c - x - -Modules/_testcapimodule.c getargs_keyword_only keywords - -Modules/_testcapimodule.c getargs_keywords keywords - -Modules/_testcapimodule.c getargs_positional_only_and_keywords keywords - -Modules/_testcapimodule.c make_exception_with_doc kwlist - -Modules/_testcapimodule.c test_empty_argparse kwlist - -Modules/_testcapimodule.c test_structmembers_new keywords - -Modules/_testcapimodule.c - ml - -Modules/_testcapimodule.c - TestMethods - +Modules/_testcapimodule.c - decimal_initialized - Modules/_testcapimodule.c - generic_alias_methods - Modules/_testcapimodule.c - generic_methods - -Modules/_testcapimodule.c - meth_instance_methods - -Modules/_testcapimodule.c - meth_class_methods - -Modules/_testcapimodule.c - meth_static_methods - -Modules/_testcapimodule.c - test_members - Modules/_testcapimodule.c - heapctype_members - +Modules/_testcapimodule.c - heapctypesetattr_members - Modules/_testcapimodule.c - heapctypesubclass_members - +Modules/_testcapimodule.c - heapctypewithdict_getsetlist - Modules/_testcapimodule.c - heapctypewithdict_members - Modules/_testcapimodule.c - heapctypewithnegativedict_members - Modules/_testcapimodule.c - heapctypewithweakref_members - -Modules/_testcapimodule.c - heapctypesetattr_members - -Modules/_testcapimodule.c - ContainerNoGC_members - -Modules/_testcapimodule.c - matmulType_as_number - +Modules/_testcapimodule.c - ipowType - Modules/_testcapimodule.c - ipowType_as_number - -Modules/_testcapimodule.c - awaitType_as_async - -Modules/_testcapimodule.c - heapctypewithdict_getsetlist - -Modules/_testcapimodule.c - HeapDocCType_slots - -Modules/_testcapimodule.c - HeapGcCType_slots - -Modules/_testcapimodule.c - HeapCType_slots - -Modules/_testcapimodule.c - HeapCTypeSubclass_slots - -Modules/_testcapimodule.c - HeapCTypeWithBuffer_slots - -Modules/_testcapimodule.c - HeapCTypeSubclassWithFinalizer_slots - -Modules/_testcapimodule.c - HeapCTypeWithDict_slots - -Modules/_testcapimodule.c - HeapCTypeWithNegativeDict_slots - -Modules/_testcapimodule.c - HeapCTypeWithWeakref_slots - -Modules/_testcapimodule.c - HeapCTypeSetattr_slots - -Modules/_testimportmultiple.c - _foomodule - +Modules/_testcapimodule.c - matmulType - +Modules/_testcapimodule.c - matmulType_as_number - +Modules/_testcapimodule.c - meth_class_methods - +Modules/_testcapimodule.c - meth_instance_methods - +Modules/_testcapimodule.c - meth_static_methods - +Modules/_testcapimodule.c - ml - +Modules/_testcapimodule.c - str1 - +Modules/_testcapimodule.c - str2 - +Modules/_testcapimodule.c - test_members - +Modules/_testcapimodule.c - test_run_counter - +Modules/_testcapimodule.c - test_structmembersType - +Modules/_testcapimodule.c - thread_done - +Modules/_testcapimodule.c - x - +Modules/_testcapimodule.c getargs_keyword_only keywords - +Modules/_testcapimodule.c getargs_keywords keywords - +Modules/_testcapimodule.c getargs_positional_only_and_keywords keywords - +Modules/_testcapimodule.c make_exception_with_doc kwlist - +Modules/_testcapimodule.c raise_SIGINT_then_send_None PyId_send - +Modules/_testcapimodule.c slot_tp_del PyId___tp_del__ - +Modules/_testcapimodule.c test_capsule buffer - +Modules/_testcapimodule.c test_empty_argparse kwlist - +Modules/_testcapimodule.c test_structmembers_new keywords - +Modules/_testcapimodule.c getargs_s_hash_int keywords - Modules/_testimportmultiple.c - _barmodule - +Modules/_testimportmultiple.c - _foomodule - Modules/_testimportmultiple.c - _testimportmultiple - -Modules/_testinternalcapi.c - _testcapimodule - Modules/_testinternalcapi.c - TestMethods - -Modules/_testmultiphase.c - slots_create_nonmodule - -Modules/_testmultiphase.c - def_nonmodule - -Modules/_testmultiphase.c - main_def - -Modules/_testmultiphase.c - def_nonmodule_with_methods - -Modules/_testmultiphase.c - def_nonascii_latin - -Modules/_testmultiphase.c - def_nonascii_kana - -Modules/_testmultiphase.c - null_slots_def - +Modules/_testinternalcapi.c - _testcapimodule - +Modules/_testmultiphase.c - Example_Type_slots - +Modules/_testmultiphase.c - Example_Type_spec - +Modules/_testmultiphase.c - Example_methods - +Modules/_testmultiphase.c - StateAccessType_Type_slots - +Modules/_testmultiphase.c - StateAccessType_methods - +Modules/_testmultiphase.c - StateAccessType_spec - +Modules/_testmultiphase.c - Str_Type_slots - +Modules/_testmultiphase.c - Str_Type_spec - Modules/_testmultiphase.c - def_bad_large - Modules/_testmultiphase.c - def_bad_negative - Modules/_testmultiphase.c - def_create_int_with_state - -Modules/_testmultiphase.c - def_negative_size - -Modules/_testmultiphase.c - uninitialized_def - Modules/_testmultiphase.c - def_create_null - Modules/_testmultiphase.c - def_create_raise - Modules/_testmultiphase.c - def_create_unreported_exception - -Modules/_testmultiphase.c - def_nonmodule_with_exec_slots - Modules/_testmultiphase.c - def_exec_err - Modules/_testmultiphase.c - def_exec_raise - Modules/_testmultiphase.c - def_exec_unreported_exception - Modules/_testmultiphase.c - def_meth_state_access - +Modules/_testmultiphase.c - def_negative_size - +Modules/_testmultiphase.c - def_nonascii_kana - +Modules/_testmultiphase.c - def_nonascii_latin - +Modules/_testmultiphase.c - def_nonmodule - +Modules/_testmultiphase.c - def_nonmodule_with_exec_slots - +Modules/_testmultiphase.c - def_nonmodule_with_methods - Modules/_testmultiphase.c - imp_dummy_def - -Modules/_testmultiphase.c - Example_Type_slots - -Modules/_testmultiphase.c - StateAccessType_Type_slots - -Modules/_testmultiphase.c - Str_Type_slots - +Modules/_testmultiphase.c - main_def - Modules/_testmultiphase.c - main_slots - -Modules/_testmultiphase.c - slots_create_nonmodule - +Modules/_testmultiphase.c - meth_state_access_slots - +Modules/_testmultiphase.c - nonmodule_methods - +Modules/_testmultiphase.c - null_slots_def - Modules/_testmultiphase.c - slots_bad_large - Modules/_testmultiphase.c - slots_bad_negative - +Modules/_testmultiphase.c - slots_create_nonmodule - +Modules/_testmultiphase.c - slots_create_nonmodule - Modules/_testmultiphase.c - slots_create_null - Modules/_testmultiphase.c - slots_create_raise - Modules/_testmultiphase.c - slots_create_unreported_exception - -Modules/_testmultiphase.c - slots_nonmodule_with_exec_slots - Modules/_testmultiphase.c - slots_exec_err - Modules/_testmultiphase.c - slots_exec_raise - Modules/_testmultiphase.c - slots_exec_unreported_exception - -Modules/_testmultiphase.c - meth_state_access_slots - -Modules/_testmultiphase.c - Example_methods - -Modules/_testmultiphase.c - StateAccessType_methods - +Modules/_testmultiphase.c - slots_nonmodule_with_exec_slots - Modules/_testmultiphase.c - testexport_methods - -Modules/_testmultiphase.c - nonmodule_methods - -Modules/_testmultiphase.c - Example_Type_spec - -Modules/_testmultiphase.c - StateAccessType_spec - -Modules/_testmultiphase.c - Str_Type_spec - -Modules/_xxtestfuzz/_xxtestfuzz.c - module_methods - +Modules/_testmultiphase.c - uninitialized_def - Modules/_xxtestfuzz/_xxtestfuzz.c - _fuzzmodule - +Modules/_xxtestfuzz/_xxtestfuzz.c - module_methods - +Modules/_xxtestfuzz/fuzzer.c - SRE_FLAG_DEBUG - +Modules/_xxtestfuzz/fuzzer.c - ast_literal_eval_method - +Modules/_xxtestfuzz/fuzzer.c - compiled_patterns - +Modules/_xxtestfuzz/fuzzer.c - csv_error - Modules/_xxtestfuzz/fuzzer.c - csv_module - -Modules/_xxtestfuzz/fuzzer.c - regex_patterns - -Modules/_xxtestfuzz/fuzzer.c - struct_unpack_method - -Modules/_xxtestfuzz/fuzzer.c - struct_error - Modules/_xxtestfuzz/fuzzer.c - json_loads_method - +Modules/_xxtestfuzz/fuzzer.c - regex_patterns - Modules/_xxtestfuzz/fuzzer.c - sre_compile_method - Modules/_xxtestfuzz/fuzzer.c - sre_error_exception - -Modules/_xxtestfuzz/fuzzer.c - compiled_patterns - -Modules/_xxtestfuzz/fuzzer.c - csv_error - -Modules/_xxtestfuzz/fuzzer.c - SRE_FLAG_DEBUG - -Modules/_xxtestfuzz/fuzzer.c LLVMFuzzerTestOneInput STRUCT_UNPACK_INITIALIZED - +Modules/_xxtestfuzz/fuzzer.c - struct_error - +Modules/_xxtestfuzz/fuzzer.c - struct_unpack_method - +Modules/_xxtestfuzz/fuzzer.c LLVMFuzzerTestOneInput CSV_READER_INITIALIZED - Modules/_xxtestfuzz/fuzzer.c LLVMFuzzerTestOneInput JSON_LOADS_INITIALIZED - Modules/_xxtestfuzz/fuzzer.c LLVMFuzzerTestOneInput SRE_COMPILE_INITIALIZED - Modules/_xxtestfuzz/fuzzer.c LLVMFuzzerTestOneInput SRE_MATCH_INITIALIZED - -Modules/_xxtestfuzz/fuzzer.c LLVMFuzzerTestOneInput CSV_READER_INITIALIZED - +Modules/_xxtestfuzz/fuzzer.c LLVMFuzzerTestOneInput STRUCT_UNPACK_INITIALIZED - +Modules/_xxtestfuzz/fuzzer.c LLVMFuzzerTestOneInput AST_LITERAL_EVAL_INITIALIZED - ################################## -# temporary whitelist - should be const +# should be const +# XXX Make them const. -# These are all variables that we will be making non-global. +# These are all variables that we will be leaving global. #----------------------- # keywords for PyArg_ParseTupleAndKeywords() # "static char *name[]" -> "static const char * const name[]" -# [] Modules/cjkcodecs/multibytecodec.c - incnewkwarglist - Modules/cjkcodecs/multibytecodec.c - streamkwarglist - @@ -376,301 +382,346 @@ Modules/_zoneinfo.c zoneinfo_no_cache kwlist - Objects/exceptions.c ImportError_init kwlist - Objects/interpreteridobject.c interpid_new kwlist - Objects/weakrefobject.c weakref_call kwlist - +Objects/exceptions.c NameError_init kwlist - +Objects/exceptions.c AttributeError_init kwlist - + +#----------------------- +# PyModuleDef + +Modules/_multiprocessing/posixshmem.c - _posixshmemmodule - +Modules/_sqlite/module.h - _sqlite3module - +Modules/_ssl.c - _sslmodule_def - +Modules/_ssl.h - _sslmodule_def - +Modules/_testmultiphase.c - def_module_state_shared - +Modules/_threadmodule.c - thread_module - +Modules/_typingmodule.c - typingmodule - +Modules/signalmodule.c - signal_module - +Modules/xxlimited_35.c - xxmodule - #----------------------- # PyModuleDef_Slot -# [] Modules/_abc.c - _abcmodule_slots - -Modules/arraymodule.c - arrayslots - -Modules/atexitmodule.c - atexit_slots - -Modules/audioop.c - audioop_slots - -Modules/binascii.c - binascii_slots - Modules/_blake2/blake2module.c - _blake2_slots - -Modules/cmathmodule.c - cmath_slots - +Modules/_bz2module.c - _bz2_slots - Modules/_codecsmodule.c - _codecs_slots - +Modules/_collectionsmodule.c - collections_slots - +Modules/_contextvarsmodule.c - _contextvars_slots - Modules/_cryptmodule.c - _crypt_slots - +Modules/_csv.c - csv_slots - Modules/_curses_panel.c - _curses_slots - Modules/_dbmmodule.c - _dbmmodule_slots - -Modules/errnomodule.c - errno_slots - -Modules/faulthandler.c - faulthandler_slots - -Modules/fcntlmodule.c - fcntl_slots - +Modules/_functoolsmodule.c - _functools_slots - Modules/_gdbmmodule.c - _gdbm_module_slots - Modules/_hashopenssl.c - hashlib_slots - +Modules/_heapqmodule.c - heapq_slots - Modules/_json.c - _json_slots - +Modules/_localemodule.c - _locale_slots - Modules/_lsprof.c - _lsprofslots - Modules/_lzmamodule.c - lzma_slots - +Modules/_multiprocessing/multiprocessing.c - multiprocessing_slots - +Modules/_operator.c - operator_slots - +Modules/_posixsubprocess.c - _posixsubprocess_slots - +Modules/_queuemodule.c - queuemodule_slots - +Modules/_randommodule.c - _random_slots - +Modules/_scproxy.c - _scproxy_slots - +Modules/_sha3/sha3module.c - _sha3_slots - +Modules/_sqlite/module.c - module_slots - +Modules/_sre.c - sre_slots - +Modules/_ssl.c - sslmodule_slots - +Modules/_stat.c - stat_slots - +Modules/_statisticsmodule.c - _statisticsmodule_slots - +Modules/_struct.c - _structmodule_slots - +Modules/_threadmodule.c - thread_module_slots - +Modules/_typingmodule.c - _typingmodule_slots - +Modules/_uuidmodule.c - uuid_slots - +Modules/_weakref.c - weakref_slots - +Modules/_winapi.c - winapi_slots - +Modules/_zoneinfo.c - zoneinfomodule_slots - +Modules/arraymodule.c - arrayslots - +Modules/atexitmodule.c - atexit_slots - +Modules/audioop.c - audioop_slots - +Modules/binascii.c - binascii_slots - +Modules/cjkcodecs/cjkcodecs.h - _cjk_slots - +Modules/cjkcodecs/multibytecodec.c - _multibytecodec_slots - +Modules/cmathmodule.c - cmath_slots - +Modules/errnomodule.c - errno_slots - +Modules/faulthandler.c - faulthandler_slots - +Modules/fcntlmodule.c - fcntl_slots - +Modules/gcmodule.c - gcmodule_slots - +Modules/gcmodule.c - gcmodule_slots - +Modules/grpmodule.c - grpmodule_slots - +Modules/itertoolsmodule.c - itertoolsmodule_slots - Modules/mathmodule.c - math_slots - Modules/md5module.c - _md5_slots - Modules/mmapmodule.c - mmap_slots - -Modules/_multiprocessing/multiprocessing.c - multiprocessing_slots - Modules/nismodule.c - nis_slots - Modules/overlapped.c - overlapped_slots - Modules/posixmodule.c - posixmodile_slots - -Modules/_scproxy.c - _scproxy_slots - +Modules/pwdmodule.c - pwdmodule_slots - +Modules/pyexpat.c - pyexpat_slots - +Modules/resource.c - resource_slots - +Modules/selectmodule.c - _select_slots - Modules/sha1module.c - _sha1_slots - Modules/sha256module.c - _sha256_slots - -Modules/_sha3/sha3module.c - _sha3_slots - Modules/sha512module.c - _sha512_slots - -Modules/_stat.c - stat_slots - +Modules/signalmodule.c - signal_slots - +Modules/spwdmodule.c - spwdmodule_slots - +Modules/symtablemodule.c - symtable_slots - Modules/syslogmodule.c - syslog_slots - Modules/termios.c - termios_slots - +Modules/timemodule.c - time_slots - Modules/unicodedata.c - unicodedata_slots - -Modules/_uuidmodule.c - uuid_slots - -Modules/_winapi.c - winapi_slots - Modules/xxlimited.c - xx_slots - -Modules/zlibmodule.c - zlib_slots - -Modules/_zoneinfo.c - zoneinfomodule_slots - -Python/marshal.c - marshalmodule_slots - -Python/Python-ast.c - astmodule_slots - -Modules/_bz2module.c - _bz2_slots - -Modules/_collectionsmodule.c - collections_slots - -Modules/_contextvarsmodule.c - _contextvars_slots - -Modules/_functoolsmodule.c - _functools_slots - -Modules/_heapqmodule.c - heapq_slots - -Modules/itertoolsmodule.c - itertoolsmodule_slots - -Modules/_localemodule.c - _locale_slots - -Modules/_operator.c - operator_slots - -Modules/resource.c - resource_slots - -Modules/_statisticsmodule.c - _statisticsmodule_slots - -Modules/timemodule.c - time_slots - -Modules/_weakref.c - weakref_slots - +Modules/xxlimited_35.c - xx_slots - Modules/xxmodule.c - xx_slots - Modules/xxsubtype.c - xxsubtype_slots - +Modules/zlibmodule.c - zlib_slots - +Python/Python-ast.c - astmodule_slots - +Python/marshal.c - marshalmodule_slots - #----------------------- # PyMethodDef and PyMethodDef[], for static types and modules -# [] -Modules/_abc.c - _destroy_def - Modules/_abc.c - _abcmodule_methods - -Modules/arraymodule.c - array_methods - -Modules/arraymodule.c - arrayiter_methods - -Modules/arraymodule.c - a_methods - -Modules/_asynciomodule.c - FutureType_methods - +Modules/_abc.c - _destroy_def - Modules/_asynciomodule.c - FutureIter_methods - +Modules/_asynciomodule.c - FutureType_methods - Modules/_asynciomodule.c - TaskType_methods - -Modules/_asynciomodule.c - asyncio_methods - Modules/_asynciomodule.c - TaskWakeupDef - -Modules/atexitmodule.c - atexit_methods - -Modules/audioop.c - audioop_methods - -Modules/binascii.c - binascii_module_methods - +Modules/_asynciomodule.c - asyncio_methods - Modules/_bisectmodule.c - bisect_methods - Modules/_blake2/blake2b_impl.c - py_blake2b_methods - Modules/_blake2/blake2module.c - blake2mod_functions - Modules/_blake2/blake2s_impl.c - py_blake2s_methods - Modules/_bz2module.c - BZ2Compressor_methods - Modules/_bz2module.c - BZ2Decompressor_methods - -Modules/cjkcodecs/multibytecodec.c - multibytecodec_methods - -Modules/cjkcodecs/multibytecodec.c - mbiencoder_methods - -Modules/cjkcodecs/multibytecodec.c - mbidecoder_methods - -Modules/cjkcodecs/multibytecodec.c - mbstreamreader_methods - -Modules/cjkcodecs/multibytecodec.c - mbstreamwriter_methods - -Modules/cjkcodecs/multibytecodec.c - __methods - -Modules/cmathmodule.c - cmath_methods - Modules/_codecsmodule.c - _codecs_functions - +Modules/_collectionsmodule.c - collections_methods - +Modules/_collectionsmodule.c - defdict_methods - Modules/_collectionsmodule.c - deque_methods - Modules/_collectionsmodule.c - dequeiter_methods - -Modules/_collectionsmodule.c - defdict_methods - Modules/_collectionsmodule.c - tuplegetter_methods - -Modules/_collectionsmodule.c - collections_methods - Modules/_contextvarsmodule.c - _contextvars_methods - Modules/_cryptmodule.c - crypt_methods - Modules/_csv.c - Reader_methods - Modules/_csv.c - Writer_methods - Modules/_csv.c - csv_methods - -Modules/_ctypes/callproc.c - _ctypes_module_methods - +Modules/_csv.c - dialect_methods - +Modules/_ctypes/_ctypes.c - Array_methods - Modules/_ctypes/_ctypes.c - CDataType_methods - +Modules/_ctypes/_ctypes.c - PyCData_methods - Modules/_ctypes/_ctypes.c - PyCPointerType_methods - -Modules/_ctypes/_ctypes.c - c_void_p_method - -Modules/_ctypes/_ctypes.c - c_char_p_method - -Modules/_ctypes/_ctypes.c - c_wchar_p_method - Modules/_ctypes/_ctypes.c - PyCSimpleType_methods - -Modules/_ctypes/_ctypes.c - PyCData_methods - -Modules/_ctypes/_ctypes.c - Array_methods - Modules/_ctypes/_ctypes.c - Simple_methods - +Modules/_ctypes/_ctypes.c - c_char_p_method - +Modules/_ctypes/_ctypes.c - c_void_p_method - +Modules/_ctypes/_ctypes.c - c_wchar_p_method - +Modules/_ctypes/callproc.c - _ctypes_module_methods - Modules/_ctypes/stgdict.c - PyCStgDict_methods - -Modules/_cursesmodule.c - PyCursesWindow_Methods - -Modules/_cursesmodule.c - PyCurses_methods - Modules/_curses_panel.c - PyCursesPanel_Methods - Modules/_curses_panel.c - PyCurses_methods - -Modules/_datetimemodule.c - delta_methods - -Modules/_datetimemodule.c - iso_calendar_date_methods - +Modules/_cursesmodule.c - PyCursesWindow_Methods - +Modules/_cursesmodule.c - PyCurses_methods - Modules/_datetimemodule.c - date_methods - -Modules/_datetimemodule.c - tzinfo_methods - -Modules/_datetimemodule.c - timezone_methods - -Modules/_datetimemodule.c - time_methods - Modules/_datetimemodule.c - datetime_methods - +Modules/_datetimemodule.c - delta_methods - +Modules/_datetimemodule.c - iso_calendar_date_methods - Modules/_datetimemodule.c - module_methods - +Modules/_datetimemodule.c - time_methods - +Modules/_datetimemodule.c - timezone_methods - +Modules/_datetimemodule.c - tzinfo_methods - Modules/_dbmmodule.c - dbm_methods - Modules/_dbmmodule.c - dbmmodule_methods - -Modules/_decimal/_decimal.c - signaldict_methods - +Modules/_decimal/_decimal.c - _decimal_methods - +Modules/_decimal/_decimal.c - context_methods - Modules/_decimal/_decimal.c - ctxmanager_methods - Modules/_decimal/_decimal.c - dec_methods - -Modules/_decimal/_decimal.c - context_methods - -Modules/_decimal/_decimal.c - _decimal_methods - +Modules/_decimal/_decimal.c - signaldict_methods - +Modules/_elementtree.c - _functions - Modules/_elementtree.c - element_methods - Modules/_elementtree.c - treebuilder_methods - Modules/_elementtree.c - xmlparser_methods - -Modules/_elementtree.c - _functions - -Modules/errnomodule.c - errno_methods - -Modules/faulthandler.c - module_methods - -Modules/fcntlmodule.c - fcntl_methods - -Modules/_functoolsmodule.c - partial_methods - -Modules/_functoolsmodule.c - lru_cache_methods - Modules/_functoolsmodule.c - _functools_methods - -Modules/gcmodule.c - GcMethods - -Modules/_gdbmmodule.c - gdbm_methods - +Modules/_functoolsmodule.c - lru_cache_methods - +Modules/_functoolsmodule.c - partial_methods - Modules/_gdbmmodule.c - _gdbm_module_methods - -Modules/grpmodule.c - grp_methods - -Modules/_hashopenssl.c - EVP_methods - +Modules/_gdbmmodule.c - gdbm_methods - Modules/_hashopenssl.c - EVPXOF_methods - -Modules/_hashopenssl.c - HMAC_methods - Modules/_hashopenssl.c - EVP_functions - +Modules/_hashopenssl.c - EVP_methods - +Modules/_hashopenssl.c - HMAC_methods - Modules/_heapqmodule.c - heapq_methods - +Modules/_io/_iomodule.c - module_methods - Modules/_io/bufferedio.c - bufferediobase_methods - +Modules/_io/bufferedio.c - bufferedrandom_methods - Modules/_io/bufferedio.c - bufferedreader_methods - -Modules/_io/bufferedio.c - bufferedwriter_methods - Modules/_io/bufferedio.c - bufferedrwpair_methods - -Modules/_io/bufferedio.c - bufferedrandom_methods - +Modules/_io/bufferedio.c - bufferedwriter_methods - Modules/_io/bytesio.c - bytesio_methods - Modules/_io/fileio.c - fileio_methods - Modules/_io/iobase.c - iobase_methods - Modules/_io/iobase.c - rawiobase_methods - -Modules/_io/_iomodule.c - module_methods - Modules/_io/stringio.c - stringio_methods - -Modules/_io/textio.c - textiobase_methods - Modules/_io/textio.c - incrementalnewlinedecoder_methods - +Modules/_io/textio.c - textiobase_methods - Modules/_io/textio.c - textiowrapper_methods - Modules/_io/winconsoleio.c - winconsoleio_methods - -Modules/itertoolsmodule.c - groupby_methods - -Modules/itertoolsmodule.c - _grouper_methods - -Modules/itertoolsmodule.c - teedataobject_methods - -Modules/itertoolsmodule.c - tee_methods - -Modules/itertoolsmodule.c - cycle_methods - -Modules/itertoolsmodule.c - dropwhile_methods - -Modules/itertoolsmodule.c - takewhile_reduce_methods - -Modules/itertoolsmodule.c - islice_methods - -Modules/itertoolsmodule.c - starmap_methods - -Modules/itertoolsmodule.c - chain_methods - -Modules/itertoolsmodule.c - product_methods - -Modules/itertoolsmodule.c - combinations_methods - -Modules/itertoolsmodule.c - cwr_methods - -Modules/itertoolsmodule.c - permuations_methods - -Modules/itertoolsmodule.c - accumulate_methods - -Modules/itertoolsmodule.c - compress_methods - -Modules/itertoolsmodule.c - filterfalse_methods - -Modules/itertoolsmodule.c - count_methods - -Modules/itertoolsmodule.c - repeat_methods - -Modules/itertoolsmodule.c - zip_longest_methods - -Modules/itertoolsmodule.c - module_methods - Modules/_json.c - speedups_methods - Modules/_localemodule.c - PyLocale_Methods - -Modules/_lsprof.c - profiler_methods - Modules/_lsprof.c - moduleMethods - +Modules/_lsprof.c - profiler_methods - Modules/_lzmamodule.c - Compressor_methods - Modules/_lzmamodule.c - Decompressor_methods - Modules/_lzmamodule.c - lzma_methods - -Modules/mathmodule.c - math_methods - -Modules/md5module.c - MD5_methods - -Modules/md5module.c - MD5_functions - -Modules/mmapmodule.c - mmap_object_methods - Modules/_multiprocessing/multiprocessing.c - module_methods - Modules/_multiprocessing/posixshmem.c - module_methods - Modules/_multiprocessing/semaphore.c - semlock_methods - -Modules/nismodule.c - nis_methods - Modules/_opcode.c - opcode_functions - -Modules/_operator.c - operator_methods - -Modules/_operator.c - itemgetter_methods - Modules/_operator.c - attrgetter_methods - +Modules/_operator.c - itemgetter_methods - Modules/_operator.c - methodcaller_methods - -Modules/ossaudiodev.c - oss_methods - -Modules/ossaudiodev.c - oss_mixer_methods - -Modules/ossaudiodev.c - ossaudiodev_methods - -Modules/overlapped.c - Overlapped_methods - -Modules/overlapped.c - overlapped_functions - +Modules/_operator.c - operator_methods - Modules/_pickle.c - Pickler_methods - -Modules/_pickle.c - picklerproxy_methods - Modules/_pickle.c - Unpickler_methods - -Modules/_pickle.c - unpicklerproxy_methods - Modules/_pickle.c - pickle_methods - -Modules/posixmodule.c - DirEntry_methods - -Modules/posixmodule.c - ScandirIterator_methods - -Modules/posixmodule.c - posix_methods - +Modules/_pickle.c - picklerproxy_methods - +Modules/_pickle.c - unpicklerproxy_methods - Modules/_posixsubprocess.c - module_methods - -Modules/pwdmodule.c - pwd_methods - -Modules/pyexpat.c - xmlparse_methods - -Modules/pyexpat.c - pyexpat_methods - Modules/_queuemodule.c - simplequeue_methods - Modules/_randommodule.c - random_methods - -Modules/readline.c - readline_methods - -Modules/resource.c - resource_methods - Modules/_scproxy.c - mod_methods - -Modules/selectmodule.c - poll_methods - -Modules/selectmodule.c - devpoll_methods - -Modules/selectmodule.c - pyepoll_methods - -Modules/selectmodule.c - kqueue_queue_methods - -Modules/selectmodule.c - select_methods - -Modules/sha1module.c - SHA1_methods - -Modules/sha1module.c - SHA1_functions - -Modules/sha256module.c - SHA_methods - -Modules/sha256module.c - SHA_functions - Modules/_sha3/sha3module.c - SHA3_methods - Modules/_sha3/sha3module.c - SHAKE_methods - -Modules/sha512module.c - SHA_methods - -Modules/sha512module.c - SHA_functions - -Modules/signalmodule.c - signal_methods - -Modules/socketmodule.c - sock_methods - -Modules/socketmodule.c - socket_methods - -Modules/spwdmodule.c - spwd_methods - Modules/_sqlite/connection.c - connection_methods - Modules/_sqlite/cursor.c - cursor_methods - Modules/_sqlite/module.c - module_methods - Modules/_sqlite/row.c - row_methods - -Modules/_sre.c - pattern_methods - +Modules/_sre.c - _functions - Modules/_sre.c - match_methods - +Modules/_sre.c - pattern_methods - Modules/_sre.c - scanner_methods - -Modules/_sre.c - _functions - Modules/_ssl.c - PySSLMethods - +Modules/_ssl.c - PySSL_methods - Modules/_ssl.c - context_methods - Modules/_ssl.c - memory_bio_methods - -Modules/_ssl.c - PySSL_methods - +Modules/_ssl/cert.c - certificate_methods - Modules/_stat.c - stat_methods - Modules/_statisticsmodule.c - statistics_methods - -Modules/_struct.c - unpackiter_methods - -Modules/_struct.c - s_methods - Modules/_struct.c - module_functions - -Modules/symtablemodule.c - symtable_methods - -Modules/syslogmodule.c - syslog_methods - -Modules/termios.c - termios_methods - +Modules/_struct.c - s_methods - +Modules/_struct.c - unpackiter_methods - Modules/_threadmodule.c - lock_methods - Modules/_threadmodule.c - rlock_methods - -Modules/_threadmodule.c local_new wr_callback_def - Modules/_threadmodule.c - thread_methods - -Modules/timemodule.c - time_methods - -Modules/_tkinter.c - Tktt_methods - +Modules/_threadmodule.c local_new wr_callback_def - Modules/_tkinter.c - Tkapp_methods - +Modules/_tkinter.c - Tktt_methods - Modules/_tkinter.c - moduleMethods - Modules/_tracemalloc.c - module_methods - -Modules/unicodedata.c - unicodedata_functions - +Modules/_typingmodule.c - typing_methods - Modules/_uuidmodule.c - uuid_methods - Modules/_weakref.c - weakref_functions - Modules/_winapi.c - overlapped_methods - Modules/_winapi.c - winapi_functions - +Modules/_xxsubinterpretersmodule.c - module_functions - +Modules/_zoneinfo.c - module_methods - +Modules/_zoneinfo.c - zoneinfo_methods - +Modules/arraymodule.c - a_methods - +Modules/arraymodule.c - array_methods - +Modules/arraymodule.c - arrayiter_methods - +Modules/atexitmodule.c - atexit_methods - +Modules/audioop.c - audioop_methods - +Modules/binascii.c - binascii_module_methods - +Modules/cjkcodecs/cjkcodecs.h - __methods - +Modules/cjkcodecs/cjkcodecs.h - _cjk_methods - +Modules/cjkcodecs/multibytecodec.c - __methods - +Modules/cjkcodecs/multibytecodec.c - _multibytecodec_methods - +Modules/cjkcodecs/multibytecodec.c - mbidecoder_methods - +Modules/cjkcodecs/multibytecodec.c - mbiencoder_methods - +Modules/cjkcodecs/multibytecodec.c - mbstreamreader_methods - +Modules/cjkcodecs/multibytecodec.c - mbstreamwriter_methods - +Modules/cjkcodecs/multibytecodec.c - multibytecodec_methods - +Modules/cmathmodule.c - cmath_methods - +Modules/errnomodule.c - errno_methods - +Modules/faulthandler.c - module_methods - +Modules/fcntlmodule.c - fcntl_methods - +Modules/gcmodule.c - GcMethods - +Modules/getpath.c - getpath_methods - +Modules/getpath.c - getpath_nowarn_method - +Modules/getpath.c - getpath_warn_method - +Modules/grpmodule.c - grp_methods - +Modules/itertoolsmodule.c - _grouper_methods - +Modules/itertoolsmodule.c - accumulate_methods - +Modules/itertoolsmodule.c - chain_methods - +Modules/itertoolsmodule.c - combinations_methods - +Modules/itertoolsmodule.c - compress_methods - +Modules/itertoolsmodule.c - count_methods - +Modules/itertoolsmodule.c - cwr_methods - +Modules/itertoolsmodule.c - cycle_methods - +Modules/itertoolsmodule.c - dropwhile_methods - +Modules/itertoolsmodule.c - filterfalse_methods - +Modules/itertoolsmodule.c - groupby_methods - +Modules/itertoolsmodule.c - islice_methods - +Modules/itertoolsmodule.c - module_methods - +Modules/itertoolsmodule.c - permuations_methods - +Modules/itertoolsmodule.c - product_methods - +Modules/itertoolsmodule.c - repeat_methods - +Modules/itertoolsmodule.c - starmap_methods - +Modules/itertoolsmodule.c - takewhile_reduce_methods - +Modules/itertoolsmodule.c - tee_methods - +Modules/itertoolsmodule.c - teedataobject_methods - +Modules/itertoolsmodule.c - zip_longest_methods - +Modules/mathmodule.c - math_methods - +Modules/md5module.c - MD5_functions - +Modules/md5module.c - MD5_methods - +Modules/mmapmodule.c - mmap_object_methods - +Modules/nismodule.c - nis_methods - +Modules/ossaudiodev.c - oss_methods - +Modules/ossaudiodev.c - oss_mixer_methods - +Modules/ossaudiodev.c - ossaudiodev_methods - +Modules/overlapped.c - Overlapped_methods - +Modules/overlapped.c - overlapped_functions - +Modules/posixmodule.c - DirEntry_methods - +Modules/posixmodule.c - ScandirIterator_methods - +Modules/posixmodule.c - posix_methods - +Modules/pwdmodule.c - pwd_methods - +Modules/pyexpat.c - pyexpat_methods - +Modules/pyexpat.c - xmlparse_methods - +Modules/readline.c - readline_methods - +Modules/resource.c - resource_methods - +Modules/selectmodule.c - devpoll_methods - +Modules/selectmodule.c - kqueue_queue_methods - +Modules/selectmodule.c - poll_methods - +Modules/selectmodule.c - pyepoll_methods - +Modules/selectmodule.c - select_methods - +Modules/sha1module.c - SHA1_functions - +Modules/sha1module.c - SHA1_methods - +Modules/sha256module.c - SHA_functions - +Modules/sha256module.c - SHA_methods - +Modules/sha512module.c - SHA_functions - +Modules/sha512module.c - SHA_methods - +Modules/signalmodule.c - signal_methods - +Modules/socketmodule.c - sock_methods - +Modules/socketmodule.c - socket_methods - +Modules/spwdmodule.c - spwd_methods - +Modules/symtablemodule.c - symtable_methods - +Modules/syslogmodule.c - syslog_methods - +Modules/termios.c - termios_methods - +Modules/timemodule.c - time_methods - +Modules/unicodedata.c - unicodedata_functions - Modules/xxlimited.c - Xxo_methods - Modules/xxlimited.c - xx_methods - +Modules/xxlimited_35.c - Xxo_methods - +Modules/xxlimited_35.c - xx_methods - Modules/xxmodule.c - Xxo_methods - Modules/xxmodule.c - xx_methods - -Modules/_xxsubinterpretersmodule.c - module_functions - -Modules/xxsubtype.c - spamlist_methods - Modules/xxsubtype.c - spamdict_methods - +Modules/xxsubtype.c - spamlist_methods - Modules/xxsubtype.c - xxsubtype_functions - -Modules/zlibmodule.c - comp_methods - Modules/zlibmodule.c - Decomp_methods - +Modules/zlibmodule.c - comp_methods - Modules/zlibmodule.c - zlib_methods - -Modules/_zoneinfo.c - zoneinfo_methods - -Modules/_zoneinfo.c - module_methods - -Modules/cjkcodecs/cjkcodecs.h - __methods - Objects/bytearrayobject.c - bytearray_methods - Objects/bytearrayobject.c - bytearrayiter_methods - Objects/bytesobject.c - bytes_methods - @@ -680,30 +731,34 @@ Objects/codeobject.c - code_methods - Objects/complexobject.c - complex_methods - Objects/descrobject.c - descr_methods - Objects/descrobject.c - mappingproxy_methods - -Objects/descrobject.c - wrapper_methods - Objects/descrobject.c - property_methods - -Objects/dictobject.c - mapp_methods - +Objects/descrobject.c - wrapper_methods - +Objects/dictobject.c - dictitems_methods - Objects/dictobject.c - dictiter_methods - Objects/dictobject.c - dictkeys_methods - -Objects/dictobject.c - dictitems_methods - Objects/dictobject.c - dictvalues_methods - +Objects/dictobject.c - mapp_methods - Objects/enumobject.c - enum_methods - Objects/enumobject.c - reversediter_methods - +Objects/exceptions.c - AttributeError_methods - +Objects/exceptions.c - BaseExceptionGroup_methods - Objects/exceptions.c - BaseException_methods - Objects/exceptions.c - ImportError_methods - +Objects/exceptions.c - NameError_methods - Objects/exceptions.c - OSError_methods - Objects/fileobject.c - stdprinter_methods - Objects/floatobject.c - float_methods - Objects/frameobject.c - frame_methods - Objects/genericaliasobject.c - ga_methods - -Objects/genobject.c - gen_methods - -Objects/genobject.c - coro_methods - -Objects/genobject.c - coro_wrapper_methods - -Objects/genobject.c - async_gen_methods - Objects/genobject.c - async_gen_asend_methods - Objects/genobject.c - async_gen_athrow_methods - -Objects/iterobject.c - seqiter_methods - +Objects/genobject.c - async_gen_methods - +Objects/genobject.c - coro_methods - +Objects/genobject.c - coro_wrapper_methods - +Objects/genobject.c - gen_methods - +Objects/iterobject.c - anextawaitable_methods - Objects/iterobject.c - calliter_methods - +Objects/iterobject.c - seqiter_methods - Objects/listobject.c - list_methods - Objects/listobject.c - listiter_methods - Objects/listobject.c - listreviter_methods - @@ -714,139 +769,148 @@ Objects/moduleobject.c - module_methods - Objects/namespaceobject.c - namespace_methods - Objects/object.c - notimplemented_methods - Objects/odictobject.c - odict_methods - +Objects/odictobject.c - odictitems_methods - Objects/odictobject.c - odictiter_methods - Objects/odictobject.c - odictkeys_methods - -Objects/odictobject.c - odictitems_methods - Objects/odictobject.c - odictvalues_methods - Objects/picklebufobject.c - picklebuf_methods - +Objects/rangeobject.c - longrangeiter_methods - Objects/rangeobject.c - range_methods - Objects/rangeobject.c - rangeiter_methods - -Objects/rangeobject.c - longrangeiter_methods - -Objects/setobject.c - setiter_methods - -Objects/setobject.c - set_methods - Objects/setobject.c - frozenset_methods - +Objects/setobject.c - set_methods - +Objects/setobject.c - setiter_methods - Objects/sliceobject.c - ellipsis_methods - Objects/sliceobject.c - slice_methods - +Objects/stringlib/unicode_format.h - fieldnameiter_methods - +Objects/stringlib/unicode_format.h - formatteriter_methods - Objects/structseq.c - structseq_methods - Objects/tupleobject.c - tuple_methods - Objects/tupleobject.c - tupleiter_methods - -Objects/typeobject.c - type_methods - Objects/typeobject.c - object_methods - Objects/typeobject.c - tp_new_methoddef - +Objects/typeobject.c - type_methods - +Objects/unicodeobject.c - _string_methods - Objects/unicodeobject.c - encoding_map_methods - Objects/unicodeobject.c - unicode_methods - Objects/unicodeobject.c - unicodeiter_methods - -Objects/unicodeobject.c - _string_methods - Objects/unionobject.c - union_methods - -Objects/weakrefobject.c - weakref_methods - Objects/weakrefobject.c - proxy_methods - -Objects/stringlib/unicode_format.h - formatteriter_methods - -Objects/stringlib/unicode_format.h - fieldnameiter_methods - +Objects/weakrefobject.c - weakref_methods - +Python/Python-ast.c - ast_type_methods - +Python/_warnings.c - warnings_functions - +Python/bltinmodule.c - builtin_methods - Python/bltinmodule.c - filter_methods - Python/bltinmodule.c - map_methods - Python/bltinmodule.c - zip_methods - -Python/bltinmodule.c - builtin_methods - -Python/context.c - PyContext_methods - -Python/context.c - PyContextVar_methods - Python/context.c - PyContextTokenType_methods - +Python/context.c - PyContextVar_methods - +Python/context.c - PyContext_methods - Python/hamt.c - PyHamt_methods - Python/import.c - imp_methods - Python/marshal.c - marshal_methods - -Python/Python-ast.c - ast_type_methods - Python/sysmodule.c - sys_methods - Python/traceback.c - tb_methods - -Python/_warnings.c - warnings_functions - #----------------------- # PyMemberDef[], for static types and strucseq -# [] Modules/_bz2module.c - BZ2Decompressor_members - -Modules/cjkcodecs/multibytecodec.c - mbstreamreader_members - -Modules/cjkcodecs/multibytecodec.c - mbstreamwriter_members - Modules/_collectionsmodule.c - defdict_members - Modules/_collectionsmodule.c - tuplegetter_members - Modules/_csv.c - Dialect_memberlist - Modules/_csv.c - Reader_memberlist - Modules/_csv.c - Writer_memberlist - -Modules/_ctypes/callproc.c - PyCArgType_members - Modules/_ctypes/_ctypes.c - PyCData_members - +Modules/_ctypes/callproc.c - PyCArgType_members - Modules/_datetimemodule.c - delta_members - Modules/_elementtree.c - xmlparser_members - -Modules/_functoolsmodule.c - partial_memberlist - Modules/_functoolsmodule.c - keyobject_members - +Modules/_functoolsmodule.c - lru_cache_memberlist - +Modules/_functoolsmodule.c - partial_memberlist - +Modules/_io/bufferedio.c - bufferedrandom_members - Modules/_io/bufferedio.c - bufferedreader_members - Modules/_io/bufferedio.c - bufferedwriter_members - -Modules/_io/bufferedio.c - bufferedrandom_members - Modules/_io/fileio.c - fileio_members - Modules/_io/textio.c - textiowrapper_members - Modules/_io/winconsoleio.c - winconsoleio_members - -Modules/_json.c - scanner_members - Modules/_json.c - encoder_members - +Modules/_json.c - scanner_members - Modules/_lzmamodule.c - Decompressor_members - Modules/_multiprocessing/semaphore.c - semlock_members - +Modules/_pickle.c - Pickler_members - +Modules/_queuemodule.c - simplequeue_members - +Modules/_sqlite/connection.c - connection_members - +Modules/_sqlite/cursor.c - cursor_members - +Modules/_sqlite/statement.c - stmt_members - +Modules/_sre.c - match_members - +Modules/_sre.c - pattern_members - +Modules/_sre.c - scanner_members - +Modules/_struct.c - s_members - +Modules/_threadmodule.c - local_dummy_type_members - +Modules/_threadmodule.c - local_type_members - +Modules/_threadmodule.c - lock_type_members - +Modules/_threadmodule.c - rlock_type_members - +Modules/_winapi.c - overlapped_members - +Modules/_zoneinfo.c - zoneinfo_members - +Modules/arraymodule.c - array_members - +Modules/cjkcodecs/multibytecodec.c - mbstreamreader_members - +Modules/cjkcodecs/multibytecodec.c - mbstreamwriter_members - +Modules/mmapmodule.c - mmap_object_members - Modules/ossaudiodev.c - oss_members - Modules/overlapped.c - Overlapped_members - -Modules/_pickle.c - Pickler_members - Modules/posixmodule.c - DirEntry_members - Modules/pyexpat.c - xmlparse_members - Modules/selectmodule.c - kqueue_event_members - Modules/sha256module.c - SHA_members - Modules/sha512module.c - SHA_members - Modules/socketmodule.c - sock_memberlist - -Modules/_sqlite/connection.c - connection_members - -Modules/_sqlite/cursor.c - cursor_members - -Modules/_sqlite/statement.c - stmt_members - -Modules/_sre.c - pattern_members - -Modules/_sre.c - match_members - -Modules/_sre.c - scanner_members - -Modules/_struct.c - s_members - Modules/unicodedata.c - DB_members - -Modules/_winapi.c - overlapped_members - Modules/xxsubtype.c - spamdict_members - Modules/zlibmodule.c - Decomp_members - -Modules/_zoneinfo.c - zoneinfo_members - -Objects/classobject.c - method_memberlist - Objects/classobject.c - instancemethod_memberlist - +Objects/classobject.c - method_memberlist - Objects/codeobject.c - code_memberlist - Objects/complexobject.c - complex_members - Objects/descrobject.c - descr_members - -Objects/descrobject.c - wrapper_members - Objects/descrobject.c - property_members - +Objects/descrobject.c - wrapper_members - +Objects/exceptions.c - AttributeError_members - +Objects/exceptions.c - BaseExceptionGroup_members - Objects/exceptions.c - BaseException_members - -Objects/exceptions.c - StopIteration_members - -Objects/exceptions.c - SystemExit_members - Objects/exceptions.c - ImportError_members - +Objects/exceptions.c - NameError_members - Objects/exceptions.c - OSError_members - +Objects/exceptions.c - StopIteration_members - Objects/exceptions.c - SyntaxError_members - +Objects/exceptions.c - SystemExit_members - Objects/exceptions.c - UnicodeError_members - Objects/frameobject.c - frame_memberlist - -Objects/funcobject.c - func_memberlist - Objects/funcobject.c - cm_memberlist - +Objects/funcobject.c - func_memberlist - Objects/funcobject.c - sm_memberlist - Objects/genericaliasobject.c - ga_members - -Objects/genobject.c - gen_memberlist - -Objects/genobject.c - coro_memberlist - Objects/genobject.c - async_gen_memberlist - +Objects/genobject.c - coro_memberlist - +Objects/genobject.c - gen_memberlist - Objects/methodobject.c - meth_members - Objects/moduleobject.c - module_members - Objects/namespaceobject.c - namespace_members - Objects/rangeobject.c - range_members - Objects/sliceobject.c - slice_members - -Objects/typeobject.c - type_members - Objects/typeobject.c - super_members - +Objects/typeobject.c - type_members - Objects/unionobject.c - union_members - Objects/weakrefobject.c - weakref_members - -Python/context.c - PyContextVar_members - Python/Python-ast.c - ast_type_members - +Python/context.c - PyContextVar_members - Python/symtable.c - ste_memberlist - Python/traceback.c - tb_memberlist - #----------------------- # for static types -# [] -# PyNumberMethods [] +# PyNumberMethods Modules/_collectionsmodule.c - deque_as_number - Modules/_collectionsmodule.c - defdict_as_number - Modules/_ctypes/_ctypes.c - PyCFuncPtr_as_number - @@ -865,6 +929,7 @@ Objects/descrobject.c - mappingproxy_as_number - Objects/dictobject.c - dict_as_number - Objects/dictobject.c - dictviews_as_number - Objects/floatobject.c - float_as_number - +Objects/genericaliasobject.c - ga_as_number - Objects/interpreteridobject.c - interpid_as_number - Objects/longobject.c - long_as_number - Objects/object.c - none_as_number - @@ -878,7 +943,7 @@ Objects/unicodeobject.c - unicode_as_number - Objects/unionobject.c - union_as_number - Objects/weakrefobject.c - proxy_as_number - -# PySequenceMethods [] +# PySequenceMethods Modules/arraymodule.c - array_as_sequence - Modules/_collectionsmodule.c - deque_as_sequence - Modules/_ctypes/_ctypes.c - CDataType_as_sequence - @@ -903,7 +968,7 @@ Objects/weakrefobject.c - proxy_as_sequence - Python/context.c - PyContext_as_sequence - Python/hamt.c - PyHamt_as_sequence - -# PyMappingMethods [] +# PyMappingMethods Modules/arraymodule.c - array_as_mapping - Modules/_ctypes/_ctypes.c - Array_as_mapping - Modules/_ctypes/_ctypes.c - Pointer_as_mapping - @@ -922,19 +987,23 @@ Objects/odictobject.c - odict_as_mapping - Objects/rangeobject.c - range_as_mapping - Objects/tupleobject.c - tuple_as_mapping - Objects/unicodeobject.c - unicode_as_mapping - +Objects/unionobject.c - union_as_mapping - Objects/weakrefobject.c - proxy_as_mapping - Python/context.c - PyContext_as_mapping - Python/hamt.c - PyHamtIterator_as_mapping - Python/hamt.c - PyHamt_as_mapping - -# PyAsyncMethods [] +# PyAsyncMethods +Modules/_asynciomodule.c - FutureIterType_as_async - Modules/_asynciomodule.c - FutureType_as_async - -Objects/genobject.c - coro_as_async - Objects/genobject.c - async_gen_as_async - Objects/genobject.c - async_gen_asend_as_async - Objects/genobject.c - async_gen_athrow_as_async - +Objects/genobject.c - coro_as_async - +Objects/genobject.c - gen_as_async - +Objects/iterobject.c - anextawaitable_as_async - -# PyBufferProcs [] +# PyBufferProcs Modules/arraymodule.c - array_as_buffer - Modules/_ctypes/_ctypes.c - PyCData_as_buffer - Modules/_io/bytesio.c - bytesiobuf_as_buffer - @@ -944,243 +1013,315 @@ Objects/bytesobject.c - bytes_as_buffer - Objects/memoryobject.c - memory_as_buffer - Objects/picklebufobject.c - picklebuf_as_buffer - -# PyGetSetDef [] -Modules/arraymodule.c - array_getsets - +# PyGetSetDef Modules/_asynciomodule.c - FutureType_getsetlist - Modules/_asynciomodule.c - TaskStepMethWrapper_getsetlist - Modules/_asynciomodule.c - TaskType_getsetlist - Modules/_blake2/blake2b_impl.c - py_blake2b_getsetters - Modules/_blake2/blake2s_impl.c - py_blake2s_getsetters - -Modules/cjkcodecs/multibytecodec.c - codecctx_getsets - Modules/_collectionsmodule.c - deque_getset - Modules/_csv.c - Dialect_getsetlist - -Modules/_ctypes/cfield.c - PyCField_getset - Modules/_ctypes/_ctypes.c - CharArray_getsets - -Modules/_ctypes/_ctypes.c - WCharArray_getsets - +Modules/_ctypes/_ctypes.c - Pointer_getsets - Modules/_ctypes/_ctypes.c - PyCFuncPtr_getsets - Modules/_ctypes/_ctypes.c - Simple_getsets - -Modules/_ctypes/_ctypes.c - Pointer_getsets - +Modules/_ctypes/_ctypes.c - WCharArray_getsets - +Modules/_ctypes/cfield.c - PyCField_getset - Modules/_cursesmodule.c - PyCursesWindow_getsets - Modules/_datetimemodule.c - date_getset - +Modules/_datetimemodule.c - datetime_getset - Modules/_datetimemodule.c - iso_calendar_date_getset - Modules/_datetimemodule.c - time_getset - -Modules/_datetimemodule.c - datetime_getset - Modules/_decimal/_decimal.c - context_getsets - Modules/_decimal/_decimal.c - dec_getsets - -Modules/_elementtree.c - xmlparser_getsetlist - Modules/_elementtree.c - element_getsetlist - -Modules/_functoolsmodule.c - partial_getsetlist - +Modules/_elementtree.c - xmlparser_getsetlist - Modules/_functoolsmodule.c - lru_cache_getsetlist - -Modules/_hashopenssl.c - EVP_getseters - +Modules/_functoolsmodule.c - partial_getsetlist - Modules/_hashopenssl.c - EVPXOF_getseters - +Modules/_hashopenssl.c - EVP_getseters - Modules/_hashopenssl.c - HMAC_getset - +Modules/_io/bufferedio.c - bufferedrandom_getset - Modules/_io/bufferedio.c - bufferedreader_getset - -Modules/_io/bufferedio.c - bufferedwriter_getset - Modules/_io/bufferedio.c - bufferedrwpair_getset - -Modules/_io/bufferedio.c - bufferedrandom_getset - +Modules/_io/bufferedio.c - bufferedwriter_getset - Modules/_io/bytesio.c - bytesio_getsetlist - Modules/_io/fileio.c - fileio_getsetlist - Modules/_io/iobase.c - iobase_getset - Modules/_io/stringio.c - stringio_getset - -Modules/_io/textio.c - textiobase_getset - Modules/_io/textio.c - incrementalnewlinedecoder_getset - +Modules/_io/textio.c - textiobase_getset - Modules/_io/textio.c - textiowrapper_getset - Modules/_io/winconsoleio.c - winconsoleio_getsetlist - +Modules/_pickle.c - Pickler_getsets - +Modules/_pickle.c - Unpickler_getsets - +Modules/_sha3/sha3module.c - SHA3_getseters - +Modules/_sqlite/connection.c - connection_getset - +Modules/_sre.c - match_getset - +Modules/_sre.c - pattern_getset - +Modules/_ssl.c - PySSLSession_getsetlist - +Modules/_ssl.c - context_getsetlist - +Modules/_ssl.c - memory_bio_getsetlist - +Modules/_ssl.c - ssl_getsetlist - +Modules/_struct.c - s_getsetlist - +Modules/_tkinter.c - PyTclObject_getsetlist - +Modules/_xxsubinterpretersmodule.c - channelid_getsets - +Modules/arraymodule.c - array_getsets - +Modules/cjkcodecs/multibytecodec.c - codecctx_getsets - Modules/md5module.c - MD5_getseters - Modules/mmapmodule.c - mmap_object_getset - Modules/ossaudiodev.c - oss_getsetlist - Modules/overlapped.c - Overlapped_getsets - -Modules/_pickle.c - Pickler_getsets - -Modules/_pickle.c - Unpickler_getsets - Modules/pyexpat.c - xmlparse_getsetlist - Modules/selectmodule.c - devpoll_getsetlist - -Modules/selectmodule.c - pyepoll_getsetlist - Modules/selectmodule.c - kqueue_queue_getsetlist - +Modules/selectmodule.c - pyepoll_getsetlist - Modules/sha1module.c - SHA1_getseters - Modules/sha256module.c - SHA_getseters - -Modules/_sha3/sha3module.c - SHA3_getseters - Modules/sha512module.c - SHA_getseters - Modules/socketmodule.c - sock_getsetlist - -Modules/_sqlite/connection.c - connection_getset - -Modules/_sre.c - pattern_getset - -Modules/_sre.c - match_getset - -Modules/_ssl.c - ssl_getsetlist - -Modules/_ssl.c - context_getsetlist - -Modules/_ssl.c - memory_bio_getsetlist - -Modules/_ssl.c - PySSLSession_getsetlist - -Modules/_struct.c - s_getsetlist - -Modules/_tkinter.c - PyTclObject_getsetlist - -Modules/_xxsubinterpretersmodule.c - channelid_getsets - +Modules/xxlimited.c - Xxo_getsetlist - Modules/xxsubtype.c - spamlist_getsets - Objects/cellobject.c - cell_getsetlist - -Objects/classobject.c - method_getset - Objects/classobject.c - instancemethod_getset - -Objects/descrobject.c - method_getset - -Objects/descrobject.c - member_getset - +Objects/classobject.c - method_getset - +Objects/codeobject.c - code_getsetlist - Objects/descrobject.c - getset_getset - -Objects/descrobject.c - wrapperdescr_getset - -Objects/descrobject.c - wrapper_getsets - +Objects/descrobject.c - member_getset - +Objects/descrobject.c - method_getset - Objects/descrobject.c - property_getsetlist - +Objects/descrobject.c - wrapper_getsets - +Objects/descrobject.c - wrapperdescr_getset - Objects/dictobject.c - dictview_getset - Objects/exceptions.c - BaseException_getset - Objects/exceptions.c - OSError_getset - Objects/fileobject.c - stdprinter_getsetlist - Objects/floatobject.c - float_getset - Objects/frameobject.c - frame_getsetlist - -Objects/funcobject.c - func_getsetlist - Objects/funcobject.c - cm_getsetlist - +Objects/funcobject.c - func_getsetlist - Objects/funcobject.c - sm_getsetlist - Objects/genericaliasobject.c - ga_properties - -Objects/genobject.c - gen_getsetlist - -Objects/genobject.c - coro_getsetlist - Objects/genobject.c - async_gen_getsetlist - +Objects/genobject.c - coro_getsetlist - +Objects/genobject.c - gen_getsetlist - Objects/longobject.c - long_getset - Objects/memoryobject.c - memory_getsetlist - Objects/methodobject.c - meth_getsets - +Objects/moduleobject.c - module_getsets - Objects/odictobject.c - odict_getset - -Objects/typeobject.c - type_getsets - -Objects/typeobject.c - subtype_getsets_full - +Objects/typeobject.c - object_getsets - Objects/typeobject.c - subtype_getsets_dict_only - +Objects/typeobject.c - subtype_getsets_full - Objects/typeobject.c - subtype_getsets_weakref_only - -Objects/typeobject.c - object_getsets - -Python/context.c - PyContextTokenType_getsetlist - +Objects/typeobject.c - type_getsets - +Objects/unionobject.c - union_properties - Python/Python-ast.c - ast_type_getsets - +Python/context.c - PyContextTokenType_getsetlist - Python/traceback.c - tb_getsetters - #----------------------- # for heap types -# [] -# PyType_Slot [] +# PyType_Slot Modules/_abc.c - _abc_data_type_spec_slots - Modules/_blake2/blake2b_impl.c - blake2b_type_slots - Modules/_blake2/blake2s_impl.c - blake2s_type_slots - Modules/_bz2module.c - bz2_compressor_type_slots - Modules/_bz2module.c - bz2_decompressor_type_slots - +Modules/_csv.c - Dialect_Type_slots - +Modules/_csv.c - Reader_Type_slots - +Modules/_csv.c - Writer_Type_slots - +Modules/_csv.c - error_slots - Modules/_curses_panel.c - PyCursesPanel_Type_slots - Modules/_dbmmodule.c - dbmtype_spec_slots - +Modules/_functoolsmodule.c - keyobject_type_slots - +Modules/_functoolsmodule.c - lru_cache_type_slots - +Modules/_functoolsmodule.c - lru_list_elem_type_slots - +Modules/_functoolsmodule.c - partial_type_slots - Modules/_gdbmmodule.c - gdbmtype_spec_slots - -Modules/_hashopenssl.c - EVPtype_slots - Modules/_hashopenssl.c - EVPXOFtype_slots - +Modules/_hashopenssl.c - EVPtype_slots - Modules/_hashopenssl.c - HMACtype_slots - -Modules/_json.c - PyScannerType_slots - Modules/_json.c - PyEncoderType_slots - +Modules/_json.c - PyScannerType_slots - Modules/_lsprof.c - _lsprof_profiler_type_spec_slots - Modules/_lzmamodule.c - lzma_compressor_type_slots - Modules/_lzmamodule.c - lzma_decompressor_type_slots - -Modules/md5module.c - md5_type_slots - -Modules/_operator.c - itemgetter_type_slots - Modules/_operator.c - attrgetter_type_slots - +Modules/_operator.c - itemgetter_type_slots - Modules/_operator.c - methodcaller_type_slots - -Modules/overlapped.c - overlapped_type_slots - -Modules/posixmodule.c - DirEntryType_slots - -Modules/posixmodule.c - ScandirIteratorType_slots - +Modules/_queuemodule.c - simplequeue_slots - Modules/_randommodule.c - Random_Type_slots - -Modules/selectmodule.c - devpoll_Type_slots - -Modules/selectmodule.c - kqueue_event_Type_slots - -Modules/selectmodule.c - poll_Type_slots - -Modules/selectmodule.c - pyEpoll_Type_slots - -Modules/selectmodule.c - kqueue_queue_Type_slots - -Modules/sha1module.c - sha1_type_slots - -Modules/sha256module.c - sha256_types_slots - +Modules/_sha3/sha3module.c - SHAKE128slots - +Modules/_sha3/sha3module.c - SHAKE256slots - Modules/_sha3/sha3module.c - sha3_224_slots - Modules/_sha3/sha3module.c - sha3_256_slots - Modules/_sha3/sha3module.c - sha3_384_slots - Modules/_sha3/sha3module.c - sha3_512_slots - -Modules/_sha3/sha3module.c - SHAKE128slots - -Modules/_sha3/sha3module.c - SHAKE256slots - Modules/_sha3/sha3module.c - type_slots_obj - -Modules/sha512module.c - sha512_sha384_type_slots - -Modules/sha512module.c - sha512_sha512_type_slots - Modules/_sqlite/connection.c - connection_slots - Modules/_sqlite/cursor.c - cursor_slots - Modules/_sqlite/prepare_protocol.c - type_slots - Modules/_sqlite/row.c - row_slots - Modules/_sqlite/statement.c - stmt_slots - +Modules/_sre.c - match_slots - +Modules/_sre.c - pattern_slots - +Modules/_sre.c - scanner_slots - +Modules/_ssl.c - PySSLContext_slots - +Modules/_ssl.c - PySSLMemoryBIO_slots - +Modules/_ssl.c - PySSLSession_slots - +Modules/_ssl.c - PySSLSocket_slots - Modules/_ssl.c - sslerror_type_slots - -Modules/_struct.c - unpackiter_type_slots - +Modules/_ssl/cert.c - PySSLCertificate_slots - Modules/_struct.c - PyStructType_slots - +Modules/_struct.c - unpackiter_type_slots - +Modules/_testcapimodule.c - HeapTypeNameType_slots - +Modules/_testcapimodule.c - NullTpDocType_slots - +Modules/_threadmodule.c - local_dummy_type_slots - +Modules/_threadmodule.c - local_type_slots - +Modules/_threadmodule.c - lock_type_slots - +Modules/_threadmodule.c - rlock_type_slots - Modules/_tkinter.c - PyTclObject_Type_slots - -Modules/_tkinter.c - Tktt_Type_slots - Modules/_tkinter.c - Tkapp_Type_slots - -Modules/unicodedata.c - ucd_type_slots - +Modules/_tkinter.c - Tktt_Type_slots - Modules/_winapi.c - winapi_overlapped_type_slots - -Modules/xxlimited.c - Xxo_Type_slots - -Modules/xxlimited.c - Str_Type_slots - +Modules/arraymodule.c - array_slots - +Modules/arraymodule.c - arrayiter_slots - +Modules/cjkcodecs/multibytecodec.c - decoder_slots - +Modules/cjkcodecs/multibytecodec.c - encoder_slots - +Modules/cjkcodecs/multibytecodec.c - multibytecodec_slots - +Modules/cjkcodecs/multibytecodec.c - reader_slots - +Modules/cjkcodecs/multibytecodec.c - writer_slots - +Modules/md5module.c - md5_type_slots - +Modules/mmapmodule.c - mmap_object_slots - +Modules/overlapped.c - overlapped_type_slots - +Modules/posixmodule.c - DirEntryType_slots - +Modules/posixmodule.c - ScandirIteratorType_slots - +Modules/pyexpat.c - _xml_parse_type_spec_slots - +Modules/selectmodule.c - devpoll_Type_slots - +Modules/selectmodule.c - kqueue_event_Type_slots - +Modules/selectmodule.c - kqueue_queue_Type_slots - +Modules/selectmodule.c - poll_Type_slots - +Modules/selectmodule.c - pyEpoll_Type_slots - +Modules/sha1module.c - sha1_type_slots - +Modules/sha256module.c - sha256_types_slots - +Modules/sha512module.c - sha512_sha384_type_slots - +Modules/sha512module.c - sha512_sha512_type_slots - +Modules/unicodedata.c - ucd_type_slots - Modules/xxlimited.c - Null_Type_slots - +Modules/xxlimited.c - Str_Type_slots - +Modules/xxlimited.c - Xxo_Type_slots - +Modules/xxlimited_35.c - Null_Type_slots - +Modules/xxlimited_35.c - Str_Type_slots - +Modules/xxlimited_35.c - Xxo_Type_slots - Modules/zlibmodule.c - Comptype_slots - Modules/zlibmodule.c - Decomptype_slots - Python/Python-ast.c - AST_type_slots - -# PyType_Spec [] +# PyType_Spec Modules/_abc.c - _abc_data_type_spec - Modules/_blake2/blake2b_impl.c - blake2b_type_spec - Modules/_blake2/blake2s_impl.c - blake2s_type_spec - Modules/_bz2module.c - bz2_compressor_type_spec - Modules/_bz2module.c - bz2_decompressor_type_spec - +Modules/_csv.c - Dialect_Type_spec - +Modules/_csv.c - Reader_Type_spec - +Modules/_csv.c - Writer_Type_spec - +Modules/_csv.c - error_spec - Modules/_curses_panel.c - PyCursesPanel_Type_spec - Modules/_dbmmodule.c - dbmtype_spec - +Modules/_functoolsmodule.c - keyobject_type_spec - +Modules/_functoolsmodule.c - lru_cache_type_spec - +Modules/_functoolsmodule.c - lru_list_elem_type_spec - +Modules/_functoolsmodule.c - partial_type_spec - Modules/_gdbmmodule.c - gdbmtype_spec - -Modules/_hashopenssl.c - EVPtype_spec - Modules/_hashopenssl.c - EVPXOFtype_spec - +Modules/_hashopenssl.c - EVPtype_spec - Modules/_hashopenssl.c - HMACtype_spec - -Modules/_json.c - PyScannerType_spec - Modules/_json.c - PyEncoderType_spec - +Modules/_json.c - PyScannerType_spec - Modules/_lsprof.c - _lsprof_profiler_type_spec - Modules/_lzmamodule.c - lzma_compressor_type_spec - Modules/_lzmamodule.c - lzma_decompressor_type_spec - -Modules/_operator.c - itemgetter_type_spec - Modules/_operator.c - attrgetter_type_spec - +Modules/_operator.c - itemgetter_type_spec - Modules/_operator.c - methodcaller_type_spec - +Modules/_queuemodule.c - simplequeue_spec - Modules/_randommodule.c - Random_Type_spec - +Modules/_sha3/sha3module.c - SHAKE128_spec - +Modules/_sha3/sha3module.c - SHAKE256_spec - Modules/_sha3/sha3module.c - sha3_224_spec - Modules/_sha3/sha3module.c - sha3_256_spec - Modules/_sha3/sha3module.c - sha3_384_spec - Modules/_sha3/sha3module.c - sha3_512_spec - -Modules/_sha3/sha3module.c - SHAKE128_spec - -Modules/_sha3/sha3module.c - SHAKE256_spec - Modules/_sha3/sha3module.c - type_spec_obj - Modules/_sqlite/connection.c - connection_spec - Modules/_sqlite/cursor.c - cursor_spec - Modules/_sqlite/prepare_protocol.c - type_spec - Modules/_sqlite/row.c - row_spec - Modules/_sqlite/statement.c - stmt_spec - +Modules/_sre.c - match_spec - +Modules/_sre.c - pattern_spec - +Modules/_sre.c - scanner_spec - +Modules/_ssl.c - PySSLContext_spec - +Modules/_ssl.c - PySSLMemoryBIO_spec - +Modules/_ssl.c - PySSLSession_spec - +Modules/_ssl.c - PySSLSocket_spec - Modules/_ssl.c - sslerror_type_spec - -Modules/_struct.c - unpackiter_type_spec - +Modules/_ssl/cert.c - PySSLCertificate_spec - Modules/_struct.c - PyStructType_spec - +Modules/_struct.c - unpackiter_type_spec - +Modules/_testcapimodule.c - HeapTypeNameType_Spec - +Modules/_testcapimodule.c - NullTpDocType_spec - +Modules/_threadmodule.c - local_dummy_type_spec - +Modules/_threadmodule.c - local_type_spec - +Modules/_threadmodule.c - lock_type_spec - +Modules/_threadmodule.c - rlock_type_spec - Modules/_tkinter.c - PyTclObject_Type_spec - -Modules/_tkinter.c - Tktt_Type_spec - Modules/_tkinter.c - Tkapp_Type_spec - +Modules/_tkinter.c - Tktt_Type_spec - Modules/_winapi.c - winapi_overlapped_type_spec - -Modules/_zoneinfo.c - DAYS_IN_MONTH - Modules/_zoneinfo.c - DAYS_BEFORE_MONTH - +Modules/_zoneinfo.c - DAYS_IN_MONTH - +Modules/arraymodule.c - array_spec - +Modules/arraymodule.c - arrayiter_spec - +Modules/cjkcodecs/multibytecodec.c - decoder_spec - +Modules/cjkcodecs/multibytecodec.c - encoder_spec - +Modules/cjkcodecs/multibytecodec.c - multibytecodec_spec - +Modules/cjkcodecs/multibytecodec.c - reader_spec - +Modules/cjkcodecs/multibytecodec.c - writer_spec - Modules/md5module.c - md5_type_spec - +Modules/mmapmodule.c - mmap_object_spec - Modules/overlapped.c - overlapped_type_spec - Modules/posixmodule.c - DirEntryType_spec - Modules/posixmodule.c - ScandirIteratorType_spec - +Modules/pyexpat.c - _xml_parse_type_spec - Modules/selectmodule.c - devpoll_Type_spec - Modules/selectmodule.c - kqueue_event_Type_spec - +Modules/selectmodule.c - kqueue_queue_Type_spec - Modules/selectmodule.c - poll_Type_spec - Modules/selectmodule.c - pyEpoll_Type_spec - -Modules/selectmodule.c - kqueue_queue_Type_spec - Modules/sha1module.c - sha1_type_spec - Modules/sha256module.c - sha224_type_spec - Modules/sha256module.c - sha256_type_spec - Modules/sha512module.c - sha512_sha384_type_spec - Modules/sha512module.c - sha512_sha512_type_spec - Modules/unicodedata.c - ucd_type_spec - -Modules/xxlimited.c - Xxo_Type_spec - -Modules/xxlimited.c - Str_Type_spec - Modules/xxlimited.c - Null_Type_spec - +Modules/xxlimited.c - Str_Type_spec - +Modules/xxlimited.c - Xxo_Type_spec - +Modules/xxlimited_35.c - Null_Type_spec - +Modules/xxlimited_35.c - Str_Type_spec - +Modules/xxlimited_35.c - Xxo_Type_spec - Modules/zlibmodule.c - Comptype_spec - Modules/zlibmodule.c - Decomptype_spec - Python/Python-ast.c - AST_type_spec - #----------------------- # for structseq -# [] -# PyStructSequence_Field[] [] +# PyStructSequence_Field[] Modules/_cursesmodule.c - ncurses_version_fields - Modules/grpmodule.c - struct_group_type_fields - Modules/_lsprof.c - profiler_entry_fields - @@ -1208,7 +1349,7 @@ Python/sysmodule.c - flags_fields - Python/sysmodule.c - version_info_fields - Python/thread.c - threadinfo_fields - -# PyStructSequence_Desc [] +# PyStructSequence_Desc Modules/_cursesmodule.c - ncurses_version_desc - Modules/grpmodule.c - struct_group_type_desc - Modules/_lsprof.c - profiler_entry_desc - @@ -1236,243 +1377,23 @@ Python/sysmodule.c - flags_desc - Python/sysmodule.c - version_info_desc - Python/thread.c - threadinfo_desc - -#----------------------- -# _PyArg_Parser -# [] - -Modules/clinic/md5module.c.h MD5Type_copy _parser - -Modules/clinic/md5module.c.h _md5_md5 _parser - -Modules/clinic/_dbmmodule.c.h _dbm_dbm_keys _parser - -Modules/clinic/_dbmmodule.c.h _dbm_dbm_get _parser - -Modules/clinic/_dbmmodule.c.h _dbm_dbm_setdefault _parser - -Modules/clinic/posixmodule.c.h os_stat _parser - -Modules/clinic/posixmodule.c.h os_lstat _parser - -Modules/clinic/posixmodule.c.h os_access _parser - -Modules/clinic/posixmodule.c.h os_chdir _parser - -Modules/clinic/posixmodule.c.h os_chmod _parser - -Modules/clinic/posixmodule.c.h os_listdir _parser - -Modules/clinic/posixmodule.c.h os_mkdir _parser - -Modules/clinic/posixmodule.c.h os_rename _parser - -Modules/clinic/posixmodule.c.h os_replace _parser - -Modules/clinic/posixmodule.c.h os_rmdir _parser - -Modules/clinic/posixmodule.c.h os_unlink _parser - -Modules/clinic/posixmodule.c.h os_remove _parser - -Modules/clinic/posixmodule.c.h os_utime _parser - -Modules/clinic/posixmodule.c.h os__exit _parser - -Modules/clinic/posixmodule.c.h os_open _parser - -Modules/clinic/posixmodule.c.h os_close _parser - -Modules/clinic/posixmodule.c.h os_dup2 _parser - -Modules/clinic/posixmodule.c.h os_fstat _parser - -Modules/clinic/posixmodule.c.h os_device_encoding _parser - -Modules/clinic/posixmodule.c.h os_DirEntry_is_symlink _parser - -Modules/clinic/posixmodule.c.h os_DirEntry_stat _parser - -Modules/clinic/posixmodule.c.h os_DirEntry_is_dir _parser - -Modules/clinic/posixmodule.c.h os_DirEntry_is_file _parser - -Modules/clinic/posixmodule.c.h os_scandir _parser - -Modules/clinic/posixmodule.c.h os_fspath _parser - -Modules/clinic/cmathmodule.c.h cmath_isclose _parser - -Modules/clinic/sha256module.c.h SHA256Type_copy _parser - -Modules/clinic/sha256module.c.h _sha256_sha256 _parser - -Modules/clinic/sha256module.c.h _sha256_sha224 _parser - -Modules/clinic/_hashopenssl.c.h EVP_new _parser - -Modules/clinic/_hashopenssl.c.h _hashlib_openssl_md5 _parser - -Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha1 _parser - -Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha224 _parser - -Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha256 _parser - -Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha384 _parser - -Modules/clinic/_hashopenssl.c.h _hashlib_openssl_sha512 _parser - -Modules/clinic/_hashopenssl.c.h pbkdf2_hmac _parser - -Modules/clinic/_hashopenssl.c.h _hashlib_hmac_singleshot _parser - -Modules/clinic/_hashopenssl.c.h _hashlib_hmac_new _parser - -Modules/clinic/_hashopenssl.c.h _hashlib_HMAC_update _parser - -Modules/clinic/_ssl.c.h _ssl__SSLSocket_get_channel_binding _parser - -Modules/clinic/_ssl.c.h _ssl__SSLContext_load_cert_chain _parser - -Modules/clinic/_ssl.c.h _ssl__SSLContext_load_verify_locations _parser - -Modules/clinic/_ssl.c.h _ssl__SSLContext__wrap_socket _parser - -Modules/clinic/_ssl.c.h _ssl__SSLContext__wrap_bio _parser - -Modules/clinic/_ssl.c.h _ssl__SSLContext_get_ca_certs _parser - -Modules/clinic/_ssl.c.h _ssl_txt2obj _parser - -Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_put _parser - -Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_put_nowait _parser - -Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_get _parser - -Modules/clinic/_lsprof.c.h _lsprof_Profiler_getstats _parser - -Modules/clinic/_datetimemodule.c.h iso_calendar_date_new _parser - -Modules/clinic/_datetimemodule.c.h datetime_datetime_now _parser - -Modules/clinic/_opcode.c.h _opcode_stack_effect _parser - -Modules/clinic/_lzmamodule.c.h _lzma_LZMADecompressor_decompress _parser - -Modules/clinic/_lzmamodule.c.h _lzma_LZMADecompressor___init__ _parser - -Modules/clinic/pyexpat.c.h pyexpat_ParserCreate _parser - -Modules/clinic/mathmodule.c.h math_isclose _parser - -Modules/clinic/mathmodule.c.h math_prod _parser - -Modules/clinic/_curses_panel.c.h _curses_panel_panel_bottom _parser - -Modules/clinic/_curses_panel.c.h _curses_panel_panel_hide _parser - -Modules/clinic/_curses_panel.c.h _curses_panel_panel_show _parser - -Modules/clinic/_curses_panel.c.h _curses_panel_panel_top _parser - -Modules/clinic/_curses_panel.c.h _curses_panel_panel_move _parser - -Modules/clinic/_curses_panel.c.h _curses_panel_panel_replace _parser - -Modules/clinic/_curses_panel.c.h _curses_panel_panel_set_userptr _parser - -Modules/clinic/_curses_panel.c.h _curses_panel_panel_userptr _parser - -Modules/clinic/_elementtree.c.h _elementtree_Element_find _parser - -Modules/clinic/_elementtree.c.h _elementtree_Element_findtext _parser - -Modules/clinic/_elementtree.c.h _elementtree_Element_findall _parser - -Modules/clinic/_elementtree.c.h _elementtree_Element_iterfind _parser - -Modules/clinic/_elementtree.c.h _elementtree_Element_get _parser - -Modules/clinic/_elementtree.c.h _elementtree_Element_iter _parser - -Modules/clinic/_elementtree.c.h _elementtree_TreeBuilder___init__ _parser - -Modules/clinic/_elementtree.c.h _elementtree_XMLParser___init__ _parser - -Modules/clinic/_asynciomodule.c.h _asyncio_Future___init__ _parser - -Modules/clinic/_asynciomodule.c.h _asyncio_Future_add_done_callback _parser - -Modules/clinic/_asynciomodule.c.h _asyncio_Future_cancel _parser - -Modules/clinic/_asynciomodule.c.h _asyncio_Task___init__ _parser - -Modules/clinic/_asynciomodule.c.h _asyncio_Task_cancel _parser - -Modules/clinic/_asynciomodule.c.h _asyncio_Task_get_stack _parser - -Modules/clinic/_asynciomodule.c.h _asyncio_Task_print_stack _parser - -Modules/clinic/_asynciomodule.c.h _asyncio__register_task _parser - -Modules/clinic/_asynciomodule.c.h _asyncio__unregister_task _parser - -Modules/clinic/_asynciomodule.c.h _asyncio__enter_task _parser - -Modules/clinic/_asynciomodule.c.h _asyncio__leave_task _parser - -Modules/clinic/gcmodule.c.h gc_collect _parser - -Modules/clinic/gcmodule.c.h gc_get_objects _parser - -Modules/clinic/grpmodule.c.h grp_getgrgid _parser - -Modules/clinic/grpmodule.c.h grp_getgrnam _parser - -Modules/clinic/_pickle.c.h _pickle_Pickler___init__ _parser - -Modules/clinic/_pickle.c.h _pickle_Unpickler___init__ _parser - -Modules/clinic/_pickle.c.h _pickle_dump _parser - -Modules/clinic/_pickle.c.h _pickle_dumps _parser - -Modules/clinic/_pickle.c.h _pickle_load _parser - -Modules/clinic/_pickle.c.h _pickle_loads _parser - -Modules/clinic/_struct.c.h Struct___init__ _parser - -Modules/clinic/_struct.c.h Struct_unpack_from _parser - -Modules/clinic/_struct.c.h unpack_from _parser - -Modules/clinic/_testmultiphase.c.h _testmultiphase_StateAccessType_get_defining_module _parser - -Modules/clinic/_testmultiphase.c.h _testmultiphase_StateAccessType_increment_count_clinic _parser - -Modules/clinic/_testmultiphase.c.h _testmultiphase_StateAccessType_get_count _parser - -Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_keys _parser - -Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_firstkey _parser - -Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_nextkey _parser - -Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_reorganize _parser - -Modules/clinic/_gdbmmodule.c.h _gdbm_gdbm_sync _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_match _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_fullmatch _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_search _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_findall _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_finditer _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_scanner _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_split _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_sub _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_subn _parser - -Modules/clinic/_sre.c.h _sre_compile _parser - -Modules/clinic/_sre.c.h _sre_SRE_Match_expand _parser - -Modules/clinic/_sre.c.h _sre_SRE_Match_groups _parser - -Modules/clinic/_sre.c.h _sre_SRE_Match_groupdict _parser - -Modules/clinic/overlapped.c.h _overlapped_Overlapped _parser - -Modules/clinic/_bisectmodule.c.h _bisect_bisect_right _parser - -Modules/clinic/_bisectmodule.c.h _bisect_insort_right _parser - -Modules/clinic/_bisectmodule.c.h _bisect_bisect_left _parser - -Modules/clinic/_bisectmodule.c.h _bisect_insort_left _parser - -Modules/clinic/zlibmodule.c.h zlib_compress _parser - -Modules/clinic/zlibmodule.c.h zlib_decompress _parser - -Modules/clinic/zlibmodule.c.h zlib_compressobj _parser - -Modules/clinic/zlibmodule.c.h zlib_decompressobj _parser - -Modules/clinic/zlibmodule.c.h zlib_Compress_compress _parser - -Modules/clinic/zlibmodule.c.h zlib_Decompress_decompress _parser - -Modules/clinic/zlibmodule.c.h zlib_Compress_flush _parser - -Modules/clinic/zlibmodule.c.h zlib_Decompress_flush _parser - -Modules/clinic/sha512module.c.h SHA512Type_copy _parser - -Modules/clinic/sha512module.c.h _sha512_sha512 _parser - -Modules/clinic/sha512module.c.h _sha512_sha384 _parser - -Modules/clinic/_bz2module.c.h _bz2_BZ2Decompressor_decompress _parser - -Modules/clinic/sha1module.c.h SHA1Type_copy _parser - -Modules/clinic/sha1module.c.h _sha1_sha1 _parser - -Modules/clinic/_winapi.c.h _winapi_ConnectNamedPipe _parser - -Modules/clinic/_winapi.c.h _winapi_ReadFile _parser - -Modules/clinic/_winapi.c.h _winapi_WriteFile _parser - -Modules/clinic/_winapi.c.h _winapi_GetFileType _parser - -Modules/clinic/_codecsmodule.c.h _codecs_encode _parser - -Modules/clinic/_codecsmodule.c.h _codecs_decode _parser - -Modules/clinic/_cursesmodule.c.h _curses_setupterm _parser - -Modules/clinic/itertoolsmodule.c.h itertools_groupby _parser - -Modules/clinic/itertoolsmodule.c.h itertools_combinations _parser - -Modules/clinic/itertoolsmodule.c.h itertools_combinations_with_replacement _parser - -Modules/clinic/itertoolsmodule.c.h itertools_permutations _parser - -Modules/clinic/itertoolsmodule.c.h itertools_accumulate _parser - -Modules/clinic/itertoolsmodule.c.h itertools_compress _parser - -Modules/clinic/itertoolsmodule.c.h itertools_count _parser - -Modules/clinic/binascii.c.h binascii_b2a_uu _parser - -Modules/clinic/binascii.c.h binascii_b2a_base64 _parser - -Modules/clinic/binascii.c.h binascii_b2a_hex _parser - -Modules/clinic/binascii.c.h binascii_hexlify _parser - -Modules/clinic/binascii.c.h binascii_a2b_qp _parser - -Modules/clinic/binascii.c.h binascii_b2a_qp _parser - -Objects/clinic/enumobject.c.h enum_new _parser - -Objects/clinic/bytearrayobject.c.h bytearray___init__ _parser - -Objects/clinic/bytearrayobject.c.h bytearray_translate _parser - -Objects/clinic/bytearrayobject.c.h bytearray_split _parser - -Objects/clinic/bytearrayobject.c.h bytearray_rsplit _parser - -Objects/clinic/bytearrayobject.c.h bytearray_decode _parser - -Objects/clinic/bytearrayobject.c.h bytearray_splitlines _parser - -Objects/clinic/bytearrayobject.c.h bytearray_hex _parser - -Objects/clinic/descrobject.c.h mappingproxy_new _parser - -Objects/clinic/descrobject.c.h property_init _parser - -Objects/clinic/longobject.c.h long_new _parser - -Objects/clinic/longobject.c.h int_to_bytes _parser - -Objects/clinic/longobject.c.h int_from_bytes _parser - -Objects/clinic/moduleobject.c.h module___init__ _parser - -Objects/clinic/structseq.c.h structseq_new _parser - -Objects/clinic/memoryobject.c.h memoryview _parser - -Objects/clinic/memoryobject.c.h memoryview_cast _parser - -Objects/clinic/memoryobject.c.h memoryview_tobytes _parser - -Objects/clinic/memoryobject.c.h memoryview_hex _parser - -Objects/clinic/listobject.c.h list_sort _parser - -Objects/clinic/odictobject.c.h OrderedDict_fromkeys _parser - -Objects/clinic/odictobject.c.h OrderedDict_setdefault _parser - -Objects/clinic/odictobject.c.h OrderedDict_pop _parser - -Objects/clinic/odictobject.c.h OrderedDict_popitem _parser - -Objects/clinic/odictobject.c.h OrderedDict_move_to_end _parser - -Objects/clinic/complexobject.c.h complex_new _parser - -Objects/clinic/unicodeobject.c.h unicode_encode _parser - -Objects/clinic/unicodeobject.c.h unicode_expandtabs _parser - -Objects/clinic/unicodeobject.c.h unicode_split _parser - -Objects/clinic/unicodeobject.c.h unicode_rsplit _parser - -Objects/clinic/unicodeobject.c.h unicode_splitlines _parser - -Objects/clinic/unicodeobject.c.h unicode_new _parser - -Objects/clinic/bytesobject.c.h bytes_split _parser - -Objects/clinic/bytesobject.c.h bytes_rsplit _parser - -Objects/clinic/bytesobject.c.h bytes_translate _parser - -Objects/clinic/bytesobject.c.h bytes_decode _parser - -Objects/clinic/bytesobject.c.h bytes_splitlines _parser - -Objects/clinic/bytesobject.c.h bytes_hex _parser - -Objects/clinic/bytesobject.c.h bytes_new _parser - -Objects/clinic/funcobject.c.h func_new _parser - -Objects/clinic/codeobject.c.h code_replace _parser - -Python/clinic/traceback.c.h tb_new _parser - -Python/clinic/bltinmodule.c.h builtin_compile _parser - -Python/clinic/bltinmodule.c.h builtin_pow _parser - -Python/clinic/bltinmodule.c.h builtin_round _parser - -Python/clinic/bltinmodule.c.h builtin_sum _parser - -Python/clinic/import.c.h _imp_source_hash _parser - -Python/clinic/sysmodule.c.h sys_addaudithook _parser - -Python/clinic/sysmodule.c.h sys_set_coroutine_origin_tracking_depth _parser - -Python/clinic/_warnings.c.h warnings_warn _parser - - #----------------------- # other vars that are actually constant -# [] Modules/_csv.c - quote_styles - -Modules/_ctypes/cfield.c - ffi_type_void - -Modules/_ctypes/cfield.c - ffi_type_uint8 - +Modules/_ctypes/cfield.c - ffi_type_double - +Modules/_ctypes/cfield.c - ffi_type_float - +Modules/_ctypes/cfield.c - ffi_type_longdouble - +Modules/_ctypes/cfield.c - ffi_type_pointer - +Modules/_ctypes/cfield.c - ffi_type_sint16 - +Modules/_ctypes/cfield.c - ffi_type_sint32 - +Modules/_ctypes/cfield.c - ffi_type_sint64 - Modules/_ctypes/cfield.c - ffi_type_sint8 - Modules/_ctypes/cfield.c - ffi_type_uint16 - -Modules/_ctypes/cfield.c - ffi_type_sint16 - Modules/_ctypes/cfield.c - ffi_type_uint32 - -Modules/_ctypes/cfield.c - ffi_type_sint32 - Modules/_ctypes/cfield.c - ffi_type_uint64 - -Modules/_ctypes/cfield.c - ffi_type_sint64 - -Modules/_ctypes/cfield.c - ffi_type_float - -Modules/_ctypes/cfield.c - ffi_type_double - -Modules/_ctypes/cfield.c - ffi_type_longdouble - -Modules/_ctypes/cfield.c - ffi_type_pointer - +Modules/_ctypes/cfield.c - ffi_type_uint8 - +Modules/_ctypes/cfield.c - ffi_type_void - Modules/_datetimemodule.c - epoch - Modules/_datetimemodule.c - max_fold_seconds - Modules/_datetimemodule.c datetime_isoformat specs - @@ -1486,14 +1407,16 @@ Modules/_decimal/_decimal.c - invalid_signals_err - Modules/_decimal/_decimal.c - signal_map - Modules/_decimal/_decimal.c - ssize_constants - Modules/_elementtree.c - ExpatMemoryHandler - +Modules/_io/_iomodule.c - static_types - Modules/_io/textio.c - encodefuncs - Modules/_localemodule.c - langinfo_constants - +Modules/_sqlite/module.c - error_codes - Modules/_sre.c pattern_repr flag_names - Modules/_struct.c - bigendian_table - Modules/_struct.c - lilendian_table - Modules/_tkinter.c - state_key - -Modules/_xxsubinterpretersmodule.c - _channelid_end_send - Modules/_xxsubinterpretersmodule.c - _channelid_end_recv - +Modules/_xxsubinterpretersmodule.c - _channelid_end_send - Modules/arraymodule.c - descriptors - Modules/arraymodule.c - emptybuf - Modules/cjkcodecs/cjkcodecs.h - __methods - @@ -1504,1396 +1427,46 @@ Modules/cmathmodule.c - atanh_special_values - Modules/cmathmodule.c - cosh_special_values - Modules/cmathmodule.c - exp_special_values - Modules/cmathmodule.c - log_special_values - +Modules/cmathmodule.c - rect_special_values - Modules/cmathmodule.c - sinh_special_values - Modules/cmathmodule.c - sqrt_special_values - Modules/cmathmodule.c - tanh_special_values - -Modules/cmathmodule.c - rect_special_values - Modules/config.c - _PyImport_Inittab - Modules/faulthandler.c - faulthandler_handlers - Modules/getnameinfo.c - gni_afdl - +Modules/nismodule.c - TIMEOUT - +Modules/nismodule.c - aliases - Modules/ossaudiodev.c - control_labels - Modules/ossaudiodev.c - control_names - -Modules/nismodule.c - aliases - -Modules/nismodule.c - TIMEOUT - -Modules/posixmodule.c - posix_constants_pathconf - Modules/posixmodule.c - posix_constants_confstr - +Modules/posixmodule.c - posix_constants_pathconf - Modules/posixmodule.c - posix_constants_sysconf - Modules/pyexpat.c - ExpatMemoryHandler - +Modules/pyexpat.c - error_info_of - Modules/pyexpat.c - handler_info - Modules/termios.c - termios_constants - Modules/timemodule.c init_timezone YEAR - Objects/bytearrayobject.c - _PyByteArray_empty_string - Objects/complexobject.c - c_1 - -Objects/genobject.c - NON_INIT_CORO_MSG - +Objects/exceptions.c - static_exceptions - Objects/genobject.c - ASYNC_GEN_IGNORED_EXIT_MSG - +Objects/genobject.c - NON_INIT_CORO_MSG - Objects/longobject.c - _PyLong_DigitValue - -Objects/object.c - _Py_abstract_hack - Objects/object.c - _Py_SwappedOp - +Objects/object.c - _Py_abstract_hack - +Objects/object.c - static_types - Objects/obmalloc.c - _PyMem - Objects/obmalloc.c - _PyMem_Debug - Objects/obmalloc.c - _PyMem_Raw - Objects/obmalloc.c - _PyObject - Objects/obmalloc.c - usedpools - +Objects/unicodeobject.c - stripfuncnames - +Objects/unicodeobject.c - utf7_category - Objects/unicodeobject.c unicode_decode_call_errorhandler_wchar argparse - Objects/unicodeobject.c unicode_decode_call_errorhandler_writer argparse - Objects/unicodeobject.c unicode_encode_call_errorhandler argparse - Objects/unicodeobject.c unicode_translate_call_errorhandler argparse - -Objects/unicodeobject.c - stripfuncnames - -Objects/unicodeobject.c - utf7_category - Parser/parser.c - reserved_keywords - +Parser/parser.c - soft_keywords - Parser/tokenizer.c - type_comment_prefix - Python/opcode_targets.h - opcode_targets - - - -################################## -# temporary whitelist - globals to fix - -# These are all variables that we will be making non-global. - -#----------------------- -# runtime static types -# [] - -Objects/floatobject.c - FloatInfoType - -Objects/floatobject.c - PyFloat_Type - -Objects/listobject.c - PyList_Type - -Objects/listobject.c - PyListIter_Type - -Objects/listobject.c - PyListRevIter_Type - -Objects/setobject.c - _PySetDummy_Type - -Objects/setobject.c - PySetIter_Type - -Objects/setobject.c - PySet_Type - -Objects/setobject.c - PyFrozenSet_Type - -Objects/genobject.c - PyGen_Type - -Objects/genobject.c - PyCoro_Type - -Objects/genobject.c - _PyCoroWrapper_Type - -Objects/genobject.c - PyAsyncGen_Type - -Objects/genobject.c - _PyAsyncGenASend_Type - -Objects/genobject.c - _PyAsyncGenWrappedValue_Type - -Objects/genobject.c - _PyAsyncGenAThrow_Type - -Objects/classobject.c - PyMethod_Type - -Objects/classobject.c - PyInstanceMethod_Type - -Objects/complexobject.c - PyComplex_Type - -Objects/sliceobject.c - PyEllipsis_Type - -Objects/sliceobject.c - PySlice_Type - -Objects/bytesobject.c - PyBytes_Type - -Objects/bytesobject.c - PyBytesIter_Type - -Objects/descrobject.c - PyMethodDescr_Type - -Objects/descrobject.c - PyClassMethodDescr_Type - -Objects/descrobject.c - PyMemberDescr_Type - -Objects/descrobject.c - PyGetSetDescr_Type - -Objects/descrobject.c - PyWrapperDescr_Type - -Objects/descrobject.c - _PyMethodWrapper_Type - -Objects/descrobject.c - PyDictProxy_Type - -Objects/descrobject.c - PyProperty_Type - -Objects/unicodeobject.c - EncodingMapType - -Objects/unicodeobject.c - PyUnicode_Type - -Objects/unicodeobject.c - PyUnicodeIter_Type - -Objects/unionobject.c - _Py_UnionType - -Objects/moduleobject.c - PyModuleDef_Type - -Objects/moduleobject.c - PyModule_Type - -Objects/capsule.c - PyCapsule_Type - -Objects/methodobject.c - PyCFunction_Type - -Objects/methodobject.c - PyCMethod_Type - -Objects/bytearrayobject.c - PyByteArray_Type - -Objects/bytearrayobject.c - PyByteArrayIter_Type - -Objects/interpreteridobject.c - _PyInterpreterID_Type - -Objects/enumobject.c - PyEnum_Type - -Objects/enumobject.c - PyReversed_Type - -Objects/picklebufobject.c - PyPickleBuffer_Type - -Objects/object.c - _PyNone_Type - -Objects/object.c - _PyNotImplemented_Type - -Objects/fileobject.c - PyStdPrinter_Type - -Objects/weakrefobject.c - _PyWeakref_RefType - -Objects/weakrefobject.c - _PyWeakref_ProxyType - -Objects/weakrefobject.c - _PyWeakref_CallableProxyType - -Objects/genericaliasobject.c - Py_GenericAliasType - -Objects/rangeobject.c - PyRange_Type - -Objects/rangeobject.c - PyRangeIter_Type - -Objects/rangeobject.c - PyLongRangeIter_Type - -Objects/namespaceobject.c - _PyNamespace_Type - -Objects/iterobject.c - PySeqIter_Type - -Objects/iterobject.c - PyCallIter_Type - -Objects/boolobject.c - PyBool_Type - -Objects/frameobject.c - PyFrame_Type - -Objects/longobject.c - Int_InfoType - -Objects/longobject.c - PyLong_Type - -Objects/funcobject.c - PyFunction_Type - -Objects/funcobject.c - PyClassMethod_Type - -Objects/funcobject.c - PyStaticMethod_Type - -Objects/typeobject.c - PyType_Type - -Objects/typeobject.c - PyBaseObject_Type - -Objects/typeobject.c - PySuper_Type - -Objects/cellobject.c - PyCell_Type - -Objects/odictobject.c - PyODict_Type - -Objects/odictobject.c - PyODictIter_Type - -Objects/odictobject.c - PyODictKeys_Type - -Objects/odictobject.c - PyODictItems_Type - -Objects/odictobject.c - PyODictValues_Type - -Objects/dictobject.c - PyDict_Type - -Objects/dictobject.c - PyDictIterKey_Type - -Objects/dictobject.c - PyDictIterValue_Type - -Objects/dictobject.c - PyDictIterItem_Type - -Objects/dictobject.c - PyDictRevIterKey_Type - -Objects/dictobject.c - PyDictRevIterItem_Type - -Objects/dictobject.c - PyDictRevIterValue_Type - -Objects/dictobject.c - PyDictKeys_Type - -Objects/dictobject.c - PyDictItems_Type - -Objects/dictobject.c - PyDictValues_Type - -Objects/memoryobject.c - PyMemoryIter_Type - -Objects/memoryobject.c - _PyManagedBuffer_Type - -Objects/memoryobject.c - PyMemoryView_Type - -Objects/tupleobject.c - PyTuple_Type - -Objects/tupleobject.c - PyTupleIter_Type - -Objects/codeobject.c - PyCode_Type - - -#----------------------- -# builtin exception types -# [] - -Objects/exceptions.c - _PyExc_BaseException - -Objects/exceptions.c - _PyExc_UnicodeEncodeError - -Objects/exceptions.c - _PyExc_UnicodeDecodeError - -Objects/exceptions.c - _PyExc_UnicodeTranslateError - -Objects/exceptions.c - _PyExc_MemoryError - -Objects/exceptions.c - _PyExc_Exception - -Objects/exceptions.c - _PyExc_TypeError - -Objects/exceptions.c - _PyExc_StopAsyncIteration - -Objects/exceptions.c - _PyExc_StopIteration - -Objects/exceptions.c - _PyExc_GeneratorExit - -Objects/exceptions.c - _PyExc_SystemExit - -Objects/exceptions.c - _PyExc_KeyboardInterrupt - -Objects/exceptions.c - _PyExc_ImportError - -Objects/exceptions.c - _PyExc_ModuleNotFoundError - -Objects/exceptions.c - _PyExc_OSError - -Objects/exceptions.c - _PyExc_BlockingIOError - -Objects/exceptions.c - _PyExc_ConnectionError - -Objects/exceptions.c - _PyExc_ChildProcessError - -Objects/exceptions.c - _PyExc_BrokenPipeError - -Objects/exceptions.c - _PyExc_ConnectionAbortedError - -Objects/exceptions.c - _PyExc_ConnectionRefusedError - -Objects/exceptions.c - _PyExc_ConnectionResetError - -Objects/exceptions.c - _PyExc_FileExistsError - -Objects/exceptions.c - _PyExc_FileNotFoundError - -Objects/exceptions.c - _PyExc_IsADirectoryError - -Objects/exceptions.c - _PyExc_NotADirectoryError - -Objects/exceptions.c - _PyExc_InterruptedError - -Objects/exceptions.c - _PyExc_PermissionError - -Objects/exceptions.c - _PyExc_ProcessLookupError - -Objects/exceptions.c - _PyExc_TimeoutError - -Objects/exceptions.c - _PyExc_EOFError - -Objects/exceptions.c - _PyExc_RuntimeError - -Objects/exceptions.c - _PyExc_RecursionError - -Objects/exceptions.c - _PyExc_NotImplementedError - -Objects/exceptions.c - _PyExc_NameError - -Objects/exceptions.c - _PyExc_UnboundLocalError - -Objects/exceptions.c - _PyExc_AttributeError - -Objects/exceptions.c - _PyExc_SyntaxError - -Objects/exceptions.c - _PyExc_IndentationError - -Objects/exceptions.c - _PyExc_TabError - -Objects/exceptions.c - _PyExc_LookupError - -Objects/exceptions.c - _PyExc_IndexError - -Objects/exceptions.c - _PyExc_KeyError - -Objects/exceptions.c - _PyExc_ValueError - -Objects/exceptions.c - _PyExc_UnicodeError - -Objects/exceptions.c - _PyExc_AssertionError - -Objects/exceptions.c - _PyExc_ArithmeticError - -Objects/exceptions.c - _PyExc_FloatingPointError - -Objects/exceptions.c - _PyExc_OverflowError - -Objects/exceptions.c - _PyExc_ZeroDivisionError - -Objects/exceptions.c - _PyExc_SystemError - -Objects/exceptions.c - _PyExc_ReferenceError - -Objects/exceptions.c - _PyExc_BufferError - -Objects/exceptions.c - _PyExc_Warning - -Objects/exceptions.c - _PyExc_UserWarning - -Objects/exceptions.c - _PyExc_DeprecationWarning - -Objects/exceptions.c - _PyExc_PendingDeprecationWarning - -Objects/exceptions.c - _PyExc_SyntaxWarning - -Objects/exceptions.c - _PyExc_RuntimeWarning - -Objects/exceptions.c - _PyExc_FutureWarning - -Objects/exceptions.c - _PyExc_ImportWarning - -Objects/exceptions.c - _PyExc_UnicodeWarning - -Objects/exceptions.c - _PyExc_BytesWarning - -Objects/exceptions.c - _PyExc_ResourceWarning - -Objects/exceptions.c - PyExc_EnvironmentError - -Objects/exceptions.c - PyExc_IOError - -Objects/exceptions.c - PyExc_BaseException - -Objects/exceptions.c - PyExc_Exception - -Objects/exceptions.c - PyExc_TypeError - -Objects/exceptions.c - PyExc_StopAsyncIteration - -Objects/exceptions.c - PyExc_StopIteration - -Objects/exceptions.c - PyExc_GeneratorExit - -Objects/exceptions.c - PyExc_SystemExit - -Objects/exceptions.c - PyExc_KeyboardInterrupt - -Objects/exceptions.c - PyExc_ImportError - -Objects/exceptions.c - PyExc_ModuleNotFoundError - -Objects/exceptions.c - PyExc_OSError - -Objects/exceptions.c - PyExc_BlockingIOError - -Objects/exceptions.c - PyExc_ConnectionError - -Objects/exceptions.c - PyExc_ChildProcessError - -Objects/exceptions.c - PyExc_BrokenPipeError - -Objects/exceptions.c - PyExc_ConnectionAbortedError - -Objects/exceptions.c - PyExc_ConnectionRefusedError - -Objects/exceptions.c - PyExc_ConnectionResetError - -Objects/exceptions.c - PyExc_FileExistsError - -Objects/exceptions.c - PyExc_FileNotFoundError - -Objects/exceptions.c - PyExc_IsADirectoryError - -Objects/exceptions.c - PyExc_NotADirectoryError - -Objects/exceptions.c - PyExc_InterruptedError - -Objects/exceptions.c - PyExc_PermissionError - -Objects/exceptions.c - PyExc_ProcessLookupError - -Objects/exceptions.c - PyExc_TimeoutError - -Objects/exceptions.c - PyExc_EOFError - -Objects/exceptions.c - PyExc_RuntimeError - -Objects/exceptions.c - PyExc_RecursionError - -Objects/exceptions.c - PyExc_NotImplementedError - -Objects/exceptions.c - PyExc_NameError - -Objects/exceptions.c - PyExc_UnboundLocalError - -Objects/exceptions.c - PyExc_AttributeError - -Objects/exceptions.c - PyExc_SyntaxError - -Objects/exceptions.c - PyExc_IndentationError - -Objects/exceptions.c - PyExc_TabError - -Objects/exceptions.c - PyExc_LookupError - -Objects/exceptions.c - PyExc_IndexError - -Objects/exceptions.c - PyExc_KeyError - -Objects/exceptions.c - PyExc_ValueError - -Objects/exceptions.c - PyExc_UnicodeError - -Objects/exceptions.c - PyExc_UnicodeEncodeError - -Objects/exceptions.c - PyExc_UnicodeDecodeError - -Objects/exceptions.c - PyExc_UnicodeTranslateError - -Objects/exceptions.c - PyExc_AssertionError - -Objects/exceptions.c - PyExc_ArithmeticError - -Objects/exceptions.c - PyExc_FloatingPointError - -Objects/exceptions.c - PyExc_OverflowError - -Objects/exceptions.c - PyExc_ZeroDivisionError - -Objects/exceptions.c - PyExc_SystemError - -Objects/exceptions.c - PyExc_ReferenceError - -Objects/exceptions.c - PyExc_MemoryError - -Objects/exceptions.c - PyExc_BufferError - -Objects/exceptions.c - PyExc_Warning - -Objects/exceptions.c - PyExc_UserWarning - -Objects/exceptions.c - PyExc_DeprecationWarning - -Objects/exceptions.c - PyExc_PendingDeprecationWarning - -Objects/exceptions.c - PyExc_SyntaxWarning - -Objects/exceptions.c - PyExc_RuntimeWarning - -Objects/exceptions.c - PyExc_FutureWarning - -Objects/exceptions.c - PyExc_ImportWarning - -Objects/exceptions.c - PyExc_UnicodeWarning - -Objects/exceptions.c - PyExc_BytesWarning - -Objects/exceptions.c - PyExc_ResourceWarning - - -#----------------------- -# singletons -# [] - -Objects/boolobject.c - _Py_FalseStruct - -Objects/boolobject.c - _Py_TrueStruct - -Objects/dictobject.c - empty_keys_struct - -Objects/dictobject.c - empty_values - -Objects/object.c - _Py_NoneStruct - -Objects/object.c - _Py_NotImplementedStruct - -Objects/setobject.c - _dummy_struct - -Objects/setobject.c - _PySet_Dummy - -Objects/sliceobject.c - _Py_EllipsisObject - - -#----------------------- -# runtime initialized once - cached PyUnicode -# [] - -# Py_IDENTIFIER (global) [] -Objects/classobject.c - PyId___name__ - -Objects/classobject.c - PyId___qualname__ - -Objects/structseq.c - PyId_n_sequence_fields - -Objects/structseq.c - PyId_n_fields - -Objects/structseq.c - PyId_n_unnamed_fields - -Objects/bytesobject.c - PyId___bytes__ - -Objects/descrobject.c - PyId_getattr - -Objects/moduleobject.c - PyId___doc__ - -Objects/moduleobject.c - PyId___name__ - -Objects/moduleobject.c - PyId___spec__ - -Objects/object.c - PyId_Py_Repr - -Objects/object.c - PyId___bytes__ - -Objects/object.c - PyId___dir__ - -Objects/object.c - PyId___isabstractmethod__ - -Objects/fileobject.c - PyId_open - -Objects/rangeobject.c - PyId_iter - -Objects/iterobject.c - PyId_iter - -Objects/frameobject.c - PyId___builtins__ - -Objects/longobject.c - PyId_little - -Objects/longobject.c - PyId_big - -Objects/typeobject.c - PyId___abstractmethods__ - -Objects/typeobject.c - PyId___class__ - -Objects/typeobject.c - PyId___class_getitem__ - -Objects/typeobject.c - PyId___delitem__ - -Objects/typeobject.c - PyId___dict__ - -Objects/typeobject.c - PyId___doc__ - -Objects/typeobject.c - PyId___getattribute__ - -Objects/typeobject.c - PyId___getitem__ - -Objects/typeobject.c - PyId___hash__ - -Objects/typeobject.c - PyId___init_subclass__ - -Objects/typeobject.c - PyId___len__ - -Objects/typeobject.c - PyId___module__ - -Objects/typeobject.c - PyId___name__ - -Objects/typeobject.c - PyId___new__ - -Objects/typeobject.c - PyId___set_name__ - -Objects/typeobject.c - PyId___setitem__ - -Objects/typeobject.c - PyId_builtins - -Objects/typeobject.c - PyId_mro - -Objects/odictobject.c - PyId_items - - -# Py_IDENTIFIER (local) [] -Objects/listobject.c listiter_reduce_general PyId_iter - -Objects/listobject.c listiter_reduce_general PyId_reversed - -Objects/setobject.c setiter_reduce PyId_iter - -Objects/setobject.c set_reduce PyId___dict__ - -Objects/abstract.c PyObject_LengthHint PyId___length_hint__ - -Objects/abstract.c PyObject_GetItem PyId___class_getitem__ - -Objects/abstract.c PyObject_Format PyId___format__ - -Objects/abstract.c PyNumber_Long PyId___trunc__ - -Objects/abstract.c PyMapping_Keys PyId_keys - -Objects/abstract.c PyMapping_Items PyId_items - -Objects/abstract.c PyMapping_Values PyId_values - -Objects/abstract.c abstract_get_bases PyId___bases__ - -Objects/abstract.c object_isinstance PyId___class__ - -Objects/abstract.c object_recursive_isinstance PyId___instancecheck__ - -Objects/abstract.c object_issubclass PyId___subclasscheck__ - -Objects/genobject.c PyIter_Send PyId_send - -Objects/genobject.c gen_close_iter PyId_close - -Objects/genobject.c _gen_throw PyId_throw - -Objects/classobject.c method_reduce PyId_getattr - -Objects/complexobject.c try_complex_special_method PyId___complex__ - -Objects/bytesobject.c striter_reduce PyId_iter - -Objects/descrobject.c calculate_qualname PyId___qualname__ - -Objects/descrobject.c mappingproxy_get PyId_get - -Objects/descrobject.c mappingproxy_keys PyId_keys - -Objects/descrobject.c mappingproxy_values PyId_values - -Objects/descrobject.c mappingproxy_items PyId_items - -Objects/descrobject.c mappingproxy_copy PyId_copy - -Objects/descrobject.c mappingproxy_reversed PyId___reversed__ - -Objects/descrobject.c property_init_impl PyId___doc__ - -Objects/unicodeobject.c unicodeiter_reduce PyId_iter - -Objects/unionobject.c union_repr_item PyId___module__ - -Objects/unionobject.c union_repr_item PyId___qualname__ - -Objects/unionobject.c union_repr_item PyId___origin__ - -Objects/unionobject.c union_repr_item PyId___args__ - -Objects/moduleobject.c module_init_dict PyId___package__ - -Objects/moduleobject.c module_init_dict PyId___loader__ - -Objects/moduleobject.c PyModule_GetFilenameObject PyId___file__ - -Objects/moduleobject.c _PyModuleSpec_IsInitializing PyId__initializing - -Objects/moduleobject.c module_getattro PyId___getattr__ - -Objects/moduleobject.c module_dir PyId___dict__ - -Objects/moduleobject.c module_dir PyId___dir__ - -Objects/methodobject.c meth_reduce PyId_getattr - -Objects/methodobject.c meth_get__qualname__ PyId___qualname__ - -Objects/bytearrayobject.c _common_reduce PyId___dict__ - -Objects/bytearrayobject.c bytearrayiter_reduce PyId_iter - -Objects/enumobject.c reversed_new_impl PyId___reversed__ - -Objects/object.c _PyObject_FunctionStr PyId___module__ - -Objects/object.c _PyObject_FunctionStr PyId___qualname__ - -Objects/object.c _PyObject_FunctionStr PyId_builtins - -Objects/fileobject.c PyFile_GetLine PyId_readline - -Objects/fileobject.c PyFile_WriteObject PyId_write - -Objects/fileobject.c PyObject_AsFileDescriptor PyId_fileno - -Objects/weakrefobject.c weakref_repr PyId___name__ - -Objects/weakrefobject.c proxy_bytes PyId___bytes__ - -Objects/weakrefobject.c proxy_reversed PyId___reversed__ - -Objects/genericaliasobject.c ga_repr_item PyId___module__ - -Objects/genericaliasobject.c ga_repr_item PyId___qualname__ - -Objects/genericaliasobject.c ga_repr_item PyId___origin__ - -Objects/genericaliasobject.c ga_repr_item PyId___args__ - -Objects/genericaliasobject.c make_parameters PyId___parameters__ - -Objects/genericaliasobject.c subs_tvars PyId___parameters__ - -Objects/exceptions.c ImportError_getstate PyId_name - -Objects/exceptions.c ImportError_getstate PyId_path - -Objects/typeobject.c type_new PyId___qualname__ - -Objects/typeobject.c type_new PyId___slots__ - -Objects/typeobject.c type_new PyId___classcell__ - -Objects/typeobject.c type_new PyId___mro_entries__ - -Objects/typeobject.c merge_class_dict PyId___bases__ - -Objects/typeobject.c import_copyreg PyId_copyreg - -Objects/typeobject.c _PyType_GetSlotNames PyId___slotnames__ - -Objects/typeobject.c _PyType_GetSlotNames PyId__slotnames - -Objects/typeobject.c _PyObject_GetState PyId___getstate__ - -Objects/typeobject.c _PyObject_GetNewArguments PyId___getnewargs_ex__ - -Objects/typeobject.c _PyObject_GetNewArguments PyId___getnewargs__ - -Objects/typeobject.c _PyObject_GetItemsIter PyId_items - -Objects/typeobject.c reduce_newobj PyId___newobj__ - -Objects/typeobject.c reduce_newobj PyId___newobj_ex__ - -Objects/typeobject.c object___reduce_ex___impl PyId___reduce__ - -Objects/typeobject.c overrides_hash PyId___eq__ - -Objects/typeobject.c slot_sq_contains PyId___contains__ - -Objects/typeobject.c slot_nb_power PyId___pow__ - -Objects/typeobject.c slot_nb_bool PyId___bool__ - -Objects/typeobject.c slot_nb_index PyId___index__ - -Objects/typeobject.c slot_nb_inplace_power PyId___ipow__ - -Objects/typeobject.c slot_tp_repr PyId___repr__ - -Objects/typeobject.c slot_tp_call PyId___call__ - -Objects/typeobject.c slot_tp_getattr_hook PyId___getattr__ - -Objects/typeobject.c slot_tp_setattro PyId___delattr__ - -Objects/typeobject.c slot_tp_setattro PyId___setattr__ - -Objects/typeobject.c slot_tp_iter PyId___iter__ - -Objects/typeobject.c slot_tp_iternext PyId___next__ - -Objects/typeobject.c slot_tp_descr_get PyId___get__ - -Objects/typeobject.c slot_tp_descr_set PyId___delete__ - -Objects/typeobject.c slot_tp_descr_set PyId___set__ - -Objects/typeobject.c slot_tp_init PyId___init__ - -Objects/typeobject.c slot_tp_finalize PyId___del__ - -Objects/typeobject.c slot_am_await PyId___await__ - -Objects/typeobject.c slot_am_aiter PyId___aiter__ - -Objects/typeobject.c slot_am_anext PyId___anext__ - -Objects/odictobject.c odict_reduce PyId___dict__ - -Objects/odictobject.c odictiter_reduce PyId_iter - -Objects/odictobject.c mutablemapping_update_arg PyId_keys - -Objects/dictobject.c dict_subscript PyId___missing__ - -Objects/dictobject.c dict_update_arg PyId_keys - -Objects/dictobject.c dictiter_reduce PyId_iter - -Objects/dictobject.c dictviews_sub PyId_difference_update - -Objects/dictobject.c _PyDictView_Intersect PyId_intersection - -Objects/dictobject.c dictitems_xor PyId_items - -Objects/dictobject.c dictviews_xor PyId_symmetric_difference_update - -Objects/tupleobject.c tupleiter_reduce PyId_iter - -Parser/tokenizer.c fp_setreadl PyId_open - -Parser/tokenizer.c fp_setreadl PyId_readline - - -# _Py_static_string [] -Objects/typeobject.c - name_op - -Objects/typeobject.c object_new comma_id - -Objects/typeobject.c slot_mp_subscript id - -Objects/typeobject.c slot_nb_add op_id - -Objects/typeobject.c slot_nb_add rop_id - -Objects/typeobject.c slot_nb_subtract op_id - -Objects/typeobject.c slot_nb_subtract rop_id - -Objects/typeobject.c slot_nb_multiply op_id - -Objects/typeobject.c slot_nb_multiply rop_id - -Objects/typeobject.c slot_nb_matrix_multiply op_id - -Objects/typeobject.c slot_nb_matrix_multiply rop_id - -Objects/typeobject.c slot_nb_remainder op_id - -Objects/typeobject.c slot_nb_remainder rop_id - -Objects/typeobject.c slot_nb_divmod op_id - -Objects/typeobject.c slot_nb_divmod rop_id - -Objects/typeobject.c slot_nb_power_binary op_id - -Objects/typeobject.c slot_nb_power_binary rop_id - -Objects/typeobject.c slot_nb_negative id - -Objects/typeobject.c slot_nb_positive id - -Objects/typeobject.c slot_nb_absolute id - -Objects/typeobject.c slot_nb_invert id - -Objects/typeobject.c slot_nb_lshift op_id - -Objects/typeobject.c slot_nb_lshift rop_id - -Objects/typeobject.c slot_nb_rshift op_id - -Objects/typeobject.c slot_nb_rshift rop_id - -Objects/typeobject.c slot_nb_and op_id - -Objects/typeobject.c slot_nb_and rop_id - -Objects/typeobject.c slot_nb_xor op_id - -Objects/typeobject.c slot_nb_xor rop_id - -Objects/typeobject.c slot_nb_or op_id - -Objects/typeobject.c slot_nb_or rop_id - -Objects/typeobject.c slot_nb_int id - -Objects/typeobject.c slot_nb_float id - -Objects/typeobject.c slot_nb_inplace_add id - -Objects/typeobject.c slot_nb_inplace_subtract id - -Objects/typeobject.c slot_nb_inplace_multiply id - -Objects/typeobject.c slot_nb_inplace_matrix_multiply id - -Objects/typeobject.c slot_nb_inplace_remainder id - -Objects/typeobject.c slot_nb_inplace_lshift id - -Objects/typeobject.c slot_nb_inplace_rshift id - -Objects/typeobject.c slot_nb_inplace_and id - -Objects/typeobject.c slot_nb_inplace_xor id - -Objects/typeobject.c slot_nb_inplace_or id - -Objects/typeobject.c slot_nb_floor_divide op_id - -Objects/typeobject.c slot_nb_floor_divide rop_id - -Objects/typeobject.c slot_nb_true_divide op_id - -Objects/typeobject.c slot_nb_true_divide rop_id - -Objects/typeobject.c slot_nb_inplace_floor_divide id - -Objects/typeobject.c slot_nb_inplace_true_divide id - -Objects/typeobject.c slot_tp_str id - -Python/compile.c compiler_set_qualname dot - -Python/compile.c compiler_set_qualname dot_locals - - -# manually cached PyUnicodeOjbect [] -Objects/boolobject.c - false_str - -Objects/boolobject.c - true_str - -Objects/classobject.c method_get_doc docstr - -Objects/classobject.c instancemethod_get_doc docstr - -Objects/codeobject.c PyCode_NewEmpty emptystring - -Objects/exceptions.c _check_for_legacy_statements print_prefix - -Objects/exceptions.c _check_for_legacy_statements exec_prefix - -Objects/funcobject.c PyFunction_NewWithQualName __name__ - -Objects/listobject.c - indexerr - -Objects/typeobject.c object___reduce_ex___impl objreduce - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_close_br - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_dbl_close_br - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_dbl_open_br - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_inf - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_open_br - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_replace_inf - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c - __annotations__ - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c - __doc__ - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_dictcomp name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_from_import empty_string - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_genexp name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_lambda name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_listcomp name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_setcomp name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_visit_annotations return_str - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c PyImport_Import builtins_str - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c PyImport_Import import_str - -# XXX This should have been found by the analyzer but wasn't: -Python/sysmodule.c - whatstrings - -# XXX This should have been found by the analyzer but wasn't: -Python/sysmodule.c sys_displayhook newline - -# XXX This should have been found by the analyzer but wasn't: -Python/_warnings.c is_internal_frame bootstrap_string - -# XXX This should have been found by the analyzer but wasn't: -Python/_warnings.c is_internal_frame importlib_string - - -#----------------------- -# runtime initialized once - other PyObject -# [] - -# cache [] -Objects/unicodeobject.c - interned - -Objects/unicodeobject.c - static_strings - -Objects/typeobject.c - method_cache - - -# other [] -# XXX This should have been found by the analyzer but wasn't: -Python/context.c - _token_missing - -# XXX This should have been found by the analyzer but wasn't: -Python/hamt.c - _empty_bitmap_node - -# XXX This should have been found by the analyzer but wasn't: -Python/hamt.c - _empty_hamt - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c PyImport_Import silly_list - - -#----------------------- -# runtime initialized once - non-PyObject -# [] - -# during init [] -Parser/parser.c - Py_DebugFlag - - -# other [] -Objects/codeobject.c PyCode_NewEmpty nulltuple - -Objects/longobject.c PyLong_FromString log_base_BASE - -Objects/longobject.c PyLong_FromString convwidth_base - -Objects/longobject.c PyLong_FromString convmultmax_base - -Objects/typeobject.c - slotdefs - -Objects/typeobject.c - slotdefs_initialized - -Objects/unicodeobject.c - bloom_linebreak - -Objects/unicodeobject.c - ucnhash_capi - -Parser/pegen.c _PyPegen_dummy_name cache - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c - import_lock - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c import_find_and_load header - - -#----------------------- -# runtime state - -# (look at the bottom of the file) - -#----------------------- -# modules -# [119] - -Modules/pwdmodule.c - pwdmodule - -Modules/grpmodule.c - grpmodule - -Modules/_ssl.c - PySocketModule - -Modules/_ssl.c - _sslmodule - -Modules/_struct.c - _structmodule - -Modules/_sre.c - sremodule - -Modules/timemodule.c - timemodule - -Modules/xxmodule.c - xxmodule - -Modules/itertoolsmodule.c - itertoolsmodule - -Modules/_tkinter.c - _tkintermodule - -Modules/gcmodule.c - gcmodule - -Modules/mmapmodule.c - mmapmodule - -Modules/errnomodule.c - errnomodule - -Modules/_gdbmmodule.c - _gdbmmodule - -Modules/xxlimited.c - xxmodule - -Modules/arraymodule.c - arraymodule - -Modules/_uuidmodule.c - uuidmodule - -Modules/_collectionsmodule.c - _collectionsmodule - -Modules/_csv.c - _csvmodule - -Modules/_json.c - jsonmodule - -Modules/zlibmodule.c - zlibmodule - -Modules/readline.c - readlinemodule - -Modules/faulthandler.c - module_def - -Modules/_codecsmodule.c - codecsmodule - -Modules/_asynciomodule.c - _asynciomodule - -Modules/signalmodule.c - signalmodule - -Modules/binascii.c - binasciimodule - -Modules/mathmodule.c - mathmodule - -Modules/_stat.c - statmodule - -Modules/_opcode.c - opcodemodule - -Modules/_operator.c - operatormodule - -Modules/_cryptmodule.c - cryptmodule - -Modules/cmathmodule.c - cmathmodule - -Modules/_lzmamodule.c - _lzmamodule - -Modules/_zoneinfo.c - zoneinfomodule - -Modules/posixmodule.c - posixmodule - -Modules/_bz2module.c - _bz2module - -Modules/_functoolsmodule.c - _functools_module - -Modules/_abc.c - _abcmodule - -Modules/_heapqmodule.c - _heapqmodule - -Modules/_bisectmodule.c - _bisectmodule - -Modules/_tracemalloc.c - module_def - -Modules/pyexpat.c - pyexpatmodule - -Modules/_randommodule.c - _randommodule - -Modules/atexitmodule.c - atexitmodule - -Modules/syslogmodule.c - syslogmodule - -Modules/_queuemodule.c - queuemodule - -Modules/_threadmodule.c - threadmodule - -Modules/_weakref.c - weakrefmodule - -Modules/spwdmodule.c - spwdmodule - -Modules/_contextvarsmodule.c - _contextvarsmodule - -Modules/_posixsubprocess.c - _posixsubprocessmodule - -Modules/_xxsubinterpretersmodule.c - interpretersmodule - -Modules/_curses_panel.c - _curses_panelmodule - -Modules/audioop.c - audioopmodule - -Modules/nismodule.c - nismodule - -Modules/_elementtree.c - elementtreemodule - -Modules/sha256module.c - _sha256module - -Modules/resource.c - resourcemodule - -Modules/symtablemodule.c - symtablemodule - -Modules/sha1module.c - _sha1module - -Modules/selectmodule.c - selectmodule - -Modules/_pickle.c - _picklemodule - -Modules/_localemodule.c - _localemodule - -Modules/unicodedata.c - unicodedata_module - -Modules/_statisticsmodule.c - statisticsmodule - -Modules/termios.c - termiosmodule - -Modules/xxsubtype.c - xxsubtypemodule - -Modules/sha512module.c - _sha512module - -Modules/_cursesmodule.c - _cursesmodule - -Modules/md5module.c - _md5module - -Modules/socketmodule.c - socketmodule - -Modules/_datetimemodule.c - datetimemodule - -Modules/_hashopenssl.c - _hashlibmodule - -Modules/fcntlmodule.c - fcntlmodule - -Modules/ossaudiodev.c - ossaudiodevmodule - -Modules/_lsprof.c - _lsprofmodule - -Modules/_blake2/blake2module.c - blake2_module - -Modules/_multiprocessing/multiprocessing.c - multiprocessing_module - -Modules/_multiprocessing/posixshmem.c - this_module - -Modules/_sqlite/module.c - _sqlite3module - -Modules/_sha3/sha3module.c - _sha3module - -Modules/cjkcodecs/multibytecodec.c - _multibytecodecmodule - -Modules/_decimal/_decimal.c - _decimal_module - -Modules/_ctypes/_ctypes.c - _ctypesmodule - -Objects/unicodeobject.c - _string_module - -Modules/_io/_iomodule.h - _PyIO_Module - -Modules/_io/_iomodule.c - _PyIO_Module - - -#----------------------- -# module static types -# [] - -Modules/arraymodule.c - Arraytype - -Modules/arraymodule.c - PyArrayIter_Type - -Modules/_asynciomodule.c - FutureType - -Modules/_asynciomodule.c - FutureIterType - -Modules/_asynciomodule.c - TaskStepMethWrapper_Type - -Modules/_asynciomodule.c - TaskType - -Modules/_asynciomodule.c - PyRunningLoopHolder_Type - -Modules/cjkcodecs/multibytecodec.c - MultibyteCodec_Type - -Modules/cjkcodecs/multibytecodec.c - MultibyteIncrementalEncoder_Type - -Modules/cjkcodecs/multibytecodec.c - MultibyteIncrementalDecoder_Type - -Modules/cjkcodecs/multibytecodec.c - MultibyteStreamReader_Type - -Modules/cjkcodecs/multibytecodec.c - MultibyteStreamWriter_Type - -Modules/_collectionsmodule.c - deque_type - -Modules/_collectionsmodule.c - dequeiter_type - -Modules/_collectionsmodule.c - dequereviter_type - -Modules/_collectionsmodule.c - defdict_type - -Modules/_collectionsmodule.c - tuplegetter_type - -Modules/_csv.c - Dialect_Type - -Modules/_csv.c - Reader_Type - -Modules/_csv.c - Writer_Type - -Modules/_ctypes/callbacks.c - PyCThunk_Type - -Modules/_ctypes/callproc.c - PyCArg_Type - -Modules/_ctypes/cfield.c - PyCField_Type - -Modules/_ctypes/_ctypes.c - DictRemover_Type - -Modules/_ctypes/_ctypes.c - StructParam_Type - -Modules/_ctypes/_ctypes.c - PyCStructType_Type - -Modules/_ctypes/_ctypes.c - UnionType_Type - -Modules/_ctypes/_ctypes.c - PyCPointerType_Type - -Modules/_ctypes/_ctypes.c - PyCArrayType_Type - -Modules/_ctypes/_ctypes.c - PyCSimpleType_Type - -Modules/_ctypes/_ctypes.c - PyCFuncPtrType_Type - -Modules/_ctypes/_ctypes.c - PyCData_Type - -Modules/_ctypes/_ctypes.c - PyCFuncPtr_Type - -Modules/_ctypes/_ctypes.c - Struct_Type - -Modules/_ctypes/_ctypes.c - Union_Type - -Modules/_ctypes/_ctypes.c - PyCArray_Type - -Modules/_ctypes/_ctypes.c - Simple_Type - -Modules/_ctypes/_ctypes.c - PyCPointer_Type - -Modules/_ctypes/_ctypes.c - PyComError_Type - -Modules/_ctypes/stgdict.c - PyCStgDict_Type - -Modules/_cursesmodule.c - PyCursesWindow_Type - -Modules/_datetimemodule.c - PyDateTime_DeltaType - -Modules/_datetimemodule.c - PyDateTime_IsoCalendarDateType - -Modules/_datetimemodule.c - PyDateTime_DateType - -Modules/_datetimemodule.c - PyDateTime_TZInfoType - -Modules/_datetimemodule.c - PyDateTime_TimeZoneType - -Modules/_datetimemodule.c - PyDateTime_TimeType - -Modules/_datetimemodule.c - PyDateTime_DateTimeType - -Modules/_decimal/_decimal.c - PyDecSignalDictMixin_Type - -Modules/_decimal/_decimal.c - PyDecContextManager_Type - -Modules/_decimal/_decimal.c - PyDec_Type - -Modules/_decimal/_decimal.c - PyDecContext_Type - -Modules/_elementtree.c - ElementIter_Type - -Modules/_elementtree.c - Element_Type - -Modules/_elementtree.c - TreeBuilder_Type - -Modules/_elementtree.c - XMLParser_Type - -Modules/_functoolsmodule.c - partial_type - -Modules/_functoolsmodule.c - keyobject_type - -Modules/_functoolsmodule.c - lru_list_elem_type - -Modules/_functoolsmodule.c - lru_cache_type - -Modules/_io/bufferedio.c - PyBufferedIOBase_Type - -Modules/_io/bufferedio.c - PyBufferedReader_Type - -Modules/_io/bufferedio.c - PyBufferedWriter_Type - -Modules/_io/bufferedio.c - PyBufferedRWPair_Type - -Modules/_io/bufferedio.c - PyBufferedRandom_Type - -Modules/_io/bytesio.c - PyBytesIO_Type - -Modules/_io/bytesio.c - _PyBytesIOBuffer_Type - -Modules/_io/fileio.c - PyFileIO_Type - -Modules/_io/iobase.c - PyIOBase_Type - -Modules/_io/iobase.c - PyRawIOBase_Type - -Modules/_io/stringio.c - PyStringIO_Type - -Modules/_io/textio.c - PyTextIOBase_Type - -Modules/_io/textio.c - PyIncrementalNewlineDecoder_Type - -Modules/_io/textio.c - PyTextIOWrapper_Type - -Modules/_io/winconsoleio.c - PyWindowsConsoleIO_Type - -Modules/itertoolsmodule.c - groupby_type - -Modules/itertoolsmodule.c - _grouper_type - -Modules/itertoolsmodule.c - teedataobject_type - -Modules/itertoolsmodule.c - tee_type - -Modules/itertoolsmodule.c - cycle_type - -Modules/itertoolsmodule.c - dropwhile_type - -Modules/itertoolsmodule.c - takewhile_type - -Modules/itertoolsmodule.c - islice_type - -Modules/itertoolsmodule.c - starmap_type - -Modules/itertoolsmodule.c - chain_type - -Modules/itertoolsmodule.c - product_type - -Modules/itertoolsmodule.c - combinations_type - -Modules/itertoolsmodule.c - cwr_type - -Modules/itertoolsmodule.c - permutations_type - -Modules/itertoolsmodule.c - accumulate_type - -Modules/itertoolsmodule.c - compress_type - -Modules/itertoolsmodule.c - filterfalse_type - -Modules/itertoolsmodule.c - count_type - -Modules/itertoolsmodule.c - repeat_type - -Modules/itertoolsmodule.c - ziplongest_type - -Modules/mmapmodule.c - mmap_object_type - -Modules/_multiprocessing/semaphore.c - _PyMp_SemLockType - -Modules/ossaudiodev.c - OSSAudioType - -Modules/ossaudiodev.c - OSSMixerType - -Modules/_pickle.c - Pdata_Type - -Modules/_pickle.c - PicklerMemoProxyType - -Modules/_pickle.c - Pickler_Type - -Modules/_pickle.c - UnpicklerMemoProxyType - -Modules/_pickle.c - Unpickler_Type - -Modules/pyexpat.c - Xmlparsetype - -Modules/_queuemodule.c - PySimpleQueueType - -Modules/socketmodule.c - sock_type - -Modules/_sre.c - Pattern_Type - -Modules/_sre.c - Match_Type - -Modules/_sre.c - Scanner_Type - -Modules/_ssl.c - PySSLSocket_Type - -Modules/_ssl.c - PySSLContext_Type - -Modules/_ssl.c - PySSLMemoryBIO_Type - -Modules/_ssl.c - PySSLSession_Type - -Modules/_threadmodule.c - Locktype - -Modules/_threadmodule.c - RLocktype - -Modules/_threadmodule.c - localdummytype - -Modules/_threadmodule.c - localtype - -Modules/xxmodule.c - Xxo_Type - -Modules/xxmodule.c - Str_Type - -Modules/xxmodule.c - Null_Type - -Modules/_xxsubinterpretersmodule.c - ChannelIDtype - -Modules/xxsubtype.c - spamlist_type - -Modules/xxsubtype.c - spamdict_type - -Modules/_zoneinfo.c - PyZoneInfo_ZoneInfoType - - -#----------------------- -# module initialized once - non-static types -# [] - -# structseq types [6] -Modules/timemodule.c - StructTimeType - -Modules/signalmodule.c - SiginfoType - -Modules/_threadmodule.c - ExceptHookArgsType - -Modules/spwdmodule.c - StructSpwdType - -Modules/resource.c - StructRUsageType - -Modules/_cursesmodule.c - NcursesVersionType - - -# heap types [12] -Modules/_tkinter.c - Tkapp_Type - -Modules/_tkinter.c - PyTclObject_Type - -Modules/_tkinter.c - Tktt_Type - -Modules/xxlimited.c - Xxo_Type - -Modules/_decimal/_decimal.c - DecimalTuple - -Modules/_decimal/_decimal.c - PyDecSignalDict_Type - - -# exception types [] -Modules/_ctypes/_ctypes.c - PyExc_ArgError - -Modules/_cursesmodule.c - PyCursesError - -Modules/_decimal/_decimal.c - DecimalException - -Modules/_queuemodule.c - EmptyError - -Modules/_ssl.c - PySSLErrorObject - -Modules/_ssl.c - PySSLCertVerificationErrorObject - -Modules/_ssl.c - PySSLZeroReturnErrorObject - -Modules/_ssl.c - PySSLWantReadErrorObject - -Modules/_ssl.c - PySSLWantWriteErrorObject - -Modules/_ssl.c - PySSLSyscallErrorObject - -Modules/_ssl.c - PySSLEOFErrorObject - -Modules/_threadmodule.c - ThreadError - -Modules/_tkinter.c - Tkinter_TclError - -Modules/_xxsubinterpretersmodule.c - ChannelError - -Modules/_xxsubinterpretersmodule.c - ChannelNotFoundError - -Modules/_xxsubinterpretersmodule.c - ChannelClosedError - -Modules/_xxsubinterpretersmodule.c - ChannelEmptyError - -Modules/_xxsubinterpretersmodule.c - ChannelNotEmptyError - -Modules/_xxsubinterpretersmodule.c - RunFailedError - -Modules/ossaudiodev.c - OSSAudioError - -Modules/pyexpat.c - ErrorObject - -Modules/signalmodule.c - ItimerError - -Modules/socketmodule.c - socket_herror - -Modules/socketmodule.c - socket_gaierror - -Modules/socketmodule.c - socket_timeout - -Modules/xxlimited.c - ErrorObject - -Modules/xxmodule.c - ErrorObject - - -#----------------------- -# module initialized once - cached PyUnicode -# [] - -# Py_IDENTIFIER (global) [] -Modules/faulthandler.c - PyId_enable - -Modules/faulthandler.c - PyId_fileno - -Modules/faulthandler.c - PyId_flush - -Modules/faulthandler.c - PyId_stderr - -Modules/_asynciomodule.c - PyId___asyncio_running_event_loop__ - -Modules/_asynciomodule.c - PyId__asyncio_future_blocking - -Modules/_asynciomodule.c - PyId_add_done_callback - -Modules/_asynciomodule.c - PyId_call_soon - -Modules/_asynciomodule.c - PyId_cancel - -Modules/_asynciomodule.c - PyId_get_event_loop - -Modules/_asynciomodule.c - PyId_throw - -Modules/posixmodule.c - PyId___fspath__ - -Modules/_abc.c - PyId___abstractmethods__ - -Modules/_abc.c - PyId___class__ - -Modules/_abc.c - PyId___dict__ - -Modules/_abc.c - PyId___bases__ - -Modules/_abc.c - PyId__abc_impl - -Modules/_abc.c - PyId___subclasscheck__ - -Modules/_abc.c - PyId___subclasshook__ - -Modules/_bisectmodule.c - PyId_insert - -Modules/_threadmodule.c - PyId_stderr - -Modules/_threadmodule.c - PyId_flush - -Modules/unicodedata.c - PyId_NFC - -Modules/unicodedata.c - PyId_NFD - -Modules/unicodedata.c - PyId_NFKC - -Modules/unicodedata.c - PyId_NFKD - -Modules/_datetimemodule.c - PyId_as_integer_ratio - -Modules/_datetimemodule.c - PyId_fromutc - -Modules/_datetimemodule.c - PyId_isoformat - -Modules/_datetimemodule.c - PyId_strftime - -Modules/_sqlite/connection.c - PyId_cursor - -Modules/cjkcodecs/multibytecodec.c - PyId_write - -Modules/_io/textio.c - PyId_close - -Modules/_io/textio.c - PyId__dealloc_warn - -Modules/_io/textio.c - PyId_decode - -Modules/_io/textio.c - PyId_fileno - -Modules/_io/textio.c - PyId_flush - -Modules/_io/textio.c - PyId_getpreferredencoding - -Modules/_io/textio.c - PyId_isatty - -Modules/_io/textio.c - PyId_mode - -Modules/_io/textio.c - PyId_name - -Modules/_io/textio.c - PyId_raw - -Modules/_io/textio.c - PyId_read - -Modules/_io/textio.c - PyId_readable - -Modules/_io/textio.c - PyId_replace - -Modules/_io/textio.c - PyId_reset - -Modules/_io/textio.c - PyId_seek - -Modules/_io/textio.c - PyId_seekable - -Modules/_io/textio.c - PyId_setstate - -Modules/_io/textio.c - PyId_strict - -Modules/_io/textio.c - PyId_tell - -Modules/_io/textio.c - PyId_writable - -Modules/_io/fileio.c - PyId_name - -Modules/_io/bufferedio.c - PyId_close - -Modules/_io/bufferedio.c - PyId__dealloc_warn - -Modules/_io/bufferedio.c - PyId_flush - -Modules/_io/bufferedio.c - PyId_isatty - -Modules/_io/bufferedio.c - PyId_mode - -Modules/_io/bufferedio.c - PyId_name - -Modules/_io/bufferedio.c - PyId_peek - -Modules/_io/bufferedio.c - PyId_read - -Modules/_io/bufferedio.c - PyId_read1 - -Modules/_io/bufferedio.c - PyId_readable - -Modules/_io/bufferedio.c - PyId_readinto - -Modules/_io/bufferedio.c - PyId_readinto1 - -Modules/_io/bufferedio.c - PyId_writable - -Modules/_io/bufferedio.c - PyId_write - -Modules/_io/iobase.c - PyId___IOBase_closed - -Modules/_io/iobase.c - PyId_read - - -# Py_IDENTIFIER (local) [] -Modules/_ssl.c fill_and_set_sslerror PyId_reason - -Modules/_ssl.c fill_and_set_sslerror PyId_library - -Modules/_ssl.c fill_and_set_sslerror PyId_verify_message - -Modules/_ssl.c fill_and_set_sslerror PyId_verify_code - -Modules/timemodule.c time_strptime PyId__strptime_time - -Modules/itertoolsmodule.c _grouper_reduce PyId_iter - -Modules/itertoolsmodule.c itertools_tee_impl PyId___copy__ - -Modules/itertoolsmodule.c cycle_reduce PyId___setstate__ - -Modules/itertoolsmodule.c zip_longest_new PyId_fillvalue - -Modules/mmapmodule.c mmap__exit__method PyId_close - -Modules/_gdbmmodule.c gdbm__exit__ PyId_close - -Modules/arraymodule.c array_array_fromfile_impl PyId_read - -Modules/arraymodule.c array_array_tofile PyId_write - -Modules/arraymodule.c array_array___reduce_ex__ PyId__array_reconstructor - -Modules/arraymodule.c array_array___reduce_ex__ PyId___dict__ - -Modules/arraymodule.c array_arrayiterator___reduce___impl PyId_iter - -Modules/_collectionsmodule.c deque_reduce PyId___dict__ - -Modules/_collectionsmodule.c defdict_reduce PyId_items - -Modules/_collectionsmodule.c _collections__count_elements_impl PyId_get - -Modules/_collectionsmodule.c _collections__count_elements_impl PyId___setitem__ - -Modules/_csv.c csv_writer PyId_write - -Modules/_asynciomodule.c get_future_loop PyId_get_loop - -Modules/_asynciomodule.c get_future_loop PyId__loop - -Modules/_asynciomodule.c future_init PyId_get_debug - -Modules/_asynciomodule.c FutureObj_get_state PyId_PENDING - -Modules/_asynciomodule.c FutureObj_get_state PyId_CANCELLED - -Modules/_asynciomodule.c FutureObj_get_state PyId_FINISHED - -Modules/_asynciomodule.c FutureObj_repr PyId__repr_info - -Modules/_asynciomodule.c FutureObj_finalize PyId_call_exception_handler - -Modules/_asynciomodule.c FutureObj_finalize PyId_message - -Modules/_asynciomodule.c FutureObj_finalize PyId_exception - -Modules/_asynciomodule.c FutureObj_finalize PyId_future - -Modules/_asynciomodule.c FutureObj_finalize PyId_source_traceback - -Modules/_asynciomodule.c register_task PyId_add - -Modules/_asynciomodule.c unregister_task PyId_discard - -Modules/_asynciomodule.c TaskObj_finalize PyId_call_exception_handler - -Modules/_asynciomodule.c TaskObj_finalize PyId_task - -Modules/_asynciomodule.c TaskObj_finalize PyId_message - -Modules/_asynciomodule.c TaskObj_finalize PyId_source_traceback - -Modules/mathmodule.c math_ceil PyId___ceil__ - -Modules/mathmodule.c math_floor PyId___floor__ - -Modules/mathmodule.c math_trunc PyId___trunc__ - -Modules/_operator.c methodcaller_reduce PyId_partial - -Modules/_lzmamodule.c build_filter_spec PyId_id - -Modules/_lzmamodule.c build_filter_spec PyId_lc - -Modules/_lzmamodule.c build_filter_spec PyId_lp - -Modules/_lzmamodule.c build_filter_spec PyId_pb - -Modules/_lzmamodule.c build_filter_spec PyId_dict_size - -Modules/_lzmamodule.c build_filter_spec PyId_dist - -Modules/_lzmamodule.c build_filter_spec PyId_start_offset - -Modules/pyexpat.c pyexpat_xmlparser_ParseFile PyId_read - -Modules/_threadmodule.c thread_excepthook_file PyId_name - -Modules/_elementtree.c _elementtree_Element_find_impl PyId_find - -Modules/_elementtree.c _elementtree_Element_findtext_impl PyId_findtext - -Modules/_elementtree.c _elementtree_Element_findall_impl PyId_findall - -Modules/_elementtree.c _elementtree_Element_iterfind_impl PyId_iterfind - -Modules/_elementtree.c treebuilder_flush_data PyId_text - -Modules/_elementtree.c treebuilder_flush_data PyId_tail - -Modules/_elementtree.c treebuilder_add_subelement PyId_append - -Modules/_elementtree.c expat_start_doctype_handler PyId_doctype - -Modules/_pickle.c _Pickle_InitState PyId_getattr - -Modules/_pickle.c _Pickler_SetOutputStream PyId_write - -Modules/_pickle.c _Unpickler_SetInputStream PyId_peek - -Modules/_pickle.c _Unpickler_SetInputStream PyId_read - -Modules/_pickle.c _Unpickler_SetInputStream PyId_readinto - -Modules/_pickle.c _Unpickler_SetInputStream PyId_readline - -Modules/_pickle.c whichmodule PyId___module__ - -Modules/_pickle.c whichmodule PyId_modules - -Modules/_pickle.c whichmodule PyId___main__ - -Modules/_pickle.c save_bytes PyId_latin1 - -Modules/_pickle.c save_dict PyId_items - -Modules/_pickle.c save_global PyId___name__ - -Modules/_pickle.c save_global PyId___qualname__ - -Modules/_pickle.c get_class PyId___class__ - -Modules/_pickle.c save_reduce PyId___name__ - -Modules/_pickle.c save_reduce PyId___newobj_ex__ - -Modules/_pickle.c save_reduce PyId___newobj__ - -Modules/_pickle.c save_reduce PyId___new__ - -Modules/_pickle.c save PyId___reduce__ - -Modules/_pickle.c save PyId___reduce_ex__ - -Modules/_pickle.c dump PyId_reducer_override - -Modules/_pickle.c _pickle_Pickler___init___impl PyId_persistent_id - -Modules/_pickle.c _pickle_Pickler___init___impl PyId_dispatch_table - -Modules/_pickle.c find_class PyId_find_class - -Modules/_pickle.c instantiate PyId___getinitargs__ - -Modules/_pickle.c instantiate PyId___new__ - -Modules/_pickle.c do_append PyId_extend - -Modules/_pickle.c do_append PyId_append - -Modules/_pickle.c load_additems PyId_add - -Modules/_pickle.c load_build PyId___setstate__ - -Modules/_pickle.c load_build PyId___dict__ - -Modules/_pickle.c _pickle_Unpickler___init___impl PyId_persistent_load - -Modules/_cursesmodule.c _curses_window_putwin PyId_write - -Modules/_cursesmodule.c _curses_getwin PyId_read - -Modules/_cursesmodule.c update_lines_cols PyId_LINES - -Modules/_cursesmodule.c update_lines_cols PyId_COLS - -Modules/_datetimemodule.c call_tzname PyId_tzname - -Modules/_datetimemodule.c make_Zreplacement PyId_replace - -Modules/_datetimemodule.c time_time PyId_time - -Modules/_datetimemodule.c build_struct_time PyId_struct_time - -Modules/_datetimemodule.c date_today PyId_fromtimestamp - -Modules/_datetimemodule.c date_strftime PyId_timetuple - -Modules/_datetimemodule.c tzinfo_reduce PyId___getinitargs__ - -Modules/_datetimemodule.c tzinfo_reduce PyId___getstate__ - -Modules/_datetimemodule.c datetime_strptime PyId__strptime_datetime - -Modules/ossaudiodev.c oss_exit PyId_close - -Modules/main.c pymain_sys_path_add_path0 PyId_path - -Modules/_sqlite/microprotocols.c pysqlite_microprotocols_adapt PyId___adapt__ - -Modules/_sqlite/microprotocols.c pysqlite_microprotocols_adapt PyId___conform__ - -Modules/_sqlite/connection.c _pysqlite_final_callback PyId_finalize - -Modules/_sqlite/connection.c pysqlite_connection_set_isolation_level PyId_upper - -Modules/_sqlite/connection.c pysqlite_connection_iterdump PyId__iterdump - -Modules/_sqlite/connection.c pysqlite_connection_create_collation PyId_upper - -Modules/_sqlite/module.c module_register_converter PyId_upper - -Modules/_sqlite/cursor.c _pysqlite_get_converter PyId_upper - -Modules/_io/_iomodule.c _io_open_impl PyId__blksize - -Modules/_io/_iomodule.c _io_open_impl PyId_isatty - -Modules/_io/_iomodule.c _io_open_impl PyId_mode - -Modules/_io/_iomodule.c _io_open_impl PyId_close - -Modules/_io/fileio.c _io_FileIO_close_impl PyId_close - -Modules/_io/iobase.c _io__IOBase_tell_impl PyId_seek - -Modules/_io/iobase.c iobase_finalize PyId__finalizing - -Modules/_io/iobase.c _io__IOBase_readlines_impl PyId_extend - -Modules/_io/iobase.c _io__RawIOBase_read_impl PyId_readall - -Modules/_ctypes/stgdict.c MakeAnonFields PyId__anonymous_ - -Modules/_ctypes/stgdict.c PyCStructUnionType_update_stgdict PyId__swappedbytes_ - -Modules/_ctypes/stgdict.c PyCStructUnionType_update_stgdict PyId__use_broken_old_ctypes_structure_semantics_ - -Modules/_ctypes/stgdict.c PyCStructUnionType_update_stgdict PyId__pack_ - -Modules/_ctypes/callproc.c ConvParam PyId__as_parameter_ - -Modules/_ctypes/callproc.c unpickle PyId___new__ - -Modules/_ctypes/callproc.c unpickle PyId___setstate__ - -Modules/_ctypes/_ctypes.c StructUnionType_new PyId__abstract_ - -Modules/_ctypes/_ctypes.c StructUnionType_new PyId__fields_ - -Modules/_ctypes/_ctypes.c CDataType_from_param PyId__as_parameter_ - -Modules/_ctypes/_ctypes.c PyCPointerType_new PyId__type_ - -Modules/_ctypes/_ctypes.c PyCPointerType_set_type PyId__type_ - -Modules/_ctypes/_ctypes.c PyCArrayType_new PyId__length_ - -Modules/_ctypes/_ctypes.c PyCArrayType_new PyId__type_ - -Modules/_ctypes/_ctypes.c c_wchar_p_from_param PyId__as_parameter_ - -Modules/_ctypes/_ctypes.c c_char_p_from_param PyId__as_parameter_ - -Modules/_ctypes/_ctypes.c c_void_p_from_param PyId__as_parameter_ - -Modules/_ctypes/_ctypes.c PyCSimpleType_new PyId__type_ - -Modules/_ctypes/_ctypes.c PyCSimpleType_from_param PyId__as_parameter_ - -Modules/_ctypes/_ctypes.c converters_from_argtypes PyId_from_param - -Modules/_ctypes/_ctypes.c make_funcptrtype_dict PyId__flags_ - -Modules/_ctypes/_ctypes.c make_funcptrtype_dict PyId__argtypes_ - -Modules/_ctypes/_ctypes.c make_funcptrtype_dict PyId__restype_ - -Modules/_ctypes/_ctypes.c make_funcptrtype_dict PyId__check_retval_ - -Modules/_ctypes/_ctypes.c PyCFuncPtr_set_restype PyId__check_retval_ - -Modules/_ctypes/_ctypes.c _build_result PyId___ctypes_from_outparam__ - -Modules/_ctypes/_ctypes.c _init_pos_args PyId__fields_ - - -# _Py_static_string [] -Modules/_pickle.c get_dotted_path PyId_dot - - -# manually cached PyUnicodeOjbect [] -Modules/_asynciomodule.c - context_kwname - -Modules/_ctypes/callproc.c _ctypes_get_errobj error_object_name - -Modules/_ctypes/_ctypes.c CreateSwappedType suffix - -Modules/_io/_iomodule.c - _PyIO_str_close - -Modules/_io/_iomodule.c - _PyIO_str_closed - -Modules/_io/_iomodule.c - _PyIO_str_decode - -Modules/_io/_iomodule.c - _PyIO_str_encode - -Modules/_io/_iomodule.c - _PyIO_str_fileno - -Modules/_io/_iomodule.c - _PyIO_str_flush - -Modules/_io/_iomodule.c - _PyIO_str_getstate - -Modules/_io/_iomodule.c - _PyIO_str_isatty - -Modules/_io/_iomodule.c - _PyIO_str_newlines - -Modules/_io/_iomodule.c - _PyIO_str_nl - -Modules/_io/_iomodule.c - _PyIO_str_peek - -Modules/_io/_iomodule.c - _PyIO_str_read - -Modules/_io/_iomodule.c - _PyIO_str_read1 - -Modules/_io/_iomodule.c - _PyIO_str_readable - -Modules/_io/_iomodule.c - _PyIO_str_readall - -Modules/_io/_iomodule.c - _PyIO_str_readinto - -Modules/_io/_iomodule.c - _PyIO_str_readline - -Modules/_io/_iomodule.c - _PyIO_str_reset - -Modules/_io/_iomodule.c - _PyIO_str_seek - -Modules/_io/_iomodule.c - _PyIO_str_seekable - -Modules/_io/_iomodule.c - _PyIO_str_setstate - -Modules/_io/_iomodule.c - _PyIO_str_tell - -Modules/_io/_iomodule.c - _PyIO_str_truncate - -Modules/_io/_iomodule.c - _PyIO_str_writable - -Modules/_io/_iomodule.c - _PyIO_str_write - -Modules/_io/_iomodule.c - _PyIO_empty_str - -Modules/_json.c _encoded_const s_null - -Modules/_json.c _encoded_const s_true - -Modules/_json.c _encoded_const s_false - -Modules/_json.c encoder_listencode_dict open_dict - -Modules/_json.c encoder_listencode_dict close_dict - -Modules/_json.c encoder_listencode_dict empty_dict - -Modules/_json.c encoder_listencode_list open_array - -Modules/_json.c encoder_listencode_list close_array - -Modules/_json.c encoder_listencode_list empty_array - -Modules/_threadmodule.c - str_dict - -Modules/_tracemalloc.c - unknown_filename - - -#----------------------- -# module initialized once - other PyObject -# [] - -# cached during module init [] -Modules/_asynciomodule.c - asyncio_mod - -Modules/_asynciomodule.c - traceback_extract_stack - -Modules/_asynciomodule.c - asyncio_get_event_loop_policy - -Modules/_asynciomodule.c - asyncio_future_repr_info_func - -Modules/_asynciomodule.c - asyncio_iscoroutine_func - -Modules/_asynciomodule.c - asyncio_task_get_stack_func - -Modules/_asynciomodule.c - asyncio_task_print_stack_func - -Modules/_asynciomodule.c - asyncio_task_repr_info_func - -Modules/_asynciomodule.c - asyncio_InvalidStateError - -Modules/_asynciomodule.c - asyncio_CancelledError - -Modules/_zoneinfo.c - io_open - -Modules/_zoneinfo.c - _tzpath_find_tzfile - -Modules/_zoneinfo.c - _common_mod - - -# other [] -Modules/_ctypes/_ctypes.c - _unpickle - -Modules/_ctypes/_ctypes.c PyCArrayType_from_ctype cache - -Modules/_cursesmodule.c - ModDict - -Modules/_datetimemodule.c datetime_strptime module - -Modules/_datetimemodule.c - PyDateTime_TimeZone_UTC - -Modules/_datetimemodule.c - PyDateTime_Epoch - -Modules/_datetimemodule.c - us_per_ms - -Modules/_datetimemodule.c - us_per_second - -Modules/_datetimemodule.c - us_per_minute - -Modules/_datetimemodule.c - us_per_hour - -Modules/_datetimemodule.c - us_per_day - -Modules/_datetimemodule.c - us_per_week - -Modules/_datetimemodule.c - seconds_per_day - -Modules/_decimal/_decimal.c PyInit__decimal capsule - -Modules/_decimal/_decimal.c - basic_context_template - -Modules/_decimal/_decimal.c - current_context_var - -Modules/_decimal/_decimal.c - default_context_template - -Modules/_decimal/_decimal.c - extended_context_template - -Modules/_decimal/_decimal.c - round_map - -Modules/_decimal/_decimal.c - Rational - -Modules/_decimal/_decimal.c - SignalTuple - -Modules/_functoolsmodule.c - kwd_mark - -Modules/_io/_iomodule.c - _PyIO_empty_bytes - -Modules/_json.c raise_errmsg JSONDecodeError - -Modules/_sqlite/microprotocols.c - psyco_adapters - -Modules/_sqlite/module.h - _pysqlite_converters - -Modules/_ssl.c - err_codes_to_names - -Modules/_ssl.c - err_names_to_codes - -Modules/_ssl.c - lib_codes_to_names - -# XXX This should have been found by the analyzer but wasn't: -Modules/_ssl.c - _ssl_locks - -Modules/_struct.c - cache - -Modules/_tracemalloc.c - tracemalloc_empty_traceback - -Modules/arraymodule.c array_array___reduce_ex__ array_reconstructor - -Modules/cjkcodecs/cjkcodecs.h getmultibytecodec cofunc - -Modules/signalmodule.c - DefaultHandler - -Modules/signalmodule.c - IgnoreHandler - -Modules/signalmodule.c - IntHandler - - -#----------------------- -# module initialized once - non-PyObject -# [] - -# pre-allocated buffer [] -Modules/getbuildinfo.c Py_GetBuildInfo buildinfo - -Modules/nismodule.c nisproc_maplist_2 res - -Modules/pyexpat.c PyUnknownEncodingHandler template_buffer - - -# other [] -Include/datetime.h - PyDateTimeAPI - -Modules/_asynciomodule.c - module_initialized - -Modules/_ctypes/cfield.c _ctypes_get_fielddesc initialized - -Modules/_ctypes/malloc_closure.c - _pagesize - -Modules/_cursesmodule.c - initialised - -Modules/_cursesmodule.c - initialised_setupterm - -Modules/_cursesmodule.c - initialisedcolors - -Modules/_cursesmodule.c - screen_encoding - -Modules/_cursesmodule.c PyInit__curses PyCurses_API - -Modules/_datetimemodule.c - CAPI - -Modules/_decimal/_decimal.c PyInit__decimal initialized - -Modules/_decimal/_decimal.c - _py_long_multiply - -Modules/_decimal/_decimal.c - _py_long_floor_divide - -Modules/_decimal/_decimal.c - _py_long_power - -Modules/_decimal/_decimal.c - _py_float_abs - -Modules/_decimal/_decimal.c - _py_long_bit_length - -Modules/_decimal/_decimal.c - _py_float_as_integer_ratio - -Modules/_decimal/_decimal.c - _decimal_api - -Modules/_elementtree.c - expat_capi - -Modules/_io/bufferedio.c _PyIO_trap_eintr eintr_int - -Modules/_sqlite/module.h - _pysqlite_enable_callback_tracebacks - -Modules/_sqlite/module.h - pysqlite_BaseTypeAdapted - -Modules/_ssl.c - _ssl_locks_count - -Modules/cjkcodecs/cjkcodecs.h - codec_list - -Modules/cjkcodecs/cjkcodecs.h - mapping_list - -# XXX This should have been found by the analyzer but wasn't: -Python/fileutils.c - _Py_open_cloexec_works - -Modules/getaddrinfo.c - gai_afdl - -Modules/posixmodule.c os_dup2_impl dup3_works - -Modules/posixmodule.c - structseq_new - -Modules/posixmodule.c - ticks_per_second - -Modules/pyexpat.c PyInit_pyexpat capi - -Modules/readline.c - using_libedit_emulation - -Modules/readline.c - libedit_history_start - -Modules/resource.c - initialized - -Modules/signalmodule.c - initialized - -Modules/socketmodule.c - accept4_works - -Modules/socketmodule.c - sock_cloexec_works - -Modules/socketmodule.c - PySocketModuleAPI - -Modules/spwdmodule.c - initialized - -Modules/timemodule.c - initialized - -Modules/timemodule.c _PyTime_GetClockWithInfo initialized - -Modules/timemodule.c _PyTime_GetProcessTimeWithInfo ticks_per_second - - -#----------------------- -# module state -# [] - -Modules/posixmodule.c - environ - - -# PyObject [] -Modules/_asynciomodule.c - cached_running_holder - -Modules/_asynciomodule.c - fi_freelist - -Modules/_asynciomodule.c - fi_freelist_len - -Modules/_asynciomodule.c - all_tasks - -Modules/_asynciomodule.c - current_tasks - -Modules/_asynciomodule.c - iscoroutine_typecache - -Modules/_ctypes/_ctypes.c - _ctypes_ptrtype_cache - -Modules/_tkinter.c - tcl_lock - -Modules/_tkinter.c - excInCmd - -Modules/_tkinter.c - valInCmd - -Modules/_tkinter.c - trbInCmd - -Modules/_zoneinfo.c - TIMEDELTA_CACHE - -Modules/_zoneinfo.c - ZONEINFO_WEAK_CACHE - -Modules/faulthandler.c - fatal_error - -Modules/faulthandler.c - thread - -Modules/faulthandler.c - user_signals - -Modules/faulthandler.c - stack - -Modules/faulthandler.c - old_stack - -Modules/signalmodule.c - Handlers - -Modules/syslogmodule.c - S_ident_o - - -# other [] -Modules/_asynciomodule.c - cached_running_holder_tsid - -Modules/_asynciomodule.c - task_name_counter - -Modules/_ctypes/cfield.c - formattable - -Modules/_ctypes/malloc_closure.c - free_list - -Modules/_curses_panel.c - lop - -Modules/_tkinter.c - quitMainLoop - -Modules/_tkinter.c - errorInCmd - -Modules/_tkinter.c - Tkinter_busywaitinterval - -Modules/_tkinter.c - call_mutex - -Modules/_tkinter.c - var_mutex - -Modules/_tkinter.c - command_mutex - -Modules/_tkinter.c - HeadFHCD - -Modules/_tkinter.c - stdin_ready - -Modules/_tkinter.c - event_tstate - -Modules/_tracemalloc.c - allocators - -Modules/_tracemalloc.c - tables_lock - -Modules/_tracemalloc.c - tracemalloc_traced_memory - -Modules/_tracemalloc.c - tracemalloc_peak_traced_memory - -Modules/_tracemalloc.c - tracemalloc_filenames - -Modules/_tracemalloc.c - tracemalloc_traceback - -Modules/_tracemalloc.c - tracemalloc_tracebacks - -Modules/_tracemalloc.c - tracemalloc_traces - -Modules/_tracemalloc.c - tracemalloc_domains - -Modules/_tracemalloc.c - tracemalloc_reentrant_key - -Modules/_xxsubinterpretersmodule.c - _globals - -Modules/_zoneinfo.c - ZONEINFO_STRONG_CACHE - -Modules/_zoneinfo.c - ZONEINFO_STRONG_CACHE_MAX_SIZE - -Modules/_zoneinfo.c - NO_TTINFO - -Modules/faulthandler.c faulthandler_dump_traceback reentrant - -Modules/readline.c - completer_word_break_characters - -Modules/readline.c - _history_length - -Modules/readline.c - should_auto_add_history - -Modules/readline.c - sigwinch_received - -Modules/readline.c - sigwinch_ohandler - -Modules/readline.c - completed_input_string - -Modules/rotatingtree.c - random_stream - -Modules/rotatingtree.c - random_value - -Modules/signalmodule.c - is_tripped - -Modules/signalmodule.c - wakeup - -Modules/socketmodule.c - defaulttimeout - -Modules/syslogmodule.c - S_log_open - - -#----------------------- -# runtime state -# [] - -# PyObject [] -Objects/typeobject.c resolve_slotdups pname - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c - extensions - - -# allocator [] -Objects/obmalloc.c - _Py_tracemalloc_config - -Objects/obmalloc.c - _PyObject_Arena - -Objects/obmalloc.c - arenas - -Objects/obmalloc.c - maxarenas - -Objects/obmalloc.c - unused_arena_objects - -Objects/obmalloc.c - usable_arenas - -Objects/obmalloc.c - nfp2lasta - -Objects/obmalloc.c - narenas_currently_allocated - -Objects/obmalloc.c - ntimes_arena_allocated - -Objects/obmalloc.c - narenas_highwater - -Objects/obmalloc.c - raw_allocated_blocks - -Objects/obmalloc.c new_arena debug_stats - - -# REPL [] -Parser/myreadline.c - _PyOS_ReadlineLock - -Parser/myreadline.c - _PyOS_ReadlineTState - -Parser/myreadline.c - PyOS_InputHook - -Parser/myreadline.c - PyOS_ReadlineFunctionPointer - - -# other [] -Objects/dictobject.c - pydict_global_version - -Objects/floatobject.c - double_format - -Objects/floatobject.c - float_format - -Objects/floatobject.c - detected_double_format - -Objects/floatobject.c - detected_float_format - -Objects/moduleobject.c - max_module_number - -Objects/object.c - _Py_RefTotal - -Objects/typeobject.c - next_version_tag - -Objects/typeobject.c resolve_slotdups ptrs - -Parser/pegen.c - memo_statistics - -# XXX This should have been found by the analyzer but wasn't: -Python/bootstrap_hash.c - urandom_cache - -# XXX This should have been found by the analyzer but wasn't: -Python/ceval.c - lltrace - -# XXX This should have been found by the analyzer but wasn't: -Python/ceval.c make_pending_calls busy - -Python/dynload_shlib.c - handles - -Python/dynload_shlib.c - nhandles - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c - import_lock_level - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c - import_lock_thread - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c import_find_and_load accumulated - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c import_find_and_load import_level - -# XXX This should have been found by the analyzer but wasn't: -Python/pylifecycle.c - _Py_UnhandledKeyboardInterrupt - -# XXX This should have been found by the analyzer but wasn't: -Python/pylifecycle.c fatal_error reentrant - diff --git a/Tools/c-analyzer/table-file.py b/Tools/c-analyzer/table-file.py new file mode 100644 index 0000000000000..ba3dd29055e03 --- /dev/null +++ b/Tools/c-analyzer/table-file.py @@ -0,0 +1,133 @@ + +def iter_clean_lines(lines): + lines = iter(lines) + for line in lines: + line = line.strip() + if line.startswith('# XXX'): + continue + yield line + + +def parse_table_lines(lines): + lines = iter_clean_lines(lines) + + for line in lines: + if line.startswith(('####', '#----')): + kind = 0 if line[1] == '#' else 1 + try: + line = next(lines).strip() + except StopIteration: + line = '' + if not line.startswith('# '): + raise NotImplementedError(line) + yield kind, line[2:].lstrip() + continue + + maybe = None + while line.startswith('#'): + if line != '#' and line[1] == ' ': + maybe = line[2:].lstrip() + try: + line = next(lines).strip() + except StopIteration: + return + if not line: + break + else: + if line: + if maybe: + yield 2, maybe + yield 'row', line + + +def iter_sections(lines): + header = None + section = [] + for kind, value in parse_table_lines(lines): + if kind == 'row': + if not section: + if header is None: + header = value + continue + raise NotImplementedError(value) + yield tuple(section), value + else: + if header is None: + header = False + section[kind:] = [value] + + +def collect_sections(lines): + sections = {} + for section, row in iter_sections(lines): + if section not in sections: + sections[section] = [row] + else: + sections[section].append(row) + return sections + + +def collate_sections(lines): + collated = {} + for section, rows in collect_sections(lines).items(): + parent = collated + current = () + for name in section: + current += (name,) + try: + child, secrows, totalrows = parent[name] + except KeyError: + child = {} + secrows = [] + totalrows = [] + parent[name] = (child, secrows, totalrows) + parent = child + if current == section: + secrows.extend(rows) + totalrows.extend(rows) + return collated + + +############################# +# the commands + +def cmd_count_by_section(lines): + total = 0 + def render_tree(root, depth=0): + nonlocal total + indent = ' ' * depth + for name, data in root.items(): + subroot, rows, totalrows = data + sectotal = f'({len(totalrows)})' if totalrows != rows else '' + count = len(rows) if rows else '' + yield f'{sectotal:>7} {count:>4} {indent}{name}' + yield from render_tree(subroot, depth+1) + total += len(rows) + sections = collate_sections(lines) + yield from render_tree(sections) + yield f'(total: {total})' + + +############################# +# the script + +def parse_args(argv=None, prog=None): + import argparse + parser = argparse.ArgumentParser(prog=prog) + parser.add_argument('filename') + + args = parser.parse_args(argv) + ns = vars(args) + + return ns + + +def main(filename): + with open(filename) as infile: + for line in cmd_count_by_section(infile): + print(line) + + +if __name__ == '__main__': + kwargs = parse_args() + main(**kwargs) From webhook-mailer at python.org Wed Feb 9 07:30:40 2022 From: webhook-mailer at python.org (markshannon) Date: Wed, 09 Feb 2022 12:30:40 -0000 Subject: [Python-checkins] bpo-46072: Output stats as markdown with collapsible sections. (GH-31228) Message-ID: https://github.com/python/cpython/commit/f71a69aa9209cf67cc1060051b147d6afa379bba commit: f71a69aa9209cf67cc1060051b147d6afa379bba branch: main author: Mark Shannon committer: markshannon date: 2022-02-09T12:30:26Z summary: bpo-46072: Output stats as markdown with collapsible sections. (GH-31228) files: M Python/specialize.c M Tools/scripts/summarize_stats.py diff --git a/Python/specialize.c b/Python/specialize.c index 21759231d28e6..b5e4de5d6d217 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -481,112 +481,123 @@ initial_counter_value(void) { #define SPEC_FAIL_OUT_OF_VERSIONS 3 #define SPEC_FAIL_OUT_OF_RANGE 4 #define SPEC_FAIL_EXPECTED_ERROR 5 +#define SPEC_FAIL_WRONG_NUMBER_ARGUMENTS 6 + +#define SPEC_FAIL_LOAD_GLOBAL_NON_STRING_OR_SPLIT 18 /* Attributes */ -#define SPEC_FAIL_NON_STRING_OR_SPLIT 6 -#define SPEC_FAIL_MODULE_ATTR_NOT_FOUND 7 -#define SPEC_FAIL_OVERRIDING_DESCRIPTOR 8 -#define SPEC_FAIL_NON_OVERRIDING_DESCRIPTOR 9 -#define SPEC_FAIL_NOT_DESCRIPTOR 10 -#define SPEC_FAIL_METHOD 11 -#define SPEC_FAIL_MUTABLE_CLASS 12 -#define SPEC_FAIL_PROPERTY 13 -#define SPEC_FAIL_NON_OBJECT_SLOT 14 -#define SPEC_FAIL_READ_ONLY 15 -#define SPEC_FAIL_AUDITED_SLOT 16 -#define SPEC_FAIL_NOT_MANAGED_DICT 17 +#define SPEC_FAIL_ATTR_OVERRIDING_DESCRIPTOR 8 +#define SPEC_FAIL_ATTR_NON_OVERRIDING_DESCRIPTOR 9 +#define SPEC_FAIL_ATTR_NOT_DESCRIPTOR 10 +#define SPEC_FAIL_ATTR_METHOD 11 +#define SPEC_FAIL_ATTR_MUTABLE_CLASS 12 +#define SPEC_FAIL_ATTR_PROPERTY 13 +#define SPEC_FAIL_ATTR_NON_OBJECT_SLOT 14 +#define SPEC_FAIL_ATTR_READ_ONLY 15 +#define SPEC_FAIL_ATTR_AUDITED_SLOT 16 +#define SPEC_FAIL_ATTR_NOT_MANAGED_DICT 17 +#define SPEC_FAIL_ATTR_NON_STRING_OR_SPLIT 18 +#define SPEC_FAIL_ATTR_MODULE_ATTR_NOT_FOUND 19 /* Methods */ -#define SPEC_FAIL_IS_ATTR 15 -#define SPEC_FAIL_DICT_SUBCLASS 16 -#define SPEC_FAIL_BUILTIN_CLASS_METHOD 17 -#define SPEC_FAIL_CLASS_METHOD_OBJ 18 -#define SPEC_FAIL_OBJECT_SLOT 19 -#define SPEC_FAIL_HAS_DICT 20 -#define SPEC_FAIL_HAS_MANAGED_DICT 21 -#define SPEC_FAIL_INSTANCE_ATTRIBUTE 22 -#define SPEC_FAIL_METACLASS_ATTRIBUTE 23 - -/* Binary subscr */ - -#define SPEC_FAIL_ARRAY_INT 8 -#define SPEC_FAIL_ARRAY_SLICE 9 -#define SPEC_FAIL_LIST_SLICE 10 -#define SPEC_FAIL_TUPLE_SLICE 11 -#define SPEC_FAIL_STRING_INT 12 -#define SPEC_FAIL_STRING_SLICE 13 -#define SPEC_FAIL_BUFFER_INT 15 -#define SPEC_FAIL_BUFFER_SLICE 16 -#define SPEC_FAIL_SEQUENCE_INT 17 +#define SPEC_FAIL_LOAD_METHOD_OVERRIDING_DESCRIPTOR 8 +#define SPEC_FAIL_LOAD_METHOD_NON_OVERRIDING_DESCRIPTOR 9 +#define SPEC_FAIL_LOAD_METHOD_NOT_DESCRIPTOR 10 +#define SPEC_FAIL_LOAD_METHOD_METHOD 11 +#define SPEC_FAIL_LOAD_METHOD_MUTABLE_CLASS 12 +#define SPEC_FAIL_LOAD_METHOD_PROPERTY 13 +#define SPEC_FAIL_LOAD_METHOD_NON_OBJECT_SLOT 14 +#define SPEC_FAIL_LOAD_METHOD_IS_ATTR 15 +#define SPEC_FAIL_LOAD_METHOD_DICT_SUBCLASS 16 +#define SPEC_FAIL_LOAD_METHOD_BUILTIN_CLASS_METHOD 17 +#define SPEC_FAIL_LOAD_METHOD_CLASS_METHOD_OBJ 18 +#define SPEC_FAIL_LOAD_METHOD_OBJECT_SLOT 19 +#define SPEC_FAIL_LOAD_METHOD_HAS_DICT 20 +#define SPEC_FAIL_LOAD_METHOD_HAS_MANAGED_DICT 21 +#define SPEC_FAIL_LOAD_METHOD_INSTANCE_ATTRIBUTE 22 +#define SPEC_FAIL_LOAD_METHOD_METACLASS_ATTRIBUTE 23 + +/* Binary subscr and store subscr */ + +#define SPEC_FAIL_SUBSCR_ARRAY_INT 8 +#define SPEC_FAIL_SUBSCR_ARRAY_SLICE 9 +#define SPEC_FAIL_SUBSCR_LIST_SLICE 10 +#define SPEC_FAIL_SUBSCR_TUPLE_SLICE 11 +#define SPEC_FAIL_SUBSCR_STRING_INT 12 +#define SPEC_FAIL_SUBSCR_STRING_SLICE 13 +#define SPEC_FAIL_SUBSCR_BUFFER_INT 15 +#define SPEC_FAIL_SUBSCR_BUFFER_SLICE 16 +#define SPEC_FAIL_SUBSCR_SEQUENCE_INT 17 /* Store subscr */ -#define SPEC_FAIL_BYTEARRAY_INT 18 -#define SPEC_FAIL_BYTEARRAY_SLICE 19 -#define SPEC_FAIL_PY_SIMPLE 20 -#define SPEC_FAIL_PY_OTHER 21 -#define SPEC_FAIL_DICT_SUBCLASS_NO_OVERRIDE 22 +#define SPEC_FAIL_SUBSCR_BYTEARRAY_INT 18 +#define SPEC_FAIL_SUBSCR_BYTEARRAY_SLICE 19 +#define SPEC_FAIL_SUBSCR_PY_SIMPLE 20 +#define SPEC_FAIL_SUBSCR_PY_OTHER 21 +#define SPEC_FAIL_SUBSCR_DICT_SUBCLASS_NO_OVERRIDE 22 /* Binary add */ -#define SPEC_FAIL_NON_FUNCTION_SCOPE 11 -#define SPEC_FAIL_DIFFERENT_TYPES 12 +#define SPEC_FAIL_BINARY_OP_DIFFERENT_TYPES 12 /* Calls */ -#define SPEC_FAIL_COMPLEX_PARAMETERS 8 -#define SPEC_FAIL_WRONG_NUMBER_ARGUMENTS 9 -#define SPEC_FAIL_CO_NOT_OPTIMIZED 10 +#define SPEC_FAIL_CALL_COMPLEX_PARAMETERS 9 +#define SPEC_FAIL_CALL_CO_NOT_OPTIMIZED 10 /* SPEC_FAIL_METHOD defined as 11 above */ -#define SPEC_FAIL_PYCFUNCTION 13 -#define SPEC_FAIL_PYCFUNCTION_WITH_KEYWORDS 14 -#define SPEC_FAIL_PYCFUNCTION_FAST_WITH_KEYWORDS 15 -#define SPEC_FAIL_PYCFUNCTION_NOARGS 16 -#define SPEC_FAIL_BAD_CALL_FLAGS 17 -#define SPEC_FAIL_CLASS 18 -#define SPEC_FAIL_PYTHON_CLASS 19 -#define SPEC_FAIL_C_METHOD_CALL 20 -#define SPEC_FAIL_BOUND_METHOD 21 +#define SPEC_FAIL_CALL_INSTANCE_METHOD 11 +#define SPEC_FAIL_CALL_CMETHOD 12 +#define SPEC_FAIL_CALL_PYCFUNCTION 13 +#define SPEC_FAIL_CALL_PYCFUNCTION_WITH_KEYWORDS 14 +#define SPEC_FAIL_CALL_PYCFUNCTION_FAST_WITH_KEYWORDS 15 +#define SPEC_FAIL_CALL_PYCFUNCTION_NOARGS 16 +#define SPEC_FAIL_CALL_BAD_CALL_FLAGS 17 +#define SPEC_FAIL_CALL_CLASS 18 +#define SPEC_FAIL_CALL_PYTHON_CLASS 19 +#define SPEC_FAIL_CALL_C_METHOD_CALL 20 +#define SPEC_FAIL_CALL_BOUND_METHOD 21 #define SPEC_FAIL_CALL_STR 22 -#define SPEC_FAIL_CLASS_NO_VECTORCALL 23 -#define SPEC_FAIL_CLASS_MUTABLE 24 -#define SPEC_FAIL_KWNAMES 25 -#define SPEC_FAIL_METHOD_WRAPPER 26 -#define SPEC_FAIL_OPERATOR_WRAPPER 27 +#define SPEC_FAIL_CALL_CLASS_NO_VECTORCALL 23 +#define SPEC_FAIL_CALL_CLASS_MUTABLE 24 +#define SPEC_FAIL_CALL_KWNAMES 25 +#define SPEC_FAIL_CALL_METHOD_WRAPPER 26 +#define SPEC_FAIL_CALL_OPERATOR_WRAPPER 27 /* COMPARE_OP */ -#define SPEC_FAIL_STRING_COMPARE 13 -#define SPEC_FAIL_NOT_FOLLOWED_BY_COND_JUMP 14 -#define SPEC_FAIL_BIG_INT 15 -#define SPEC_FAIL_COMPARE_BYTES 16 -#define SPEC_FAIL_COMPARE_TUPLE 17 -#define SPEC_FAIL_COMPARE_LIST 18 -#define SPEC_FAIL_COMPARE_SET 19 -#define SPEC_FAIL_COMPARE_BOOL 20 -#define SPEC_FAIL_COMPARE_BASEOBJECT 21 -#define SPEC_FAIL_COMPARE_FLOAT_LONG 22 -#define SPEC_FAIL_COMPARE_LONG_FLOAT 23 +#define SPEC_FAIL_COMPARE_OP_DIFFERENT_TYPES 12 +#define SPEC_FAIL_COMPARE_OP_STRING 13 +#define SPEC_FAIL_COMPARE_OP_NOT_FOLLOWED_BY_COND_JUMP 14 +#define SPEC_FAIL_COMPARE_OP_BIG_INT 15 +#define SPEC_FAIL_COMPARE_OP_BYTES 16 +#define SPEC_FAIL_COMPARE_OP_TUPLE 17 +#define SPEC_FAIL_COMPARE_OP_LIST 18 +#define SPEC_FAIL_COMPARE_OP_SET 19 +#define SPEC_FAIL_COMPARE_OP_BOOL 20 +#define SPEC_FAIL_COMPARE_OP_BASEOBJECT 21 +#define SPEC_FAIL_COMPARE_OP_FLOAT_LONG 22 +#define SPEC_FAIL_COMPARE_OP_LONG_FLOAT 23 /* FOR_ITER */ -#define SPEC_FAIL_ITER_GENERATOR 10 -#define SPEC_FAIL_ITER_COROUTINE 11 -#define SPEC_FAIL_ITER_ASYNC_GENERATOR 12 -#define SPEC_FAIL_ITER_LIST 13 -#define SPEC_FAIL_ITER_TUPLE 14 -#define SPEC_FAIL_ITER_SET 15 -#define SPEC_FAIL_ITER_STRING 16 -#define SPEC_FAIL_ITER_BYTES 17 -#define SPEC_FAIL_ITER_RANGE 18 -#define SPEC_FAIL_ITER_ITERTOOLS 19 -#define SPEC_FAIL_ITER_DICT_KEYS 20 -#define SPEC_FAIL_ITER_DICT_ITEMS 21 -#define SPEC_FAIL_ITER_DICT_VALUES 22 -#define SPEC_FAIL_ITER_ENUMERATE 23 +#define SPEC_FAIL_FOR_ITER_GENERATOR 10 +#define SPEC_FAIL_FOR_ITER_COROUTINE 11 +#define SPEC_FAIL_FOR_ITER_ASYNC_GENERATOR 12 +#define SPEC_FAIL_FOR_ITER_LIST 13 +#define SPEC_FAIL_FOR_ITER_TUPLE 14 +#define SPEC_FAIL_FOR_ITER_SET 15 +#define SPEC_FAIL_FOR_ITER_STRING 16 +#define SPEC_FAIL_FOR_ITER_BYTES 17 +#define SPEC_FAIL_FOR_ITER_RANGE 18 +#define SPEC_FAIL_FOR_ITER_ITERTOOLS 19 +#define SPEC_FAIL_FOR_ITER_DICT_KEYS 20 +#define SPEC_FAIL_FOR_ITER_DICT_ITEMS 21 +#define SPEC_FAIL_FOR_ITER_DICT_VALUES 22 +#define SPEC_FAIL_FOR_ITER_ENUMERATE 23 /* UNPACK_SEQUENCE */ -#define SPEC_FAIL_TUPLE 10 -#define SPEC_FAIL_LIST 11 +#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE 10 +#define SPEC_FAIL_UNPACK_SEQUENCE_LIST 11 static int @@ -604,14 +615,14 @@ specialize_module_load_attr( return -1; } if (dict->ma_keys->dk_kind != DICT_KEYS_UNICODE) { - SPECIALIZATION_FAIL(opcode, SPEC_FAIL_NON_STRING_OR_SPLIT); + SPECIALIZATION_FAIL(opcode, SPEC_FAIL_ATTR_NON_STRING_OR_SPLIT); return -1; } Py_ssize_t index = _PyDict_GetItemHint(dict, &_Py_ID(__getattr__), -1, &value); assert(index != DKIX_ERROR); if (index != DKIX_EMPTY) { - SPECIALIZATION_FAIL(opcode, SPEC_FAIL_MODULE_ATTR_NOT_FOUND); + SPECIALIZATION_FAIL(opcode, SPEC_FAIL_ATTR_MODULE_ATTR_NOT_FOUND); return -1; } index = _PyDict_GetItemHint(dict, name, -1, &value); @@ -721,7 +732,7 @@ specialize_dict_access( kind == BUILTIN_CLASSMETHOD || kind == PYTHON_CLASSMETHOD); // No descriptor, or non overriding. if ((type->tp_flags & Py_TPFLAGS_MANAGED_DICT) == 0) { - SPECIALIZATION_FAIL(base_op, SPEC_FAIL_NOT_MANAGED_DICT); + SPECIALIZATION_FAIL(base_op, SPEC_FAIL_ATTR_NOT_MANAGED_DICT); return 0; } PyObject **dictptr = _PyObject_ManagedDictPointer(owner); @@ -783,13 +794,13 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, Sp DescriptorClassification kind = analyze_descriptor(type, name, &descr, 0); switch(kind) { case OVERRIDING: - SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_OVERRIDING_DESCRIPTOR); + SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_OVERRIDING_DESCRIPTOR); goto fail; case METHOD: - SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_METHOD); + SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_METHOD); goto fail; case PROPERTY: - SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_PROPERTY); + SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_PROPERTY); goto fail; case OBJECT_SLOT: { @@ -797,7 +808,7 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, Sp struct PyMemberDef *dmem = member->d_member; Py_ssize_t offset = dmem->offset; if (dmem->flags & PY_AUDIT_READ) { - SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_AUDITED_SLOT); + SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_AUDITED_SLOT); goto fail; } if (offset != (uint16_t)offset) { @@ -821,10 +832,10 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, Sp goto success; } case OTHER_SLOT: - SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_NON_OBJECT_SLOT); + SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_NON_OBJECT_SLOT); goto fail; case MUTABLE: - SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_MUTABLE_CLASS); + SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_MUTABLE_CLASS); goto fail; case GETSET_OVERRIDDEN: SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_OVERRIDDEN); @@ -872,13 +883,13 @@ _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, S DescriptorClassification kind = analyze_descriptor(type, name, &descr, 1); switch(kind) { case OVERRIDING: - SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_OVERRIDING_DESCRIPTOR); + SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_ATTR_OVERRIDING_DESCRIPTOR); goto fail; case METHOD: - SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_METHOD); + SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_ATTR_METHOD); goto fail; case PROPERTY: - SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_PROPERTY); + SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_ATTR_PROPERTY); goto fail; case OBJECT_SLOT: { @@ -886,7 +897,7 @@ _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, S struct PyMemberDef *dmem = member->d_member; Py_ssize_t offset = dmem->offset; if (dmem->flags & READONLY) { - SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_READ_ONLY); + SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_ATTR_READ_ONLY); goto fail; } if (offset != (uint16_t)offset) { @@ -902,10 +913,10 @@ _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, S } case DUNDER_CLASS: case OTHER_SLOT: - SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_NON_OBJECT_SLOT); + SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_ATTR_NON_OBJECT_SLOT); goto fail; case MUTABLE: - SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_MUTABLE_CLASS); + SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_ATTR_MUTABLE_CLASS); goto fail; case GETSET_OVERRIDDEN: SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_OVERRIDDEN); @@ -947,31 +958,31 @@ load_method_fail_kind(DescriptorClassification kind) { switch (kind) { case OVERRIDING: - return SPEC_FAIL_OVERRIDING_DESCRIPTOR; + return SPEC_FAIL_LOAD_METHOD_OVERRIDING_DESCRIPTOR; case METHOD: - return SPEC_FAIL_METHOD; + return SPEC_FAIL_LOAD_METHOD_METHOD; case PROPERTY: - return SPEC_FAIL_PROPERTY; + return SPEC_FAIL_LOAD_METHOD_PROPERTY; case OBJECT_SLOT: - return SPEC_FAIL_OBJECT_SLOT; + return SPEC_FAIL_LOAD_METHOD_OBJECT_SLOT; case OTHER_SLOT: - return SPEC_FAIL_NON_OBJECT_SLOT; + return SPEC_FAIL_LOAD_METHOD_NON_OBJECT_SLOT; case DUNDER_CLASS: return SPEC_FAIL_OTHER; case MUTABLE: - return SPEC_FAIL_MUTABLE_CLASS; + return SPEC_FAIL_LOAD_METHOD_MUTABLE_CLASS; case GETSET_OVERRIDDEN: return SPEC_FAIL_OVERRIDDEN; case BUILTIN_CLASSMETHOD: - return SPEC_FAIL_BUILTIN_CLASS_METHOD; + return SPEC_FAIL_LOAD_METHOD_BUILTIN_CLASS_METHOD; case PYTHON_CLASSMETHOD: - return SPEC_FAIL_CLASS_METHOD_OBJ; + return SPEC_FAIL_LOAD_METHOD_CLASS_METHOD_OBJ; case NON_OVERRIDING: - return SPEC_FAIL_NON_OVERRIDING_DESCRIPTOR; + return SPEC_FAIL_LOAD_METHOD_NON_OVERRIDING_DESCRIPTOR; case NON_DESCRIPTOR: - return SPEC_FAIL_NOT_DESCRIPTOR; + return SPEC_FAIL_LOAD_METHOD_NOT_DESCRIPTOR; case ABSENT: - return SPEC_FAIL_INSTANCE_ATTRIBUTE; + return SPEC_FAIL_LOAD_METHOD_INSTANCE_ATTRIBUTE; } Py_UNREACHABLE(); } @@ -995,7 +1006,7 @@ specialize_class_load_method(PyObject *owner, _Py_CODEUNIT *instr, PyObject *nam #ifdef Py_STATS case ABSENT: if (_PyType_Lookup(Py_TYPE(owner), name) != NULL) { - SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_METACLASS_ATTRIBUTE); + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_LOAD_METHOD_METACLASS_ATTRIBUTE); } else { SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_EXPECTED_ERROR); @@ -1051,13 +1062,13 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, if (owner_cls->tp_flags & Py_TPFLAGS_MANAGED_DICT) { PyObject **owner_dictptr = _PyObject_ManagedDictPointer(owner); if (*owner_dictptr) { - SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_HAS_MANAGED_DICT); + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_LOAD_METHOD_HAS_MANAGED_DICT); goto fail; } PyDictKeysObject *keys = ((PyHeapTypeObject *)owner_cls)->ht_cached_keys; Py_ssize_t index = _PyDictKeys_StringLookup(keys, name); if (index != DKIX_EMPTY) { - SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_IS_ATTR); + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_LOAD_METHOD_IS_ATTR); goto fail; } uint32_t keys_version = _PyDictKeys_GetVersionForCurrentState(keys); @@ -1073,7 +1084,7 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, *instr = _Py_MAKECODEUNIT(LOAD_METHOD_NO_DICT, _Py_OPARG(*instr)); } else { - SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_HAS_DICT); + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_LOAD_METHOD_HAS_DICT); goto fail; } } @@ -1122,7 +1133,7 @@ _Py_Specialize_LoadGlobal( PyDictKeysObject * globals_keys = ((PyDictObject *)globals)->ma_keys; Py_ssize_t index = _PyDictKeys_StringLookup(globals_keys, name); if (index == DKIX_ERROR) { - SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_NON_STRING_OR_SPLIT); + SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_LOAD_GLOBAL_NON_STRING_OR_SPLIT); goto fail; } if (index != DKIX_EMPTY) { @@ -1144,7 +1155,7 @@ _Py_Specialize_LoadGlobal( PyDictKeysObject * builtin_keys = ((PyDictObject *)builtins)->ma_keys; index = _PyDictKeys_StringLookup(builtin_keys, name); if (index == DKIX_ERROR) { - SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_NON_STRING_OR_SPLIT); + SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_LOAD_GLOBAL_NON_STRING_OR_SPLIT); goto fail; } if (index != (uint16_t)index) { @@ -1183,34 +1194,34 @@ binary_subscr_fail_kind(PyTypeObject *container_type, PyObject *sub) { if (container_type == &PyUnicode_Type) { if (PyLong_CheckExact(sub)) { - return SPEC_FAIL_STRING_INT; + return SPEC_FAIL_SUBSCR_STRING_INT; } if (PySlice_Check(sub)) { - return SPEC_FAIL_STRING_SLICE; + return SPEC_FAIL_SUBSCR_STRING_SLICE; } return SPEC_FAIL_OTHER; } else if (strcmp(container_type->tp_name, "array.array") == 0) { if (PyLong_CheckExact(sub)) { - return SPEC_FAIL_ARRAY_INT; + return SPEC_FAIL_SUBSCR_ARRAY_INT; } if (PySlice_Check(sub)) { - return SPEC_FAIL_ARRAY_SLICE; + return SPEC_FAIL_SUBSCR_ARRAY_SLICE; } return SPEC_FAIL_OTHER; } else if (container_type->tp_as_buffer) { if (PyLong_CheckExact(sub)) { - return SPEC_FAIL_BUFFER_INT; + return SPEC_FAIL_SUBSCR_BUFFER_INT; } if (PySlice_Check(sub)) { - return SPEC_FAIL_BUFFER_SLICE; + return SPEC_FAIL_SUBSCR_BUFFER_SLICE; } return SPEC_FAIL_OTHER; } else if (container_type->tp_as_sequence) { if (PyLong_CheckExact(sub) && container_type->tp_as_sequence->sq_item) { - return SPEC_FAIL_SEQUENCE_INT; + return SPEC_FAIL_SUBSCR_SEQUENCE_INT; } } return SPEC_FAIL_OTHER; @@ -1224,10 +1235,10 @@ static int function_kind(PyCodeObject *code) { int flags = code->co_flags; if ((flags & (CO_VARKEYWORDS | CO_VARARGS)) || code->co_kwonlyargcount) { - return SPEC_FAIL_COMPLEX_PARAMETERS; + return SPEC_FAIL_CALL_COMPLEX_PARAMETERS; } if ((flags & CO_OPTIMIZED) == 0) { - return SPEC_FAIL_CO_NOT_OPTIMIZED; + return SPEC_FAIL_CALL_CO_NOT_OPTIMIZED; } return SIMPLE_FUNCTION; } @@ -1244,7 +1255,7 @@ _Py_Specialize_BinarySubscr( goto success; } SPECIALIZATION_FAIL(BINARY_SUBSCR, - PySlice_Check(sub) ? SPEC_FAIL_LIST_SLICE : SPEC_FAIL_OTHER); + PySlice_Check(sub) ? SPEC_FAIL_SUBSCR_LIST_SLICE : SPEC_FAIL_OTHER); goto fail; } if (container_type == &PyTuple_Type) { @@ -1253,7 +1264,7 @@ _Py_Specialize_BinarySubscr( goto success; } SPECIALIZATION_FAIL(BINARY_SUBSCR, - PySlice_Check(sub) ? SPEC_FAIL_TUPLE_SLICE : SPEC_FAIL_OTHER); + PySlice_Check(sub) ? SPEC_FAIL_SUBSCR_TUPLE_SLICE : SPEC_FAIL_OTHER); goto fail; } if (container_type == &PyDict_Type) { @@ -1319,7 +1330,7 @@ _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *ins } } else if (PySlice_Check(sub)) { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_LIST_SLICE); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_LIST_SLICE); goto fail; } else { @@ -1336,7 +1347,7 @@ _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *ins PyMappingMethods *as_mapping = container_type->tp_as_mapping; if (as_mapping && (as_mapping->mp_ass_subscript == PyDict_Type.tp_as_mapping->mp_ass_subscript)) { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_DICT_SUBCLASS_NO_OVERRIDE); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_DICT_SUBCLASS_NO_OVERRIDE); goto fail; } if (PyObject_CheckBuffer(container)) { @@ -1345,10 +1356,10 @@ _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *ins } else if (strcmp(container_type->tp_name, "array.array") == 0) { if (PyLong_CheckExact(sub)) { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_ARRAY_INT); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_ARRAY_INT); } else if (PySlice_Check(sub)) { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_ARRAY_SLICE); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_ARRAY_SLICE); } else { SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_OTHER); @@ -1356,10 +1367,10 @@ _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *ins } else if (PyByteArray_CheckExact(container)) { if (PyLong_CheckExact(sub)) { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_BYTEARRAY_INT); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_BYTEARRAY_INT); } else if (PySlice_Check(sub)) { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_BYTEARRAY_SLICE); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_BYTEARRAY_SLICE); } else { SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_OTHER); @@ -1367,10 +1378,10 @@ _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *ins } else { if (PyLong_CheckExact(sub)) { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_BUFFER_INT); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_BUFFER_INT); } else if (PySlice_Check(sub)) { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_BUFFER_SLICE); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_BUFFER_SLICE); } else { SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_OTHER); @@ -1384,10 +1395,10 @@ _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *ins PyCodeObject *code = (PyCodeObject *)func->func_code; int kind = function_kind(code); if (kind == SIMPLE_FUNCTION) { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_PY_SIMPLE); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_PY_SIMPLE); } else { - SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_PY_OTHER); + SPECIALIZATION_FAIL(STORE_SUBSCR, SPEC_FAIL_SUBSCR_PY_OTHER); } goto fail; } @@ -1411,7 +1422,7 @@ specialize_class_call( { PyTypeObject *tp = _PyType_CAST(callable); if (tp->tp_new == PyBaseObject_Type.tp_new) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_PYTHON_CLASS); + SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_PYTHON_CLASS); return -1; } if (tp->tp_flags & Py_TPFLAGS_IMMUTABLETYPE) { @@ -1434,10 +1445,10 @@ specialize_class_call( return 0; } SPECIALIZATION_FAIL(CALL, tp == &PyUnicode_Type ? - SPEC_FAIL_CALL_STR : SPEC_FAIL_CLASS_NO_VECTORCALL); + SPEC_FAIL_CALL_STR : SPEC_FAIL_CALL_CLASS_NO_VECTORCALL); return -1; } - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CLASS_MUTABLE); + SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_CLASS_MUTABLE); return -1; } @@ -1448,19 +1459,19 @@ builtin_call_fail_kind(int ml_flags) switch (ml_flags & (METH_VARARGS | METH_FASTCALL | METH_NOARGS | METH_O | METH_KEYWORDS | METH_METHOD)) { case METH_VARARGS: - return SPEC_FAIL_PYCFUNCTION; + return SPEC_FAIL_CALL_PYCFUNCTION; case METH_VARARGS | METH_KEYWORDS: - return SPEC_FAIL_PYCFUNCTION_WITH_KEYWORDS; + return SPEC_FAIL_CALL_PYCFUNCTION_WITH_KEYWORDS; case METH_FASTCALL | METH_KEYWORDS: - return SPEC_FAIL_PYCFUNCTION_FAST_WITH_KEYWORDS; + return SPEC_FAIL_CALL_PYCFUNCTION_FAST_WITH_KEYWORDS; case METH_NOARGS: - return SPEC_FAIL_PYCFUNCTION_NOARGS; + return SPEC_FAIL_CALL_PYCFUNCTION_NOARGS; /* This case should never happen with PyCFunctionObject -- only PyMethodObject. See zlib.compressobj()'s methods for an example. */ case METH_METHOD | METH_FASTCALL | METH_KEYWORDS: default: - return SPEC_FAIL_BAD_CALL_FLAGS; + return SPEC_FAIL_CALL_BAD_CALL_FLAGS; } } #endif @@ -1473,7 +1484,7 @@ specialize_method_descriptor( int nargs, PyObject *kwnames, SpecializedCacheEntry *cache) { if (kwnames) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_KWNAMES); + SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_KWNAMES); return -1; } if (_list_append == NULL) { @@ -1528,7 +1539,7 @@ specialize_py_call( PyCodeObject *code = (PyCodeObject *)func->func_code; int kind = function_kind(code); if (kwnames) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_KWNAMES); + SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_KWNAMES); return -1; } if (kind != SIMPLE_FUNCTION) { @@ -1585,7 +1596,7 @@ specialize_c_call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, METH_KEYWORDS | METH_METHOD)) { case METH_O: { if (kwnames) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_KWNAMES); + SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_KWNAMES); return -1; } if (nargs != 1) { @@ -1606,7 +1617,7 @@ specialize_c_call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, } case METH_FASTCALL: { if (kwnames) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_KWNAMES); + SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_KWNAMES); return -1; } if (nargs == 2) { @@ -1641,26 +1652,23 @@ static int call_fail_kind(PyObject *callable) { if (PyInstanceMethod_Check(callable)) { - return SPEC_FAIL_METHOD; + return SPEC_FAIL_CALL_INSTANCE_METHOD; } else if (PyMethod_Check(callable)) { - return SPEC_FAIL_METHOD; + return SPEC_FAIL_CALL_BOUND_METHOD; } // builtin method else if (PyCMethod_Check(callable)) { - return SPEC_FAIL_METHOD; + return SPEC_FAIL_CALL_CMETHOD; } else if (PyType_Check(callable)) { - return SPEC_FAIL_CLASS; + return SPEC_FAIL_CALL_CLASS; } else if (Py_TYPE(callable) == &PyWrapperDescr_Type) { - return SPEC_FAIL_OPERATOR_WRAPPER; + return SPEC_FAIL_CALL_OPERATOR_WRAPPER; } else if (Py_TYPE(callable) == &_PyMethodWrapper_Type) { - return SPEC_FAIL_METHOD_WRAPPER; - } - else if (Py_TYPE(callable) == &PyMethod_Type) { - return SPEC_FAIL_BOUND_METHOD; + return SPEC_FAIL_CALL_METHOD_WRAPPER; } return SPEC_FAIL_OTHER; } @@ -1716,7 +1724,7 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, case NB_ADD: case NB_INPLACE_ADD: if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { - SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_DIFFERENT_TYPES); + SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_BINARY_OP_DIFFERENT_TYPES); goto failure; } if (PyUnicode_CheckExact(lhs)) { @@ -1742,7 +1750,7 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, case NB_MULTIPLY: case NB_INPLACE_MULTIPLY: if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { - SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_DIFFERENT_TYPES); + SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_BINARY_OP_DIFFERENT_TYPES); goto failure; } if (PyLong_CheckExact(lhs)) { @@ -1759,7 +1767,7 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, case NB_SUBTRACT: case NB_INPLACE_SUBTRACT: if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { - SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_DIFFERENT_TYPES); + SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_BINARY_OP_DIFFERENT_TYPES); goto failure; } if (PyLong_CheckExact(lhs)) { @@ -1796,30 +1804,30 @@ compare_op_fail_kind(PyObject *lhs, PyObject *rhs) { if (Py_TYPE(lhs) != Py_TYPE(rhs)) { if (PyFloat_CheckExact(lhs) && PyLong_CheckExact(rhs)) { - return SPEC_FAIL_COMPARE_FLOAT_LONG; + return SPEC_FAIL_COMPARE_OP_FLOAT_LONG; } if (PyLong_CheckExact(lhs) && PyFloat_CheckExact(rhs)) { - return SPEC_FAIL_COMPARE_LONG_FLOAT; + return SPEC_FAIL_COMPARE_OP_LONG_FLOAT; } - return SPEC_FAIL_DIFFERENT_TYPES; + return SPEC_FAIL_COMPARE_OP_DIFFERENT_TYPES; } if (PyBytes_CheckExact(lhs)) { - return SPEC_FAIL_COMPARE_BYTES; + return SPEC_FAIL_COMPARE_OP_BYTES; } if (PyTuple_CheckExact(lhs)) { - return SPEC_FAIL_COMPARE_TUPLE; + return SPEC_FAIL_COMPARE_OP_TUPLE; } if (PyList_CheckExact(lhs)) { - return SPEC_FAIL_COMPARE_LIST; + return SPEC_FAIL_COMPARE_OP_LIST; } if (PySet_CheckExact(lhs) || PyFrozenSet_CheckExact(lhs)) { - return SPEC_FAIL_COMPARE_SET; + return SPEC_FAIL_COMPARE_OP_SET; } if (PyBool_Check(lhs)) { - return SPEC_FAIL_COMPARE_BOOL; + return SPEC_FAIL_COMPARE_OP_BOOL; } if (Py_TYPE(lhs)->tp_richcompare == PyBaseObject_Type.tp_richcompare) { - return SPEC_FAIL_COMPARE_BASEOBJECT; + return SPEC_FAIL_COMPARE_OP_BASEOBJECT; } return SPEC_FAIL_OTHER; } @@ -1847,7 +1855,7 @@ _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, int next_opcode = _Py_OPCODE(instr[1]); if (next_opcode != POP_JUMP_IF_FALSE && next_opcode != POP_JUMP_IF_TRUE) { // Can't ever combine, so don't don't bother being adaptive. - SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_NOT_FOLLOWED_BY_COND_JUMP); + SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_COMPARE_OP_NOT_FOLLOWED_BY_COND_JUMP); *instr = _Py_MAKECODEUNIT(COMPARE_OP, adaptive->original_oparg); goto failure; } @@ -1872,13 +1880,13 @@ _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, goto success; } else { - SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_BIG_INT); + SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_COMPARE_OP_BIG_INT); goto failure; } } if (PyUnicode_CheckExact(lhs)) { if (op != Py_EQ && op != Py_NE) { - SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_STRING_COMPARE); + SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_COMPARE_OP_STRING); goto failure; } else { @@ -1901,48 +1909,48 @@ int _PySpecialization_ClassifyIterator(PyObject *iter) { if (PyGen_CheckExact(iter)) { - return SPEC_FAIL_ITER_GENERATOR; + return SPEC_FAIL_FOR_ITER_GENERATOR; } if (PyCoro_CheckExact(iter)) { - return SPEC_FAIL_ITER_COROUTINE; + return SPEC_FAIL_FOR_ITER_COROUTINE; } if (PyAsyncGen_CheckExact(iter)) { - return SPEC_FAIL_ITER_ASYNC_GENERATOR; + return SPEC_FAIL_FOR_ITER_ASYNC_GENERATOR; } PyTypeObject *t = _Py_TYPE(iter); if (t == &PyListIter_Type) { - return SPEC_FAIL_ITER_LIST; + return SPEC_FAIL_FOR_ITER_LIST; } if (t == &PyTupleIter_Type) { - return SPEC_FAIL_ITER_TUPLE; + return SPEC_FAIL_FOR_ITER_TUPLE; } if (t == &PyDictIterKey_Type) { - return SPEC_FAIL_ITER_DICT_KEYS; + return SPEC_FAIL_FOR_ITER_DICT_KEYS; } if (t == &PyDictIterValue_Type) { - return SPEC_FAIL_ITER_DICT_VALUES; + return SPEC_FAIL_FOR_ITER_DICT_VALUES; } if (t == &PyDictIterItem_Type) { - return SPEC_FAIL_ITER_DICT_ITEMS; + return SPEC_FAIL_FOR_ITER_DICT_ITEMS; } if (t == &PySetIter_Type) { - return SPEC_FAIL_ITER_SET; + return SPEC_FAIL_FOR_ITER_SET; } if (t == &PyUnicodeIter_Type) { - return SPEC_FAIL_ITER_STRING; + return SPEC_FAIL_FOR_ITER_STRING; } if (t == &PyBytesIter_Type) { - return SPEC_FAIL_ITER_BYTES; + return SPEC_FAIL_FOR_ITER_BYTES; } if (t == &PyRangeIter_Type) { - return SPEC_FAIL_ITER_RANGE; + return SPEC_FAIL_FOR_ITER_RANGE; } if (t == &PyEnum_Type) { - return SPEC_FAIL_ITER_ENUMERATE; + return SPEC_FAIL_FOR_ITER_ENUMERATE; } if (strncmp(t->tp_name, "itertools", 8) == 0) { - return SPEC_FAIL_ITER_ITERTOOLS; + return SPEC_FAIL_FOR_ITER_ITERTOOLS; } return SPEC_FAIL_OTHER; } @@ -1951,10 +1959,10 @@ int _PySpecialization_ClassifySequence(PyObject *seq) { if (PyTuple_CheckExact(seq)) { - return SPEC_FAIL_TUPLE; + return SPEC_FAIL_UNPACK_SEQUENCE_TUPLE; } if (PyList_CheckExact(seq)) { - return SPEC_FAIL_LIST; + return SPEC_FAIL_UNPACK_SEQUENCE_LIST; } return SPEC_FAIL_OTHER; } diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index 1271c194e96a1..da0bab2390f3d 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -5,6 +5,7 @@ import collections import os.path import opcode +from datetime import date if os.name == "nt": DEFAULT_DIR = "c:\\temp\\py_stats\\" @@ -24,37 +25,53 @@ TOTAL = "specialization.deferred", "specialization.hit", "specialization.miss", "execution_count" -def print_specialization_stats(name, family_stats): +def print_specialization_stats(name, family_stats, defines): if "specializable" not in family_stats: return total = sum(family_stats.get(kind, 0) for kind in TOTAL) if total == 0: return - print(name+":") - for key in sorted(family_stats): - if key.startswith("specialization.failure_kinds"): - continue - if key.startswith("specialization."): + with Section(name, 3, f"specialization stats for {name} family"): + rows = [] + for key in sorted(family_stats): + if key.startswith("specialization.failure_kinds"): + continue + if key.startswith("specialization."): + label = key[len("specialization."):] + elif key == "execution_count": + label = "unquickened" + else: + label = key + if key not in ("specialization.success", "specialization.failure", "specializable"): + rows.append((f"{label:>12}", f"{family_stats[key]:>12}", f"{100*family_stats[key]/total:0.1f}%")) + emit_table(("Kind", "Count", "Ratio"), rows) + print_title("Specialization attempts", 4) + total_attempts = 0 + for key in ("specialization.success", "specialization.failure"): + total_attempts += family_stats.get(key, 0) + rows = [] + for key in ("specialization.success", "specialization.failure"): label = key[len("specialization."):] - elif key == "execution_count": - label = "unquickened" - if key not in ("specialization.success", "specialization.failure"): - print(f"{label:>12}:{family_stats[key]:>12} {100*family_stats[key]/total:0.1f}%") - for key in ("specialization.success", "specialization.failure"): - label = key[len("specialization."):] - print(f" {label}:{family_stats.get(key, 0):>12}") - total_failures = family_stats.get("specialization.failure", 0) - failure_kinds = [ 0 ] * 30 - for key in family_stats: - if not key.startswith("specialization.failure_kind"): - continue - _, index = key[:-1].split("[") - index = int(index) - failure_kinds[index] = family_stats[key] - for index, value in enumerate(failure_kinds): - if not value: - continue - print(f" kind {index:>2}: {value:>8} {100*value/total_failures:0.1f}%") + label = label[0].upper() + label[1:] + val = family_stats.get(key, 0) + rows.append((label, val, f"{100*val/total_attempts:0.1f}%")) + emit_table(("", "Count", "Ratio"), rows) + total_failures = family_stats.get("specialization.failure", 0) + failure_kinds = [ 0 ] * 30 + for key in family_stats: + if not key.startswith("specialization.failure_kind"): + continue + _, index = key[:-1].split("[") + index = int(index) + failure_kinds[index] = family_stats[key] + failures = [(value, index) for (index, value) in enumerate(failure_kinds)] + failures.sort(reverse=True) + rows = [] + for value, index in failures: + if not value: + continue + rows.append((kind_to_text(index, defines, name), value, f"{100*value/total_failures:0.1f}%")) + emit_table(("Failure kind", "Count", "Ratio"), rows) def gather_stats(): stats = collections.Counter() @@ -76,6 +93,31 @@ def extract_opcode_stats(stats): opcode_stats[int(n)][rest.strip(".")] = value return opcode_stats +def parse_kinds(spec_src): + defines = collections.defaultdict(list) + for line in spec_src: + line = line.strip() + if not line.startswith("#define SPEC_FAIL_"): + continue + line = line[len("#define SPEC_FAIL_"):] + name, val = line.split() + defines[int(val.strip())].append(name.strip()) + return defines + +def pretty(defname): + return defname.replace("_", " ").lower() + +def kind_to_text(kind, defines, opname): + if kind < 7: + return pretty(defines[kind][0]) + if opname.endswith("ATTR"): + opname = "ATTR" + if opname.endswith("SUBSCR"): + opname = "SUBSCR" + for name in defines[kind]: + if name.startswith(opname): + return pretty(name[len(opname)+1:]) + return "kind " + str(kind) def categorized_counts(opcode_stats): basic = 0 @@ -104,57 +146,131 @@ def categorized_counts(opcode_stats): basic += count return basic, not_specialized, specialized +def print_title(name, level=2): + print("#"*level, name) + print() + +class Section: + + def __init__(self, title, level=2, summary=None): + self.title = title + self.level = level + if summary is None: + self.summary = title.lower() + else: + self.summary = summary + + def __enter__(self): + print_title(self.title, self.level) + print("
") + print("", self.summary, "") + print() + return self + + def __exit__(*args): + print() + print("
") + print() + +def emit_table(header, rows): + width = len(header) + print("|", " | ".join(header), "|") + print("|", " | ".join(["---"]*width), "|") + for row in rows: + if width is not None and len(row) != width: + raise ValueError("Wrong number of elements in row '" + str(rows) + "'") + print("|", " | ".join(str(i) for i in row), "|") + print() + +def emit_execution_counts(opcode_stats, total): + with Section("Execution counts", summary="execution counts for all instructions"): + counts = [] + for i, opcode_stat in enumerate(opcode_stats): + if "execution_count" in opcode_stat: + count = opcode_stat['execution_count'] + miss = 0 + if "specializable" not in opcode_stat: + miss = opcode_stat.get("specialization.miss") + counts.append((count, opname[i], miss)) + counts.sort(reverse=True) + cumulative = 0 + rows = [] + for (count, name, miss) in counts: + cumulative += count + if miss: + miss = f"{100*miss/count:0.1f}%" + else: + miss = "" + rows.append((name, count, f"{100*count/total:0.1f}%", + f"{100*cumulative/total:0.1f}%", miss)) + emit_table( + ("Name", "Count", "Self", "Cumulative", "Miss ratio"), + rows + ) + + +def emit_specialization_stats(opcode_stats): + spec_path = os.path.join(os.path.dirname(__file__), "../../Python/specialize.c") + with open(spec_path) as spec_src: + defines = parse_kinds(spec_src) + with Section("Specialization stats", summary="specialization stats by family"): + for i, opcode_stat in enumerate(opcode_stats): + name = opname[i] + print_specialization_stats(name, opcode_stat, defines) + +def emit_specialization_overview(opcode_stats, total): + basic, not_specialized, specialized = categorized_counts(opcode_stats) + with Section("Specialization effectiveness"): + emit_table(("Instructions", "Count", "Ratio"), ( + ("Basic", basic, f"{basic*100/total:0.1f}%"), + ("Not specialized", not_specialized, f"{not_specialized*100/total:0.1f}%"), + ("Specialized", specialized, f"{specialized*100/total:0.1f}%"), + )) + +def emit_call_stats(stats): + with Section("Call stats", summary="Inlined calls and frame stats"): + total = 0 + for key, value in stats.items(): + if "Calls to" in key: + total += value + rows = [] + for key, value in stats.items(): + if "Calls to" in key: + rows.append((key, value, f"{100*value/total:0.1f}%")) + for key, value in stats.items(): + if key.startswith("Frame"): + rows.append((key, value, f"{100*value/total:0.1f}%")) + emit_table(("", "Count", "Ratio"), rows) + +def emit_object_stats(stats): + with Section("Object stats", summary="allocations, frees and dict materializatons"): + total = stats.get("Object new values") + rows = [] + for key, value in stats.items(): + if key.startswith("Object"): + if "materialize" in key: + materialize = f"{100*value/total:0.1f}%" + else: + materialize = "" + label = key[6:].strip() + label = label[0].upper() + label[1:] + rows.append((label, value, materialize)) + emit_table(("", "Count", "Ratio"), rows) + def main(): stats = gather_stats() opcode_stats = extract_opcode_stats(stats) - print("Execution counts:") - counts = [] total = 0 for i, opcode_stat in enumerate(opcode_stats): if "execution_count" in opcode_stat: - count = opcode_stat['execution_count'] - miss = 0 - if "specializable" not in opcode_stat: - miss = opcode_stat.get("specialization.miss") - counts.append((count, opname[i], miss)) - total += count - counts.sort(reverse=True) - cummulative = 0 - for (count, name, miss) in counts: - cummulative += count - print(f"{name}: {count} {100*count/total:0.1f}% {100*cummulative/total:0.1f}%") - if miss: - print(f" Misses: {miss} {100*miss/count:0.1f}%") - print("Specialization stats:") - for i, opcode_stat in enumerate(opcode_stats): - name = opname[i] - print_specialization_stats(name, opcode_stat) - basic, not_specialized, specialized = categorized_counts(opcode_stats) - print("Specialization effectiveness:") - print(f" Base instructions {basic} {basic*100/total:0.1f}%") - print(f" Not specialized {not_specialized} {not_specialized*100/total:0.1f}%") - print(f" Specialized {specialized} {specialized*100/total:0.1f}%") - print("Call stats:") - total = 0 - for key, value in stats.items(): - if "Calls to" in key: - total += value - for key, value in stats.items(): - if "Calls to" in key: - print(f" {key}: {value} {100*value/total:0.1f}%") - for key, value in stats.items(): - if key.startswith("Frame"): - print(f" {key}: {value} {100*value/total:0.1f}%") - print("Object stats:") - total = stats.get("Object new values") - for key, value in stats.items(): - if key.startswith("Object"): - if "materialize" in key: - print(f" {key}: {value} {100*value/total:0.1f}%") - else: - print(f" {key}: {value}") - total = 0 - + total += opcode_stat['execution_count'] + emit_execution_counts(opcode_stats, total) + emit_specialization_stats(opcode_stats) + emit_specialization_overview(opcode_stats, total) + emit_call_stats(stats) + emit_object_stats(stats) + print("---") + print("Stats gathered on:", date.today()) if __name__ == "__main__": main() From webhook-mailer at python.org Wed Feb 9 09:23:21 2022 From: webhook-mailer at python.org (corona10) Date: Wed, 09 Feb 2022 14:23:21 -0000 Subject: [Python-checkins] bpo-46685: cover `TypeError` of `ForwardRef(1)` in `test_typing` (GH-31223) Message-ID: https://github.com/python/cpython/commit/d2d1d49eaccaa83eb8873ba15f2fc9562143bc56 commit: d2d1d49eaccaa83eb8873ba15f2fc9562143bc56 branch: main author: Nikita Sobolev committer: corona10 date: 2022-02-09T23:23:07+09:00 summary: bpo-46685: cover `TypeError` of `ForwardRef(1)` in `test_typing` (GH-31223) files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 78e58928332f0..6e2a2b1978ab4 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -2792,6 +2792,10 @@ def test_forwardref_subclass_type_error(self): with self.assertRaises(TypeError): issubclass(int, fr) + def test_forwardref_only_str_arg(self): + with self.assertRaises(TypeError): + typing.ForwardRef(1) # only `str` type is allowed + def test_forward_equality(self): fr = typing.ForwardRef('int') self.assertEqual(fr, typing.ForwardRef('int')) From webhook-mailer at python.org Wed Feb 9 10:12:31 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 09 Feb 2022 15:12:31 -0000 Subject: [Python-checkins] bpo-46685: cover `TypeError` of `ForwardRef(1)` in `test_typing` (GH-31223) Message-ID: https://github.com/python/cpython/commit/d29bbc22b08551afce24e933f43bfcb6e980cb08 commit: d29bbc22b08551afce24e933f43bfcb6e980cb08 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-09T07:12:17-08:00 summary: bpo-46685: cover `TypeError` of `ForwardRef(1)` in `test_typing` (GH-31223) (cherry picked from commit d2d1d49eaccaa83eb8873ba15f2fc9562143bc56) Co-authored-by: Nikita Sobolev files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 62762418b3f4a..2842e28c5ab59 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -2650,6 +2650,10 @@ def test_forwardref_subclass_type_error(self): with self.assertRaises(TypeError): issubclass(int, fr) + def test_forwardref_only_str_arg(self): + with self.assertRaises(TypeError): + typing.ForwardRef(1) # only `str` type is allowed + def test_forward_equality(self): fr = typing.ForwardRef('int') self.assertEqual(fr, typing.ForwardRef('int')) From webhook-mailer at python.org Wed Feb 9 10:12:36 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 09 Feb 2022 15:12:36 -0000 Subject: [Python-checkins] bpo-46685: cover `TypeError` of `ForwardRef(1)` in `test_typing` (GH-31223) Message-ID: https://github.com/python/cpython/commit/bde3765a3fd21e2ecf595a57998b285e3045f744 commit: bde3765a3fd21e2ecf595a57998b285e3045f744 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-09T07:12:32-08:00 summary: bpo-46685: cover `TypeError` of `ForwardRef(1)` in `test_typing` (GH-31223) (cherry picked from commit d2d1d49eaccaa83eb8873ba15f2fc9562143bc56) Co-authored-by: Nikita Sobolev files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 27ec5edd92b7a..641527329501f 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -2455,6 +2455,10 @@ def test_forwardref_subclass_type_error(self): with self.assertRaises(TypeError): issubclass(int, fr) + def test_forwardref_only_str_arg(self): + with self.assertRaises(TypeError): + typing.ForwardRef(1) # only `str` type is allowed + def test_forward_equality(self): fr = typing.ForwardRef('int') self.assertEqual(fr, typing.ForwardRef('int')) From webhook-mailer at python.org Wed Feb 9 11:19:26 2022 From: webhook-mailer at python.org (gvanrossum) Date: Wed, 09 Feb 2022 16:19:26 -0000 Subject: [Python-checkins] bpo-44289: Keep argument file object's current position in tarfile.is_tarfile (GH-26488) Message-ID: https://github.com/python/cpython/commit/128ab092cad984b73a117f58fa0e9b4105051a04 commit: 128ab092cad984b73a117f58fa0e9b4105051a04 branch: main author: Andrzej Mateja committer: gvanrossum date: 2022-02-09T08:19:16-08:00 summary: bpo-44289: Keep argument file object's current position in tarfile.is_tarfile (GH-26488) files: A Misc/NEWS.d/next/Library/2021-06-02-19-47-46.bpo-44289.xC5kuV.rst M Lib/tarfile.py M Lib/test/test_tarfile.py diff --git a/Lib/tarfile.py b/Lib/tarfile.py index e187da2b1994a..e795100158748 100755 --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -2493,7 +2493,9 @@ def is_tarfile(name): """ try: if hasattr(name, "read"): + pos = name.tell() t = open(fileobj=name) + name.seek(pos) else: t = open(name) t.close() diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py index 1357df57eb179..66c1931451748 100644 --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -375,6 +375,18 @@ def test_is_tarfile_valid(self): with open(self.tarname, "rb") as fobj: self.assertTrue(tarfile.is_tarfile(io.BytesIO(fobj.read()))) + def test_is_tarfile_keeps_position(self): + # Test for issue44289: tarfile.is_tarfile() modifies + # file object's current position + with open(self.tarname, "rb") as fobj: + tarfile.is_tarfile(fobj) + self.assertEqual(fobj.tell(), 0) + + with open(self.tarname, "rb") as fobj: + file_like = io.BytesIO(fobj.read()) + tarfile.is_tarfile(file_like) + self.assertEqual(file_like.tell(), 0) + def test_empty_tarfile(self): # Test for issue6123: Allow opening empty archives. # This test checks if tarfile.open() is able to open an empty tar diff --git a/Misc/NEWS.d/next/Library/2021-06-02-19-47-46.bpo-44289.xC5kuV.rst b/Misc/NEWS.d/next/Library/2021-06-02-19-47-46.bpo-44289.xC5kuV.rst new file mode 100644 index 0000000000000..164138f47ae3b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-06-02-19-47-46.bpo-44289.xC5kuV.rst @@ -0,0 +1 @@ +Fix an issue with :meth:`~tarfile.is_tarfile` method when using *fileobj* argument: position in the *fileobj* was advanced forward which made it unreadable with :meth:`tarfile.TarFile.open`. From webhook-mailer at python.org Wed Feb 9 11:52:55 2022 From: webhook-mailer at python.org (gvanrossum) Date: Wed, 09 Feb 2022 16:52:55 -0000 Subject: [Python-checkins] bpo-46430: Intern strings in deep-frozen modules (GH-30683) Message-ID: https://github.com/python/cpython/commit/c0a5ebeb1239020f2ecc199053bb1a70d78841a1 commit: c0a5ebeb1239020f2ecc199053bb1a70d78841a1 branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: gvanrossum date: 2022-02-09T08:52:42-08:00 summary: bpo-46430: Intern strings in deep-frozen modules (GH-30683) files: A Misc/NEWS.d/next/Build/2022-01-19-11-08-32.bpo-46430.k403m_.rst M Include/internal/pycore_code.h M Objects/codeobject.c M Tools/scripts/deepfreeze.py diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index 3897ea0ab6a1a..2d8fe20e1a6f0 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -279,6 +279,8 @@ void _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, /* Deallocator function for static codeobjects used in deepfreeze.py */ void _PyStaticCode_Dealloc(PyCodeObject *co); +/* Function to intern strings of codeobjects */ +void _PyStaticCode_InternStrings(PyCodeObject *co); #ifdef Py_STATS diff --git a/Misc/NEWS.d/next/Build/2022-01-19-11-08-32.bpo-46430.k403m_.rst b/Misc/NEWS.d/next/Build/2022-01-19-11-08-32.bpo-46430.k403m_.rst new file mode 100644 index 0000000000000..2929c5187e1d6 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2022-01-19-11-08-32.bpo-46430.k403m_.rst @@ -0,0 +1 @@ +Intern strings in deep-frozen modules. Patch by Kumar Aditya. \ No newline at end of file diff --git a/Objects/codeobject.c b/Objects/codeobject.c index bb8ffa794a96d..efb51464bd87a 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -1924,3 +1924,15 @@ _PyStaticCode_Dealloc(PyCodeObject *co) co->co_weakreflist = NULL; } } + +void +_PyStaticCode_InternStrings(PyCodeObject *co) +{ + int res = intern_strings(co->co_names); + assert(res == 0); + res = intern_string_constants(co->co_consts, NULL); + assert(res == 0); + res = intern_strings(co->co_localsplusnames); + assert(res == 0); + (void)res; +} diff --git a/Tools/scripts/deepfreeze.py b/Tools/scripts/deepfreeze.py index 080980f6d0ae5..0edf3af71d993 100644 --- a/Tools/scripts/deepfreeze.py +++ b/Tools/scripts/deepfreeze.py @@ -279,6 +279,7 @@ def generate_code(self, name: str, code: types.CodeType) -> str: self.write(f".co_cellvars = {co_cellvars},") self.write(f".co_freevars = {co_freevars},") self.deallocs.append(f"_PyStaticCode_Dealloc(&{name});") + self.patchups.append(f"_PyStaticCode_InternStrings(&{name});") return f"& {name}.ob_base" def generate_tuple(self, name: str, t: Tuple[object, ...]) -> str: From webhook-mailer at python.org Wed Feb 9 12:06:30 2022 From: webhook-mailer at python.org (vstinner) Date: Wed, 09 Feb 2022 17:06:30 -0000 Subject: [Python-checkins] bpo-45863: tarfile: don't zero out header fields unnecessarily (GH-29693) Message-ID: https://github.com/python/cpython/commit/bf2d44ffb06e8f49aacc6b1c140a6717df5cf897 commit: bf2d44ffb06e8f49aacc6b1c140a6717df5cf897 branch: main author: Joshua Root committer: vstinner date: 2022-02-09T18:06:19+01:00 summary: bpo-45863: tarfile: don't zero out header fields unnecessarily (GH-29693) Numeric fields of type float, notably mtime, can't be represented exactly in the ustar header, so the pax header is used. But it is helpful to set them to the nearest int (i.e. second rather than nanosecond precision mtimes) in the ustar header as well, for the benefit of unarchivers that don't understand the pax header. Add test for tarfile.TarInfo.create_pax_header to confirm correct behaviour. files: A Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst M Lib/tarfile.py M Lib/test/test_tarfile.py diff --git a/Lib/tarfile.py b/Lib/tarfile.py index e795100158748..8d43d0da7b988 100755 --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -888,15 +888,24 @@ def create_pax_header(self, info, encoding): # Test number fields for values that exceed the field limit or values # that like to be stored as float. for name, digits in (("uid", 8), ("gid", 8), ("size", 12), ("mtime", 12)): - if name in pax_headers: - # The pax header has priority. Avoid overflow. - info[name] = 0 - continue + needs_pax = False val = info[name] - if not 0 <= val < 8 ** (digits - 1) or isinstance(val, float): - pax_headers[name] = str(val) + val_is_float = isinstance(val, float) + val_int = round(val) if val_is_float else val + if not 0 <= val_int < 8 ** (digits - 1): + # Avoid overflow. info[name] = 0 + needs_pax = True + elif val_is_float: + # Put rounded value in ustar header, and full + # precision value in pax header. + info[name] = val_int + needs_pax = True + + # The existing pax header has priority. + if needs_pax and name not in pax_headers: + pax_headers[name] = str(val) # Create a pax extended header if necessary. if pax_headers: diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py index 66c1931451748..0a67bcb0b09d7 100644 --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1911,6 +1911,61 @@ def test_pax_extended_header(self): finally: tar.close() + def test_create_pax_header(self): + # The ustar header should contain values that can be + # represented reasonably, even if a better (e.g. higher + # precision) version is set in the pax header. + # Issue #45863 + + # values that should be kept + t = tarfile.TarInfo() + t.name = "foo" + t.mtime = 1000.1 + t.size = 100 + t.uid = 123 + t.gid = 124 + info = t.get_info() + header = t.create_pax_header(info, encoding="iso8859-1") + self.assertEqual(info['name'], "foo") + # mtime should be rounded to nearest second + self.assertIsInstance(info['mtime'], int) + self.assertEqual(info['mtime'], 1000) + self.assertEqual(info['size'], 100) + self.assertEqual(info['uid'], 123) + self.assertEqual(info['gid'], 124) + self.assertEqual(header, + b'././@PaxHeader' + bytes(86) \ + + b'0000000\x000000000\x000000000\x0000000000020\x0000000000000\x00010205\x00 x' \ + + bytes(100) + b'ustar\x0000'+ bytes(247) \ + + b'16 mtime=1000.1\n' + bytes(496) + b'foo' + bytes(97) \ + + b'0000644\x000000173\x000000174\x0000000000144\x0000000001750\x00006516\x00 0' \ + + bytes(100) + b'ustar\x0000' + bytes(247)) + + # values that should be changed + t = tarfile.TarInfo() + t.name = "foo\u3374" # can't be represented in ascii + t.mtime = 10**10 # too big + t.size = 10**10 # too big + t.uid = 8**8 # too big + t.gid = 8**8+1 # too big + info = t.get_info() + header = t.create_pax_header(info, encoding="iso8859-1") + # name is kept as-is in info but should be added to pax header + self.assertEqual(info['name'], "foo\u3374") + self.assertEqual(info['mtime'], 0) + self.assertEqual(info['size'], 0) + self.assertEqual(info['uid'], 0) + self.assertEqual(info['gid'], 0) + self.assertEqual(header, + b'././@PaxHeader' + bytes(86) \ + + b'0000000\x000000000\x000000000\x0000000000130\x0000000000000\x00010207\x00 x' \ + + bytes(100) + b'ustar\x0000' + bytes(247) \ + + b'15 path=foo\xe3\x8d\xb4\n16 uid=16777216\n' \ + + b'16 gid=16777217\n20 size=10000000000\n' \ + + b'21 mtime=10000000000\n'+ bytes(424) + b'foo?' + bytes(96) \ + + b'0000644\x000000000\x000000000\x0000000000000\x0000000000000\x00006540\x00 0' \ + + bytes(100) + b'ustar\x0000' + bytes(247)) + class UnicodeTest: diff --git a/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst b/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst new file mode 100644 index 0000000000000..3a1335cc77b9d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst @@ -0,0 +1 @@ +When the :mod:`tarfile` module creates a pax format archive, it will put an integer representation of timestamps in the ustar header (if possible) for the benefit of older unarchivers, in addition to the existing full-precision timestamps in the pax extended header. \ No newline at end of file From webhook-mailer at python.org Wed Feb 9 12:31:22 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 09 Feb 2022 17:31:22 -0000 Subject: [Python-checkins] bpo-45863: tarfile: don't zero out header fields unnecessarily (GH-29693) Message-ID: https://github.com/python/cpython/commit/42f87d435ebe4f2834c8fd0b4ce38f0b1627f3ed commit: 42f87d435ebe4f2834c8fd0b4ce38f0b1627f3ed branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-09T09:31:12-08:00 summary: bpo-45863: tarfile: don't zero out header fields unnecessarily (GH-29693) Numeric fields of type float, notably mtime, can't be represented exactly in the ustar header, so the pax header is used. But it is helpful to set them to the nearest int (i.e. second rather than nanosecond precision mtimes) in the ustar header as well, for the benefit of unarchivers that don't understand the pax header. Add test for tarfile.TarInfo.create_pax_header to confirm correct behaviour. (cherry picked from commit bf2d44ffb06e8f49aacc6b1c140a6717df5cf897) Co-authored-by: Joshua Root files: A Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst M Lib/tarfile.py M Lib/test/test_tarfile.py diff --git a/Lib/tarfile.py b/Lib/tarfile.py index e187da2b1994a..6ada9a05db9cb 100755 --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -888,15 +888,24 @@ def create_pax_header(self, info, encoding): # Test number fields for values that exceed the field limit or values # that like to be stored as float. for name, digits in (("uid", 8), ("gid", 8), ("size", 12), ("mtime", 12)): - if name in pax_headers: - # The pax header has priority. Avoid overflow. - info[name] = 0 - continue + needs_pax = False val = info[name] - if not 0 <= val < 8 ** (digits - 1) or isinstance(val, float): - pax_headers[name] = str(val) + val_is_float = isinstance(val, float) + val_int = round(val) if val_is_float else val + if not 0 <= val_int < 8 ** (digits - 1): + # Avoid overflow. info[name] = 0 + needs_pax = True + elif val_is_float: + # Put rounded value in ustar header, and full + # precision value in pax header. + info[name] = val_int + needs_pax = True + + # The existing pax header has priority. + if needs_pax and name not in pax_headers: + pax_headers[name] = str(val) # Create a pax extended header if necessary. if pax_headers: diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py index 1357df57eb179..4bf1ba38f5ea5 100644 --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1899,6 +1899,61 @@ def test_pax_extended_header(self): finally: tar.close() + def test_create_pax_header(self): + # The ustar header should contain values that can be + # represented reasonably, even if a better (e.g. higher + # precision) version is set in the pax header. + # Issue #45863 + + # values that should be kept + t = tarfile.TarInfo() + t.name = "foo" + t.mtime = 1000.1 + t.size = 100 + t.uid = 123 + t.gid = 124 + info = t.get_info() + header = t.create_pax_header(info, encoding="iso8859-1") + self.assertEqual(info['name'], "foo") + # mtime should be rounded to nearest second + self.assertIsInstance(info['mtime'], int) + self.assertEqual(info['mtime'], 1000) + self.assertEqual(info['size'], 100) + self.assertEqual(info['uid'], 123) + self.assertEqual(info['gid'], 124) + self.assertEqual(header, + b'././@PaxHeader' + bytes(86) \ + + b'0000000\x000000000\x000000000\x0000000000020\x0000000000000\x00010205\x00 x' \ + + bytes(100) + b'ustar\x0000'+ bytes(247) \ + + b'16 mtime=1000.1\n' + bytes(496) + b'foo' + bytes(97) \ + + b'0000644\x000000173\x000000174\x0000000000144\x0000000001750\x00006516\x00 0' \ + + bytes(100) + b'ustar\x0000' + bytes(247)) + + # values that should be changed + t = tarfile.TarInfo() + t.name = "foo\u3374" # can't be represented in ascii + t.mtime = 10**10 # too big + t.size = 10**10 # too big + t.uid = 8**8 # too big + t.gid = 8**8+1 # too big + info = t.get_info() + header = t.create_pax_header(info, encoding="iso8859-1") + # name is kept as-is in info but should be added to pax header + self.assertEqual(info['name'], "foo\u3374") + self.assertEqual(info['mtime'], 0) + self.assertEqual(info['size'], 0) + self.assertEqual(info['uid'], 0) + self.assertEqual(info['gid'], 0) + self.assertEqual(header, + b'././@PaxHeader' + bytes(86) \ + + b'0000000\x000000000\x000000000\x0000000000130\x0000000000000\x00010207\x00 x' \ + + bytes(100) + b'ustar\x0000' + bytes(247) \ + + b'15 path=foo\xe3\x8d\xb4\n16 uid=16777216\n' \ + + b'16 gid=16777217\n20 size=10000000000\n' \ + + b'21 mtime=10000000000\n'+ bytes(424) + b'foo?' + bytes(96) \ + + b'0000644\x000000000\x000000000\x0000000000000\x0000000000000\x00006540\x00 0' \ + + bytes(100) + b'ustar\x0000' + bytes(247)) + class UnicodeTest: diff --git a/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst b/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst new file mode 100644 index 0000000000000..3a1335cc77b9d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst @@ -0,0 +1 @@ +When the :mod:`tarfile` module creates a pax format archive, it will put an integer representation of timestamps in the ustar header (if possible) for the benefit of older unarchivers, in addition to the existing full-precision timestamps in the pax extended header. \ No newline at end of file From webhook-mailer at python.org Wed Feb 9 12:36:29 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 09 Feb 2022 17:36:29 -0000 Subject: [Python-checkins] bpo-45863: tarfile: don't zero out header fields unnecessarily (GH-29693) Message-ID: https://github.com/python/cpython/commit/b0517a12179a9fe444ca6ce1f31c307883e81c0c commit: b0517a12179a9fe444ca6ce1f31c307883e81c0c branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-09T09:35:56-08:00 summary: bpo-45863: tarfile: don't zero out header fields unnecessarily (GH-29693) Numeric fields of type float, notably mtime, can't be represented exactly in the ustar header, so the pax header is used. But it is helpful to set them to the nearest int (i.e. second rather than nanosecond precision mtimes) in the ustar header as well, for the benefit of unarchivers that don't understand the pax header. Add test for tarfile.TarInfo.create_pax_header to confirm correct behaviour. (cherry picked from commit bf2d44ffb06e8f49aacc6b1c140a6717df5cf897) Co-authored-by: Joshua Root files: A Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst M Lib/tarfile.py M Lib/test/test_tarfile.py diff --git a/Lib/tarfile.py b/Lib/tarfile.py index a738e25684c48..9438b08ae396e 100755 --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -887,15 +887,24 @@ def create_pax_header(self, info, encoding): # Test number fields for values that exceed the field limit or values # that like to be stored as float. for name, digits in (("uid", 8), ("gid", 8), ("size", 12), ("mtime", 12)): - if name in pax_headers: - # The pax header has priority. Avoid overflow. - info[name] = 0 - continue + needs_pax = False val = info[name] - if not 0 <= val < 8 ** (digits - 1) or isinstance(val, float): - pax_headers[name] = str(val) + val_is_float = isinstance(val, float) + val_int = round(val) if val_is_float else val + if not 0 <= val_int < 8 ** (digits - 1): + # Avoid overflow. info[name] = 0 + needs_pax = True + elif val_is_float: + # Put rounded value in ustar header, and full + # precision value in pax header. + info[name] = val_int + needs_pax = True + + # The existing pax header has priority. + if needs_pax and name not in pax_headers: + pax_headers[name] = str(val) # Create a pax extended header if necessary. if pax_headers: diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py index a4b63ff82b3ae..1f48dbe475472 100644 --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1898,6 +1898,61 @@ def test_pax_extended_header(self): finally: tar.close() + def test_create_pax_header(self): + # The ustar header should contain values that can be + # represented reasonably, even if a better (e.g. higher + # precision) version is set in the pax header. + # Issue #45863 + + # values that should be kept + t = tarfile.TarInfo() + t.name = "foo" + t.mtime = 1000.1 + t.size = 100 + t.uid = 123 + t.gid = 124 + info = t.get_info() + header = t.create_pax_header(info, encoding="iso8859-1") + self.assertEqual(info['name'], "foo") + # mtime should be rounded to nearest second + self.assertIsInstance(info['mtime'], int) + self.assertEqual(info['mtime'], 1000) + self.assertEqual(info['size'], 100) + self.assertEqual(info['uid'], 123) + self.assertEqual(info['gid'], 124) + self.assertEqual(header, + b'././@PaxHeader' + bytes(86) \ + + b'0000000\x000000000\x000000000\x0000000000020\x0000000000000\x00010205\x00 x' \ + + bytes(100) + b'ustar\x0000'+ bytes(247) \ + + b'16 mtime=1000.1\n' + bytes(496) + b'foo' + bytes(97) \ + + b'0000644\x000000173\x000000174\x0000000000144\x0000000001750\x00006516\x00 0' \ + + bytes(100) + b'ustar\x0000' + bytes(247)) + + # values that should be changed + t = tarfile.TarInfo() + t.name = "foo\u3374" # can't be represented in ascii + t.mtime = 10**10 # too big + t.size = 10**10 # too big + t.uid = 8**8 # too big + t.gid = 8**8+1 # too big + info = t.get_info() + header = t.create_pax_header(info, encoding="iso8859-1") + # name is kept as-is in info but should be added to pax header + self.assertEqual(info['name'], "foo\u3374") + self.assertEqual(info['mtime'], 0) + self.assertEqual(info['size'], 0) + self.assertEqual(info['uid'], 0) + self.assertEqual(info['gid'], 0) + self.assertEqual(header, + b'././@PaxHeader' + bytes(86) \ + + b'0000000\x000000000\x000000000\x0000000000130\x0000000000000\x00010207\x00 x' \ + + bytes(100) + b'ustar\x0000' + bytes(247) \ + + b'15 path=foo\xe3\x8d\xb4\n16 uid=16777216\n' \ + + b'16 gid=16777217\n20 size=10000000000\n' \ + + b'21 mtime=10000000000\n'+ bytes(424) + b'foo?' + bytes(96) \ + + b'0000644\x000000000\x000000000\x0000000000000\x0000000000000\x00006540\x00 0' \ + + bytes(100) + b'ustar\x0000' + bytes(247)) + class UnicodeTest: diff --git a/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst b/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst new file mode 100644 index 0000000000000..3a1335cc77b9d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst @@ -0,0 +1 @@ +When the :mod:`tarfile` module creates a pax format archive, it will put an integer representation of timestamps in the ustar header (if possible) for the benefit of older unarchivers, in addition to the existing full-precision timestamps in the pax extended header. \ No newline at end of file From webhook-mailer at python.org Wed Feb 9 13:10:21 2022 From: webhook-mailer at python.org (corona10) Date: Wed, 09 Feb 2022 18:10:21 -0000 Subject: [Python-checkins] bpo-46323: Reduce stack usage of ctypes python callback function. (GH-31224) Message-ID: https://github.com/python/cpython/commit/d18120cd67b4297f78bfc9bf7b36774cf0bf15f2 commit: d18120cd67b4297f78bfc9bf7b36774cf0bf15f2 branch: main author: Dong-hee Na committer: corona10 date: 2022-02-10T03:10:11+09:00 summary: bpo-46323: Reduce stack usage of ctypes python callback function. (GH-31224) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-10-02-29-12.bpo-46323.HK_cs0.rst M Modules/_ctypes/callbacks.c M Modules/_ctypes/callproc.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-10-02-29-12.bpo-46323.HK_cs0.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-10-02-29-12.bpo-46323.HK_cs0.rst new file mode 100644 index 0000000000000..16db7c5eaea71 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-10-02-29-12.bpo-46323.HK_cs0.rst @@ -0,0 +1,3 @@ +:mod:`ctypes` now allocates memory on the stack instead of on the heap +to pass arguments while calling a Python callback function. +Patch by Dong-hee Na. diff --git a/Modules/_ctypes/callbacks.c b/Modules/_ctypes/callbacks.c index f2d9a530e6e38..591b944d76fcb 100644 --- a/Modules/_ctypes/callbacks.c +++ b/Modules/_ctypes/callbacks.c @@ -14,9 +14,18 @@ #include +#ifdef MS_WIN32 +# include +#endif + #include #include "ctypes.h" +#ifdef HAVE_ALLOCA_H +/* AIX needs alloca.h for alloca() */ +#include +#endif + /**************************************************************/ static void @@ -148,7 +157,6 @@ static void _CallPythonObject(void *mem, void **pArgs) { PyObject *result = NULL; - PyObject **args = NULL; Py_ssize_t i = 0, j = 0, nargs = 0; PyObject *error_object = NULL; int *space; @@ -156,24 +164,10 @@ static void _CallPythonObject(void *mem, assert(PyTuple_Check(converters)); nargs = PyTuple_GET_SIZE(converters); - /* Hm. What to return in case of error? - For COM, 0xFFFFFFFF seems better than 0. - */ - if (nargs < 0) { - PrintError("BUG: PySequence_Length"); - goto Done; - } - - PyObject *args_stack[CTYPES_MAX_ARGCOUNT]; - if (nargs <= CTYPES_MAX_ARGCOUNT) { - args = args_stack; - } - else { - args = PyMem_Malloc(nargs * sizeof(PyObject *)); - if (args == NULL) { - PyErr_NoMemory(); - goto Done; - } + assert(nargs <= CTYPES_MAX_ARGCOUNT); + PyObject **args = NULL; + if (nargs > 0) { + args = alloca(nargs * sizeof(PyObject *)); } PyObject **cnvs = PySequence_Fast_ITEMS(converters); @@ -310,9 +304,6 @@ static void _CallPythonObject(void *mem, for (j = 0; j < i; j++) { Py_DECREF(args[j]); } - if (args != args_stack) { - PyMem_Free(args); - } PyGILState_Release(state); } diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index 6dba0ffaa5159..4a6b8ec3ee016 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -1162,11 +1162,7 @@ PyObject *_ctypes_callproc(PPROC pProc, return NULL; } - args = (struct argument *)alloca(sizeof(struct argument) * argcount); - if (!args) { - PyErr_NoMemory(); - return NULL; - } + args = alloca(sizeof(struct argument) * argcount); memset(args, 0, sizeof(struct argument) * argcount); argtype_count = argtypes ? PyTuple_GET_SIZE(argtypes) : 0; #ifdef MS_WIN32 From webhook-mailer at python.org Wed Feb 9 14:31:16 2022 From: webhook-mailer at python.org (brandtbucher) Date: Wed, 09 Feb 2022 19:31:16 -0000 Subject: [Python-checkins] bpo-46528: Check PyMem_Malloc for NULL (GH-30998) Message-ID: https://github.com/python/cpython/commit/46328d8ae6529db916ccaabb9247fb0327ce0c1e commit: 46328d8ae6529db916ccaabb9247fb0327ce0c1e branch: main author: Brandt Bucher committer: brandtbucher date: 2022-02-09T11:31:01-08:00 summary: bpo-46528: Check PyMem_Malloc for NULL (GH-30998) files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index 5f9bcf1dfd962..5fcaa0a994424 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8375,21 +8375,22 @@ fold_tuple_on_constants(struct compiler *c, #define VISITED (-1) // Replace an arbitrary run of SWAPs and NOPs with an optimal one that has the -// same effect. Return the number of instructions that were optimized. +// same effect. static int -swaptimize(basicblock *block, int ix) +swaptimize(basicblock *block, int *ix) { // NOTE: "./python -m test test_patma" serves as a good, quick stress test // for this function. Make sure to blow away cached *.pyc files first! - assert(ix < block->b_iused); - struct instr *instructions = &block->b_instr[ix]; + assert(*ix < block->b_iused); + struct instr *instructions = &block->b_instr[*ix]; // Find the length of the current sequence of SWAPs and NOPs, and record the // maximum depth of the stack manipulations: assert(instructions[0].i_opcode == SWAP); int depth = instructions[0].i_oparg; int len = 0; int more = false; - while (++len < block->b_iused - ix) { + int limit = block->b_iused - *ix; + while (++len < limit) { int opcode = instructions[len].i_opcode; if (opcode == SWAP) { depth = Py_MAX(depth, instructions[len].i_oparg); @@ -8405,6 +8406,10 @@ swaptimize(basicblock *block, int ix) } // Create an array with elements {0, 1, 2, ..., depth - 1}: int *stack = PyMem_Malloc(depth * sizeof(int)); + if (stack == NULL) { + PyErr_NoMemory(); + return -1; + } for (int i = 0; i < depth; i++) { stack[i] = i; } @@ -8462,9 +8467,9 @@ swaptimize(basicblock *block, int ix) while (0 <= current) { instructions[current--].i_opcode = NOP; } - // Done! Return the number of optimized instructions: PyMem_Free(stack); - return len - 1; + *ix += len - 1; + return 0; } // Attempt to eliminate jumps to jumps by updating inst to jump to @@ -8706,7 +8711,9 @@ optimize_basic_block(struct compiler *c, basicblock *bb, PyObject *consts) inst->i_opcode = NOP; break; } - i += swaptimize(bb, i); + if (swaptimize(bb, &i)) { + goto error; + } break; case KW_NAMES: break; From webhook-mailer at python.org Wed Feb 9 15:56:18 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 09 Feb 2022 20:56:18 -0000 Subject: [Python-checkins] bpo-43532: add version added to KW_ONLY (GH-31235) Message-ID: https://github.com/python/cpython/commit/5a3f97291eea96037cceee097ebc00bba44bc9ed commit: 5a3f97291eea96037cceee097ebc00bba44bc9ed branch: main author: Henry Schreiner committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-09T12:56:10-08:00 summary: bpo-43532: add version added to KW_ONLY (GH-31235) Minor missing version note mentioned at the end of (and affected me independently before reading that note). Automerge-Triggered-By: GH:ericvsmith files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 0bac594f05617..0f6985f0ba8c4 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -475,6 +475,8 @@ Module contents In a single dataclass, it is an error to specify more than one field whose type is :const:`KW_ONLY`. + .. versionadded:: 3.10 + .. exception:: FrozenInstanceError Raised when an implicitly defined :meth:`__setattr__` or From webhook-mailer at python.org Wed Feb 9 16:19:02 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 09 Feb 2022 21:19:02 -0000 Subject: [Python-checkins] bpo-43532: add version added to KW_ONLY (GH-31235) Message-ID: https://github.com/python/cpython/commit/7445949a4399ab19fbdd5a0b0aca53a690c3251b commit: 7445949a4399ab19fbdd5a0b0aca53a690c3251b branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-09T13:18:52-08:00 summary: bpo-43532: add version added to KW_ONLY (GH-31235) Minor missing version note mentioned at the end of (and affected me independently before reading that note). Automerge-Triggered-By: GH:ericvsmith (cherry picked from commit 5a3f97291eea96037cceee097ebc00bba44bc9ed) Co-authored-by: Henry Schreiner files: M Doc/library/dataclasses.rst diff --git a/Doc/library/dataclasses.rst b/Doc/library/dataclasses.rst index 6a9863cf56a6d..1fe5bda3b9415 100644 --- a/Doc/library/dataclasses.rst +++ b/Doc/library/dataclasses.rst @@ -475,6 +475,8 @@ Module contents In a single dataclass, it is an error to specify more than one field whose type is :const:`KW_ONLY`. + .. versionadded:: 3.10 + .. exception:: FrozenInstanceError Raised when an implicitly defined :meth:`__setattr__` or From webhook-mailer at python.org Wed Feb 9 18:15:53 2022 From: webhook-mailer at python.org (brandtbucher) Date: Wed, 09 Feb 2022 23:15:53 -0000 Subject: [Python-checkins] bpo-46528: Attempt SWAPs at compile-time (GH-30970) Message-ID: https://github.com/python/cpython/commit/78ae4cc6dc949e8bc39fab25fea5efe983dc0ad1 commit: 78ae4cc6dc949e8bc39fab25fea5efe983dc0ad1 branch: main author: Brandt Bucher committer: brandtbucher date: 2022-02-09T15:15:36-08:00 summary: bpo-46528: Attempt SWAPs at compile-time (GH-30970) files: A Misc/NEWS.d/next/Core and Builtins/2022-01-27-14-20-18.bpo-45828.kzk4fl.rst M Lib/test/test_peepholer.py M Python/compile.c diff --git a/Lib/test/test_peepholer.py b/Lib/test/test_peepholer.py index 2df5883ce7918..6f24b291b00b5 100644 --- a/Lib/test/test_peepholer.py +++ b/Lib/test/test_peepholer.py @@ -1,10 +1,25 @@ import dis from itertools import combinations, product +import textwrap import unittest from test.support.bytecode_helper import BytecodeTestCase +def compile_pattern_with_fast_locals(pattern): + source = textwrap.dedent( + f""" + def f(x): + match x: + case {pattern}: + pass + """ + ) + namespace = {} + exec(source, namespace) + return namespace["f"].__code__ + + def count_instr_recursively(f, opname): count = 0 for instr in dis.get_instructions(f): @@ -580,6 +595,58 @@ def test_format_errors(self): 'not all arguments converted during string formatting'): eval("'%s, %s' % (x, *y)", {'x': 1, 'y': [2, 3]}) + def test_static_swaps_unpack_two(self): + def f(a, b): + a, b = a, b + b, a = a, b + self.assertNotInBytecode(f, "SWAP") + + def test_static_swaps_unpack_three(self): + def f(a, b, c): + a, b, c = a, b, c + a, c, b = a, b, c + b, a, c = a, b, c + b, c, a = a, b, c + c, a, b = a, b, c + c, b, a = a, b, c + self.assertNotInBytecode(f, "SWAP") + + def test_static_swaps_match_mapping(self): + for a, b, c in product("_a", "_b", "_c"): + pattern = f"{{'a': {a}, 'b': {b}, 'c': {c}}}" + with self.subTest(pattern): + code = compile_pattern_with_fast_locals(pattern) + self.assertNotInBytecode(code, "SWAP") + + def test_static_swaps_match_class(self): + forms = [ + "C({}, {}, {})", + "C({}, {}, c={})", + "C({}, b={}, c={})", + "C(a={}, b={}, c={})" + ] + for a, b, c in product("_a", "_b", "_c"): + for form in forms: + pattern = form.format(a, b, c) + with self.subTest(pattern): + code = compile_pattern_with_fast_locals(pattern) + self.assertNotInBytecode(code, "SWAP") + + def test_static_swaps_match_sequence(self): + swaps = {"*_, b, c", "a, *_, c", "a, b, *_"} + forms = ["{}, {}, {}", "{}, {}, *{}", "{}, *{}, {}", "*{}, {}, {}"] + for a, b, c in product("_a", "_b", "_c"): + for form in forms: + pattern = form.format(a, b, c) + with self.subTest(pattern): + code = compile_pattern_with_fast_locals(pattern) + if pattern in swaps: + # If this fails... great! Remove this pattern from swaps + # to prevent regressing on any improvement: + self.assertInBytecode(code, "SWAP") + else: + self.assertNotInBytecode(code, "SWAP") + class TestBuglets(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-01-27-14-20-18.bpo-45828.kzk4fl.rst b/Misc/NEWS.d/next/Core and Builtins/2022-01-27-14-20-18.bpo-45828.kzk4fl.rst new file mode 100644 index 0000000000000..687fef035d66f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-01-27-14-20-18.bpo-45828.kzk4fl.rst @@ -0,0 +1,2 @@ +The bytecode compiler now attempts to apply runtime stack manipulations at +compile-time (whenever it is feasible to do so). diff --git a/Python/compile.c b/Python/compile.c index 5fcaa0a994424..bfe451b8c1047 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8472,6 +8472,72 @@ swaptimize(basicblock *block, int *ix) return 0; } +// This list is pretty small, since it's only okay to reorder opcodes that: +// - can't affect control flow (like jumping or raising exceptions) +// - can't invoke arbitrary code (besides finalizers) +// - only touch the TOS (and pop it when finished) +#define SWAPPABLE(opcode) \ + ((opcode) == STORE_FAST || (opcode) == POP_TOP) + +static int +next_swappable_instruction(basicblock *block, int i, int lineno) +{ + while (++i < block->b_iused) { + struct instr *instruction = &block->b_instr[i]; + if (0 <= lineno && instruction->i_lineno != lineno) { + // Optimizing across this instruction could cause user-visible + // changes in the names bound between line tracing events! + return -1; + } + if (instruction->i_opcode == NOP) { + continue; + } + if (SWAPPABLE(instruction->i_opcode)) { + return i; + } + return -1; + } + return -1; +} + +// Attempt to apply SWAPs statically by swapping *instructions* rather than +// stack items. For example, we can replace SWAP(2), POP_TOP, STORE_FAST(42) +// with the more efficient NOP, STORE_FAST(42), POP_TOP. +static void +apply_static_swaps(basicblock *block, int i) +{ + // SWAPs are to our left, and potential swaperands are to our right: + for (; 0 <= i; i--) { + assert(i < block->b_iused); + struct instr *swap = &block->b_instr[i]; + if (swap->i_opcode != SWAP) { + if (swap->i_opcode == NOP || SWAPPABLE(swap->i_opcode)) { + // Nope, but we know how to handle these. Keep looking: + continue; + } + // We can't reason about what this instruction does. Bail: + return; + } + int j = next_swappable_instruction(block, i, -1); + if (j < 0) { + return; + } + int k = j; + int lineno = block->b_instr[j].i_lineno; + for (int count = swap->i_oparg - 1; 0 < count; count--) { + k = next_swappable_instruction(block, k, lineno); + if (k < 0) { + return; + } + } + // Success! + swap->i_opcode = NOP; + struct instr temp = block->b_instr[j]; + block->b_instr[j] = block->b_instr[k]; + block->b_instr[k] = temp; + } +} + // Attempt to eliminate jumps to jumps by updating inst to jump to // target->i_target using the provided opcode. Return whether or not the // optimization was successful. @@ -8714,6 +8780,7 @@ optimize_basic_block(struct compiler *c, basicblock *bb, PyObject *consts) if (swaptimize(bb, &i)) { goto error; } + apply_static_swaps(bb, i); break; case KW_NAMES: break; From webhook-mailer at python.org Wed Feb 9 19:11:03 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 10 Feb 2022 00:11:03 -0000 Subject: [Python-checkins] bpo-36876: Minor cleanup to c-analyzer "ignored" data.' (gh-31239) Message-ID: https://github.com/python/cpython/commit/cb68788dcadf43b47292bab7816a5ed9efa69730 commit: cb68788dcadf43b47292bab7816a5ed9efa69730 branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-09T17:10:53-07:00 summary: bpo-36876: Minor cleanup to c-analyzer "ignored" data.' (gh-31239) https://bugs.python.org/issue36876 files: M Tools/c-analyzer/cpython/globals-to-fix.tsv M Tools/c-analyzer/cpython/ignored.tsv M Tools/c-analyzer/table-file.py diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 544216ca0b68b..5690481485831 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -255,236 +255,7 @@ Objects/setobject.c - _PySet_Dummy - Objects/sliceobject.c - _Py_EllipsisObject - #----------------------- -# cached PyUnicodeObject - initialized once - -# _Py_IDENTIFIER (global) -Objects/bytesobject.c - PyId___bytes__ - -Objects/classobject.c - PyId___name__ - -Objects/classobject.c - PyId___qualname__ - -Objects/descrobject.c - PyId_getattr - -Objects/fileobject.c - PyId_open - -Objects/frameobject.c - PyId___builtins__ - -Objects/iterobject.c - PyId_iter - -Objects/longobject.c - PyId_big - -Objects/longobject.c - PyId_little - -Objects/moduleobject.c - PyId___annotations__ - -Objects/moduleobject.c - PyId___dict__ - -Objects/moduleobject.c - PyId___dir__ - -Objects/moduleobject.c - PyId___doc__ - -Objects/moduleobject.c - PyId___name__ - -Objects/moduleobject.c - PyId___spec__ - -Objects/object.c - PyId_Py_Repr - -Objects/object.c - PyId___bytes__ - -Objects/object.c - PyId___dir__ - -Objects/object.c - PyId___isabstractmethod__ - -Objects/odictobject.c - PyId_items - -Objects/rangeobject.c - PyId_iter - -Objects/structseq.c - PyId_n_fields - -Objects/structseq.c - PyId_n_sequence_fields - -Objects/structseq.c - PyId_n_unnamed_fields - -Objects/typeobject.c - PyId___abstractmethods__ - -Objects/typeobject.c - PyId___annotations__ - -Objects/typeobject.c - PyId___class__ - -Objects/typeobject.c - PyId___class_getitem__ - -Objects/typeobject.c - PyId___classcell__ - -Objects/typeobject.c - PyId___delitem__ - -Objects/typeobject.c - PyId___dict__ - -Objects/typeobject.c - PyId___doc__ - -Objects/typeobject.c - PyId___getattribute__ - -Objects/typeobject.c - PyId___getitem__ - -Objects/typeobject.c - PyId___hash__ - -Objects/typeobject.c - PyId___init_subclass__ - -Objects/typeobject.c - PyId___len__ - -Objects/typeobject.c - PyId___module__ - -Objects/typeobject.c - PyId___name__ - -Objects/typeobject.c - PyId___new__ - -Objects/typeobject.c - PyId___qualname__ - -Objects/typeobject.c - PyId___set_name__ - -Objects/typeobject.c - PyId___setitem__ - -Objects/typeobject.c - PyId___weakref__ - -Objects/typeobject.c - PyId_builtins - -Objects/typeobject.c - PyId_mro - - -# _Py_IDENTIFIER (local) -Objects/abstract.c PyIter_Send PyId_send - -Objects/abstract.c PyMapping_Items PyId_items - -Objects/abstract.c PyMapping_Keys PyId_keys - -Objects/abstract.c PyMapping_Values PyId_values - -Objects/abstract.c PyNumber_Long PyId___trunc__ - -Objects/abstract.c PyObject_Format PyId___format__ - -Objects/abstract.c PyObject_GetItem PyId___class_getitem__ - -Objects/abstract.c PyObject_LengthHint PyId___length_hint__ - -Objects/abstract.c abstract_get_bases PyId___bases__ - -Objects/abstract.c object_isinstance PyId___class__ - -Objects/abstract.c object_issubclass PyId___subclasscheck__ - -Objects/abstract.c object_recursive_isinstance PyId___instancecheck__ - -Objects/bytearrayobject.c _common_reduce PyId___dict__ - -Objects/bytearrayobject.c bytearrayiter_reduce PyId_iter - -Objects/bytesobject.c striter_reduce PyId_iter - -Objects/classobject.c method_reduce PyId_getattr - -Objects/complexobject.c try_complex_special_method PyId___complex__ - -Objects/descrobject.c calculate_qualname PyId___qualname__ - -Objects/descrobject.c mappingproxy_copy PyId_copy - -Objects/descrobject.c mappingproxy_get PyId_get - -Objects/descrobject.c mappingproxy_items PyId_items - -Objects/descrobject.c mappingproxy_keys PyId_keys - -Objects/descrobject.c mappingproxy_reversed PyId___reversed__ - -Objects/descrobject.c mappingproxy_values PyId_values - -Objects/descrobject.c property_init_impl PyId___doc__ - -Objects/dictobject.c _PyDictView_Intersect PyId_intersection - -Objects/dictobject.c dict_subscript PyId___missing__ - -Objects/dictobject.c dict_update_arg PyId_keys - -Objects/dictobject.c dictitems_xor PyId_items - -Objects/dictobject.c dictiter_reduce PyId_iter - -Objects/dictobject.c dictviews_sub PyId_difference_update - -Objects/dictobject.c dictviews_xor PyId_symmetric_difference_update - -Objects/enumobject.c reversed_new_impl PyId___reversed__ - -Objects/exceptions.c ImportError_getstate PyId_name - -Objects/exceptions.c ImportError_getstate PyId_path - -Objects/fileobject.c PyFile_GetLine PyId_readline - -Objects/fileobject.c PyFile_WriteObject PyId_write - -Objects/fileobject.c PyObject_AsFileDescriptor PyId_fileno - -Objects/funcobject.c PyFunction_NewWithQualName PyId___name__ - -Objects/funcobject.c functools_wraps PyId___annotations__ - -Objects/funcobject.c functools_wraps PyId___doc__ - -Objects/funcobject.c functools_wraps PyId___module__ - -Objects/funcobject.c functools_wraps PyId___name__ - -Objects/funcobject.c functools_wraps PyId___qualname__ - -Objects/genericaliasobject.c _Py_make_parameters PyId___parameters__ - -Objects/genericaliasobject.c ga_repr_item PyId___args__ - -Objects/genericaliasobject.c ga_repr_item PyId___module__ - -Objects/genericaliasobject.c ga_repr_item PyId___origin__ - -Objects/genericaliasobject.c ga_repr_item PyId___qualname__ - -Objects/genericaliasobject.c make_parameters PyId___parameters__ - -Objects/genericaliasobject.c subs_tvars PyId___parameters__ - -Objects/genobject.c PyIter_Send PyId_send - -Objects/genobject.c _gen_throw PyId_throw - -Objects/genobject.c gen_close_iter PyId_close - -Objects/listobject.c listiter_reduce_general PyId_iter - -Objects/listobject.c listiter_reduce_general PyId_reversed - -Objects/methodobject.c meth_get__qualname__ PyId___qualname__ - -Objects/methodobject.c meth_reduce PyId_getattr - -Objects/moduleobject.c PyModule_GetFilenameObject PyId___file__ - -Objects/moduleobject.c _PyModuleSpec_IsInitializing PyId__initializing - -Objects/moduleobject.c _PyModuleSpec_IsUninitializedSubmodule PyId__uninitialized_submodules - -Objects/moduleobject.c module_dir PyId___dict__ - -Objects/moduleobject.c module_dir PyId___dir__ - -Objects/moduleobject.c module_getattro PyId___getattr__ - -Objects/moduleobject.c module_init_dict PyId___loader__ - -Objects/moduleobject.c module_init_dict PyId___package__ - -Objects/object.c _PyObject_FunctionStr PyId___module__ - -Objects/object.c _PyObject_FunctionStr PyId___qualname__ - -Objects/object.c _PyObject_FunctionStr PyId_builtins - -Objects/object.c set_attribute_error_context PyId_name - -Objects/object.c set_attribute_error_context PyId_obj - -Objects/odictobject.c mutablemapping_update_arg PyId_keys - -Objects/odictobject.c odict_reduce PyId___dict__ - -Objects/odictobject.c odictiter_reduce PyId_iter - -Objects/setobject.c set_reduce PyId___dict__ - -Objects/setobject.c setiter_reduce PyId_iter - -Objects/tupleobject.c tupleiter_reduce PyId_iter - -Objects/typeobject.c _PyObject_GetItemsIter PyId_items - -Objects/typeobject.c _PyObject_GetNewArguments PyId___getnewargs__ - -Objects/typeobject.c _PyObject_GetNewArguments PyId___getnewargs_ex__ - -Objects/typeobject.c _PyObject_GetState PyId___getstate__ - -Objects/typeobject.c _PyType_GetSlotNames PyId___slotnames__ - -Objects/typeobject.c _PyType_GetSlotNames PyId__slotnames - -Objects/typeobject.c import_copyreg PyId_copyreg - -Objects/typeobject.c merge_class_dict PyId___bases__ - -Objects/typeobject.c object___reduce_ex___impl PyId___reduce__ - -Objects/typeobject.c overrides_hash PyId___eq__ - -Objects/typeobject.c reduce_newobj PyId___newobj__ - -Objects/typeobject.c reduce_newobj PyId___newobj_ex__ - -Objects/typeobject.c slot_am_aiter PyId___aiter__ - -Objects/typeobject.c slot_am_anext PyId___anext__ - -Objects/typeobject.c slot_am_await PyId___await__ - -Objects/typeobject.c slot_nb_bool PyId___bool__ - -Objects/typeobject.c slot_nb_index PyId___index__ - -Objects/typeobject.c slot_nb_inplace_power PyId___ipow__ - -Objects/typeobject.c slot_nb_power PyId___pow__ - -Objects/typeobject.c slot_sq_contains PyId___contains__ - -Objects/typeobject.c slot_tp_call PyId___call__ - -Objects/typeobject.c slot_tp_descr_get PyId___get__ - -Objects/typeobject.c slot_tp_descr_set PyId___delete__ - -Objects/typeobject.c slot_tp_descr_set PyId___set__ - -Objects/typeobject.c slot_tp_finalize PyId___del__ - -Objects/typeobject.c slot_tp_getattr_hook PyId___getattr__ - -Objects/typeobject.c slot_tp_init PyId___init__ - -Objects/typeobject.c slot_tp_iter PyId___iter__ - -Objects/typeobject.c slot_tp_iternext PyId___next__ - -Objects/typeobject.c slot_tp_repr PyId___repr__ - -Objects/typeobject.c slot_tp_setattro PyId___delattr__ - -Objects/typeobject.c slot_tp_setattro PyId___setattr__ - -Objects/typeobject.c type_new PyId___classcell__ - -Objects/typeobject.c type_new PyId___mro_entries__ - -Objects/typeobject.c type_new PyId___qualname__ - -Objects/typeobject.c type_new PyId___slots__ - -Objects/typeobject.c type_new_get_bases PyId___mro_entries__ - -Objects/typeobject.c type_new_get_slots PyId___slots__ - -Objects/unicodeobject.c unicodeiter_reduce PyId_iter - -Objects/unionobject.c union_repr_item PyId___args__ - -Objects/unionobject.c union_repr_item PyId___module__ - -Objects/unionobject.c union_repr_item PyId___origin__ - -Objects/unionobject.c union_repr_item PyId___qualname__ - -Objects/weakrefobject.c proxy_bytes PyId___bytes__ - -Objects/weakrefobject.c proxy_reversed PyId___reversed__ - -Objects/weakrefobject.c weakref_repr PyId___name__ - -Parser/tokenizer.c fp_setreadl PyId_open - -Parser/tokenizer.c fp_setreadl PyId_readline - - -# _Py_static_string -Objects/typeobject.c - name_op - -Objects/typeobject.c object_new comma_id - -Objects/typeobject.c slot_mp_subscript id - -Objects/typeobject.c slot_nb_add op_id - -Objects/typeobject.c slot_nb_add rop_id - -Objects/typeobject.c slot_nb_subtract op_id - -Objects/typeobject.c slot_nb_subtract rop_id - -Objects/typeobject.c slot_nb_multiply op_id - -Objects/typeobject.c slot_nb_multiply rop_id - -Objects/typeobject.c slot_nb_matrix_multiply op_id - -Objects/typeobject.c slot_nb_matrix_multiply rop_id - -Objects/typeobject.c slot_nb_remainder op_id - -Objects/typeobject.c slot_nb_remainder rop_id - -Objects/typeobject.c slot_nb_divmod op_id - -Objects/typeobject.c slot_nb_divmod rop_id - -Objects/typeobject.c slot_nb_power_binary op_id - -Objects/typeobject.c slot_nb_power_binary rop_id - -Objects/typeobject.c slot_nb_negative id - -Objects/typeobject.c slot_nb_positive id - -Objects/typeobject.c slot_nb_absolute id - -Objects/typeobject.c slot_nb_invert id - -Objects/typeobject.c slot_nb_lshift op_id - -Objects/typeobject.c slot_nb_lshift rop_id - -Objects/typeobject.c slot_nb_rshift op_id - -Objects/typeobject.c slot_nb_rshift rop_id - -Objects/typeobject.c slot_nb_and op_id - -Objects/typeobject.c slot_nb_and rop_id - -Objects/typeobject.c slot_nb_xor op_id - -Objects/typeobject.c slot_nb_xor rop_id - -Objects/typeobject.c slot_nb_or op_id - -Objects/typeobject.c slot_nb_or rop_id - -Objects/typeobject.c slot_nb_int id - -Objects/typeobject.c slot_nb_float id - -Objects/typeobject.c slot_nb_inplace_add id - -Objects/typeobject.c slot_nb_inplace_subtract id - -Objects/typeobject.c slot_nb_inplace_multiply id - -Objects/typeobject.c slot_nb_inplace_matrix_multiply id - -Objects/typeobject.c slot_nb_inplace_remainder id - -Objects/typeobject.c slot_nb_inplace_lshift id - -Objects/typeobject.c slot_nb_inplace_rshift id - -Objects/typeobject.c slot_nb_inplace_and id - -Objects/typeobject.c slot_nb_inplace_xor id - -Objects/typeobject.c slot_nb_inplace_or id - -Objects/typeobject.c slot_nb_floor_divide op_id - -Objects/typeobject.c slot_nb_floor_divide rop_id - -Objects/typeobject.c slot_nb_true_divide op_id - -Objects/typeobject.c slot_nb_true_divide rop_id - -Objects/typeobject.c slot_nb_inplace_floor_divide id - -Objects/typeobject.c slot_nb_inplace_true_divide id - -Objects/typeobject.c slot_tp_str id - -Python/compile.c compiler_set_qualname dot - -Python/compile.c compiler_set_qualname dot_locals - +# cached - initialized once # manually cached PyUnicodeObject Objects/boolobject.c - false_str - @@ -540,9 +311,7 @@ Python/sysmodule.c - whatstrings - # XXX This should have been found by the analyzer but wasn't: Python/sysmodule.c sys_displayhook newline - -#----------------------- # _PyArg_Parser - Objects/clinic/bytearrayobject.c.h bytearray___init__ _parser - Objects/clinic/bytearrayobject.c.h bytearray_decode _parser - Objects/clinic/bytearrayobject.c.h bytearray_hex _parser - @@ -598,15 +367,16 @@ Python/clinic/sysmodule.c.h sys_addaudithook _parser - Python/clinic/sysmodule.c.h sys_set_coroutine_origin_tracking_depth _parser - Python/clinic/traceback.c.h tb_new _parser - -#----------------------- # other - -# cache - initialized once +Objects/typeobject.c - method_cache - +Objects/unicodeobject.c - _string_module - Objects/unicodeobject.c - interned - Objects/unicodeobject.c - static_strings - -Objects/typeobject.c - method_cache - -# non-cache - initialized once +#----------------------- +# other + +# initialized once Objects/exceptions.c - PyExc_BaseExceptionGroup - Objects/exceptions.c - PyExc_EncodingWarning - # XXX This should have been found by the analyzer but wasn't: @@ -632,6 +402,9 @@ Python/import.c - extensions - #----------------------- # initialized once +# pre-allocated buffer +Modules/getbuildinfo.c Py_GetBuildInfo buildinfo - + # during init Parser/parser.c - Py_DebugFlag - @@ -678,7 +451,6 @@ Parser/myreadline.c - PyOS_InputHook - Parser/myreadline.c - PyOS_ReadlineFunctionPointer - # other -Modules/signalmodule.c - signal_global_state - Objects/dictobject.c - _pydict_global_version - Objects/dictobject.c - next_dict_keys_version - Objects/dictobject.c - pydict_global_version - @@ -715,18 +487,254 @@ Python/pylifecycle.c fatal_error reentrant - ################################## -# global objects to fix in extension modules +# global objects to fix in builtin modules #----------------------- # modules Modules/_abc.c - _abcmodule - +Modules/_codecsmodule.c - codecsmodule - +Modules/_collectionsmodule.c - _collectionsmodule - +Modules/_functoolsmodule.c - _functools_module - +Modules/_io/_iomodule.c - _PyIO_Module - +Modules/_io/_iomodule.h - _PyIO_Module - +Modules/_localemodule.c - _localemodule - +Modules/_sre.c - sremodule - +Modules/_stat.c - statmodule - +Modules/_threadmodule.c - threadmodule - +Modules/_tracemalloc.c - module_def - +Modules/_weakref.c - weakrefmodule - +Modules/atexitmodule.c - atexitmodule - +Modules/errnomodule.c - errnomodule - +Modules/faulthandler.c - module_def - +Modules/gcmodule.c - gcmodule - +Modules/itertoolsmodule.c - itertoolsmodule - +Modules/posixmodule.c - posixmodule - +Modules/pwdmodule.c - pwdmodule - +Modules/signalmodule.c - signalmodule - +Modules/symtablemodule.c - symtablemodule - +Modules/timemodule.c - timemodule - + +#----------------------- +# static types + +Modules/_collectionsmodule.c - defdict_type - +Modules/_collectionsmodule.c - deque_type - +Modules/_collectionsmodule.c - dequeiter_type - +Modules/_collectionsmodule.c - dequereviter_type - +Modules/_collectionsmodule.c - tuplegetter_type - +Modules/_functoolsmodule.c - keyobject_type - +Modules/_functoolsmodule.c - lru_cache_type - +Modules/_functoolsmodule.c - lru_list_elem_type - +Modules/_functoolsmodule.c - partial_type - +Modules/_io/bufferedio.c - PyBufferedIOBase_Type - +Modules/_io/bufferedio.c - PyBufferedRWPair_Type - +Modules/_io/bufferedio.c - PyBufferedRandom_Type - +Modules/_io/bufferedio.c - PyBufferedReader_Type - +Modules/_io/bufferedio.c - PyBufferedWriter_Type - +Modules/_io/bytesio.c - PyBytesIO_Type - +Modules/_io/bytesio.c - _PyBytesIOBuffer_Type - +Modules/_io/fileio.c - PyFileIO_Type - +Modules/_io/iobase.c - PyIOBase_Type - +Modules/_io/iobase.c - PyRawIOBase_Type - +Modules/_io/stringio.c - PyStringIO_Type - +Modules/_io/textio.c - PyIncrementalNewlineDecoder_Type - +Modules/_io/textio.c - PyTextIOBase_Type - +Modules/_io/textio.c - PyTextIOWrapper_Type - +Modules/_io/winconsoleio.c - PyWindowsConsoleIO_Type - +Modules/_threadmodule.c - Locktype - +Modules/_threadmodule.c - RLocktype - +Modules/_threadmodule.c - localdummytype - +Modules/_threadmodule.c - localtype - +Modules/itertoolsmodule.c - _grouper_type - +Modules/itertoolsmodule.c - accumulate_type - +Modules/itertoolsmodule.c - chain_type - +Modules/itertoolsmodule.c - combinations_type - +Modules/itertoolsmodule.c - compress_type - +Modules/itertoolsmodule.c - count_type - +Modules/itertoolsmodule.c - cwr_type - +Modules/itertoolsmodule.c - cycle_type - +Modules/itertoolsmodule.c - dropwhile_type - +Modules/itertoolsmodule.c - filterfalse_type - +Modules/itertoolsmodule.c - groupby_type - +Modules/itertoolsmodule.c - islice_type - +Modules/itertoolsmodule.c - pairwise_type - +Modules/itertoolsmodule.c - permutations_type - +Modules/itertoolsmodule.c - product_type - +Modules/itertoolsmodule.c - repeat_type - +Modules/itertoolsmodule.c - starmap_type - +Modules/itertoolsmodule.c - takewhile_type - +Modules/itertoolsmodule.c - tee_type - +Modules/itertoolsmodule.c - teedataobject_type - +Modules/itertoolsmodule.c - ziplongest_type - + +#----------------------- +# non-static types - initialized once + +# structseq types +Modules/_threadmodule.c - ExceptHookArgsType - +Modules/signalmodule.c - SiginfoType - +Modules/timemodule.c - StructTimeType - + +# exception types +Modules/_threadmodule.c - ThreadError - +Modules/signalmodule.c - ItimerError - + +#----------------------- +# cached - initialized once + +# manually cached PyUnicodeOjbect +Modules/_io/_iomodule.c - _PyIO_str_close - +Modules/_io/_iomodule.c - _PyIO_str_closed - +Modules/_io/_iomodule.c - _PyIO_str_decode - +Modules/_io/_iomodule.c - _PyIO_str_encode - +Modules/_io/_iomodule.c - _PyIO_str_fileno - +Modules/_io/_iomodule.c - _PyIO_str_flush - +Modules/_io/_iomodule.c - _PyIO_str_getstate - +Modules/_io/_iomodule.c - _PyIO_str_isatty - +Modules/_io/_iomodule.c - _PyIO_str_locale - +Modules/_io/_iomodule.c - _PyIO_str_newlines - +Modules/_io/_iomodule.c - _PyIO_str_nl - +Modules/_io/_iomodule.c - _PyIO_str_peek - +Modules/_io/_iomodule.c - _PyIO_str_read - +Modules/_io/_iomodule.c - _PyIO_str_read1 - +Modules/_io/_iomodule.c - _PyIO_str_readable - +Modules/_io/_iomodule.c - _PyIO_str_readall - +Modules/_io/_iomodule.c - _PyIO_str_readinto - +Modules/_io/_iomodule.c - _PyIO_str_readline - +Modules/_io/_iomodule.c - _PyIO_str_reset - +Modules/_io/_iomodule.c - _PyIO_str_seek - +Modules/_io/_iomodule.c - _PyIO_str_seekable - +Modules/_io/_iomodule.c - _PyIO_str_setstate - +Modules/_io/_iomodule.c - _PyIO_str_tell - +Modules/_io/_iomodule.c - _PyIO_str_truncate - +Modules/_io/_iomodule.c - _PyIO_str_writable - +Modules/_io/_iomodule.c - _PyIO_str_write - +Modules/_io/_iomodule.c - _PyIO_empty_str - +Modules/_threadmodule.c - str_dict - +Modules/_tracemalloc.c - unknown_filename - + +# _PyArg_Parser +Modules/clinic/_codecsmodule.c.h _codecs_decode _parser - +Modules/clinic/_codecsmodule.c.h _codecs_encode _parser - +Modules/clinic/_sre.c.h _sre_SRE_Match_expand _parser - +Modules/clinic/_sre.c.h _sre_SRE_Match_groupdict _parser - +Modules/clinic/_sre.c.h _sre_SRE_Match_groups _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_findall _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_finditer _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_fullmatch _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_match _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_scanner _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_search _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_split _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_sub _parser - +Modules/clinic/_sre.c.h _sre_SRE_Pattern_subn _parser - +Modules/clinic/_sre.c.h _sre_SRE_Scanner_match _parser - +Modules/clinic/_sre.c.h _sre_SRE_Scanner_search _parser - +Modules/clinic/_sre.c.h _sre_compile _parser - +Modules/clinic/gcmodule.c.h gc_collect _parser - +Modules/clinic/gcmodule.c.h gc_get_objects _parser - +Modules/clinic/itertoolsmodule.c.h itertools_accumulate _parser - +Modules/clinic/itertoolsmodule.c.h itertools_combinations _parser - +Modules/clinic/itertoolsmodule.c.h itertools_combinations_with_replacement _parser - +Modules/clinic/itertoolsmodule.c.h itertools_compress _parser - +Modules/clinic/itertoolsmodule.c.h itertools_count _parser - +Modules/clinic/itertoolsmodule.c.h itertools_groupby _parser - +Modules/clinic/itertoolsmodule.c.h itertools_permutations _parser - +Modules/clinic/posixmodule.c.h os_DirEntry_is_dir _parser - +Modules/clinic/posixmodule.c.h os_DirEntry_is_file _parser - +Modules/clinic/posixmodule.c.h os_DirEntry_is_symlink _parser - +Modules/clinic/posixmodule.c.h os_DirEntry_stat _parser - +Modules/clinic/posixmodule.c.h os__exit _parser - +Modules/clinic/posixmodule.c.h os__path_normpath _parser - +Modules/clinic/posixmodule.c.h os_access _parser - +Modules/clinic/posixmodule.c.h os_chdir _parser - +Modules/clinic/posixmodule.c.h os_chmod _parser - +Modules/clinic/posixmodule.c.h os_close _parser - +Modules/clinic/posixmodule.c.h os_device_encoding _parser - +Modules/clinic/posixmodule.c.h os_dup2 _parser - +Modules/clinic/posixmodule.c.h os_fspath _parser - +Modules/clinic/posixmodule.c.h os_fstat _parser - +Modules/clinic/posixmodule.c.h os_listdir _parser - +Modules/clinic/posixmodule.c.h os_lstat _parser - +Modules/clinic/posixmodule.c.h os_mkdir _parser - +Modules/clinic/posixmodule.c.h os_open _parser - +Modules/clinic/posixmodule.c.h os_remove _parser - +Modules/clinic/posixmodule.c.h os_rename _parser - +Modules/clinic/posixmodule.c.h os_replace _parser - +Modules/clinic/posixmodule.c.h os_rmdir _parser - +Modules/clinic/posixmodule.c.h os_scandir _parser - +Modules/clinic/posixmodule.c.h os_stat _parser - +Modules/clinic/posixmodule.c.h os_unlink _parser - +Modules/clinic/posixmodule.c.h os_utime _parser - + +#----------------------- +# other + +# initialized once +Modules/_functoolsmodule.c - kwd_mark - +Modules/_io/_iomodule.c - _PyIO_empty_bytes - +Modules/_tracemalloc.c - tracemalloc_empty_traceback - +Modules/signalmodule.c - DefaultHandler - +Modules/signalmodule.c - IgnoreHandler - +Modules/signalmodule.c - IntHandler - + +# state +Modules/faulthandler.c - fatal_error - +Modules/faulthandler.c - thread - +Modules/faulthandler.c - user_signals - +Modules/faulthandler.c - stack - +Modules/faulthandler.c - old_stack - +Modules/signalmodule.c - Handlers - + + +################################## +# global non-objects to fix in builtin modules + +#----------------------- +# initialized once + +Modules/_io/bufferedio.c _PyIO_trap_eintr eintr_int - +#Modules/cjkcodecs/cjkcodecs.h - codec_list - +#Modules/cjkcodecs/cjkcodecs.h - mapping_list - +Modules/posixmodule.c os_dup2_impl dup3_works - +Modules/posixmodule.c - structseq_new - +Modules/posixmodule.c - ticks_per_second - +Modules/signalmodule.c - initialized - +Modules/timemodule.c - initialized - +Modules/timemodule.c _PyTime_GetClockWithInfo initialized - +Modules/timemodule.c _PyTime_GetProcessTimeWithInfo ticks_per_second - + +#----------------------- +# state + +Modules/_tracemalloc.c - allocators - +Modules/_tracemalloc.c - tables_lock - +Modules/_tracemalloc.c - tracemalloc_traced_memory - +Modules/_tracemalloc.c - tracemalloc_peak_traced_memory - +Modules/_tracemalloc.c - tracemalloc_filenames - +Modules/_tracemalloc.c - tracemalloc_traceback - +Modules/_tracemalloc.c - tracemalloc_tracebacks - +Modules/_tracemalloc.c - tracemalloc_traces - +Modules/_tracemalloc.c - tracemalloc_domains - +Modules/_tracemalloc.c - tracemalloc_reentrant_key - +Modules/faulthandler.c faulthandler_dump_traceback reentrant - +Modules/posixmodule.c - environ - +Modules/signalmodule.c - is_tripped - +Modules/signalmodule.c - signal_global_state - +Modules/signalmodule.c - wakeup - + + +################################## +# global objects to fix in extension modules + +#----------------------- +# modules + Modules/_asynciomodule.c - _asynciomodule - Modules/_bisectmodule.c - _bisectmodule - Modules/_blake2/blake2module.c - blake2_module - Modules/_bz2module.c - _bz2module - -Modules/_codecsmodule.c - codecsmodule - -Modules/_collectionsmodule.c - _collectionsmodule - Modules/_contextvarsmodule.c - _contextvarsmodule - Modules/_cryptmodule.c - cryptmodule - Modules/_csv.c - _csvmodule - @@ -736,14 +744,10 @@ Modules/_cursesmodule.c - _cursesmodule - Modules/_datetimemodule.c - datetimemodule - Modules/_decimal/_decimal.c - _decimal_module - Modules/_elementtree.c - elementtreemodule - -Modules/_functoolsmodule.c - _functools_module - Modules/_gdbmmodule.c - _gdbmmodule - Modules/_hashopenssl.c - _hashlibmodule - Modules/_heapqmodule.c - _heapqmodule - -Modules/_io/_iomodule.c - _PyIO_Module - -Modules/_io/_iomodule.h - _PyIO_Module - Modules/_json.c - jsonmodule - -Modules/_localemodule.c - _localemodule - Modules/_lsprof.c - _lsprofmodule - Modules/_lzmamodule.c - _lzmamodule - Modules/_multiprocessing/multiprocessing.c - multiprocessing_module - @@ -756,38 +760,26 @@ Modules/_queuemodule.c - queuemodule - Modules/_randommodule.c - _randommodule - Modules/_sha3/sha3module.c - _sha3module - Modules/_sqlite/module.c - _sqlite3module - -Modules/_sre.c - sremodule - Modules/_ssl.c - PySocketModule - Modules/_ssl.c - _sslmodule - -Modules/_stat.c - statmodule - Modules/_statisticsmodule.c - statisticsmodule - Modules/_struct.c - _structmodule - -Modules/_threadmodule.c - threadmodule - Modules/_tkinter.c - _tkintermodule - -Modules/_tracemalloc.c - module_def - Modules/_uuidmodule.c - uuidmodule - -Modules/_weakref.c - weakrefmodule - Modules/_xxsubinterpretersmodule.c - interpretersmodule - Modules/_zoneinfo.c - zoneinfomodule - Modules/arraymodule.c - arraymodule - -Modules/atexitmodule.c - atexitmodule - Modules/audioop.c - audioopmodule - Modules/binascii.c - binasciimodule - Modules/cjkcodecs/multibytecodec.c - _multibytecodecmodule - Modules/cmathmodule.c - cmathmodule - -Modules/errnomodule.c - errnomodule - -Modules/faulthandler.c - module_def - Modules/fcntlmodule.c - fcntlmodule - -Modules/gcmodule.c - gcmodule - Modules/grpmodule.c - grpmodule - -Modules/itertoolsmodule.c - itertoolsmodule - Modules/mathmodule.c - mathmodule - Modules/md5module.c - _md5module - Modules/mmapmodule.c - mmapmodule - Modules/nismodule.c - nismodule - Modules/ossaudiodev.c - ossaudiodevmodule - -Modules/posixmodule.c - posixmodule - -Modules/pwdmodule.c - pwdmodule - Modules/pyexpat.c - pyexpatmodule - Modules/readline.c - readlinemodule - Modules/resource.c - resourcemodule - @@ -795,19 +787,15 @@ Modules/selectmodule.c - selectmodule - Modules/sha1module.c - _sha1module - Modules/sha256module.c - _sha256module - Modules/sha512module.c - _sha512module - -Modules/signalmodule.c - signalmodule - Modules/socketmodule.c - socketmodule - Modules/spwdmodule.c - spwdmodule - -Modules/symtablemodule.c - symtablemodule - Modules/syslogmodule.c - syslogmodule - Modules/termios.c - termiosmodule - -Modules/timemodule.c - timemodule - Modules/unicodedata.c - unicodedata_module - Modules/xxlimited.c - xxmodule - Modules/xxmodule.c - xxmodule - Modules/xxsubtype.c - xxsubtypemodule - Modules/zlibmodule.c - zlibmodule - -Objects/unicodeobject.c - _string_module - #----------------------- # static types @@ -817,11 +805,6 @@ Modules/_asynciomodule.c - FutureType - Modules/_asynciomodule.c - PyRunningLoopHolder_Type - Modules/_asynciomodule.c - TaskStepMethWrapper_Type - Modules/_asynciomodule.c - TaskType - -Modules/_collectionsmodule.c - defdict_type - -Modules/_collectionsmodule.c - deque_type - -Modules/_collectionsmodule.c - dequeiter_type - -Modules/_collectionsmodule.c - dequereviter_type - -Modules/_collectionsmodule.c - tuplegetter_type - Modules/_csv.c - Dialect_Type - Modules/_csv.c - Reader_Type - Modules/_csv.c - Writer_Type - @@ -861,25 +844,6 @@ Modules/_elementtree.c - ElementIter_Type - Modules/_elementtree.c - Element_Type - Modules/_elementtree.c - TreeBuilder_Type - Modules/_elementtree.c - XMLParser_Type - -Modules/_functoolsmodule.c - keyobject_type - -Modules/_functoolsmodule.c - lru_cache_type - -Modules/_functoolsmodule.c - lru_list_elem_type - -Modules/_functoolsmodule.c - partial_type - -Modules/_io/bufferedio.c - PyBufferedIOBase_Type - -Modules/_io/bufferedio.c - PyBufferedRWPair_Type - -Modules/_io/bufferedio.c - PyBufferedRandom_Type - -Modules/_io/bufferedio.c - PyBufferedReader_Type - -Modules/_io/bufferedio.c - PyBufferedWriter_Type - -Modules/_io/bytesio.c - PyBytesIO_Type - -Modules/_io/bytesio.c - _PyBytesIOBuffer_Type - -Modules/_io/fileio.c - PyFileIO_Type - -Modules/_io/iobase.c - PyIOBase_Type - -Modules/_io/iobase.c - PyRawIOBase_Type - -Modules/_io/stringio.c - PyStringIO_Type - -Modules/_io/textio.c - PyIncrementalNewlineDecoder_Type - -Modules/_io/textio.c - PyTextIOBase_Type - -Modules/_io/textio.c - PyTextIOWrapper_Type - -Modules/_io/winconsoleio.c - PyWindowsConsoleIO_Type - Modules/_multiprocessing/semaphore.c - _PyMp_SemLockType - Modules/_pickle.c - Pdata_Type - Modules/_pickle.c - PicklerMemoProxyType - @@ -894,10 +858,6 @@ Modules/_ssl.c - PySSLContext_Type - Modules/_ssl.c - PySSLMemoryBIO_Type - Modules/_ssl.c - PySSLSession_Type - Modules/_ssl.c - PySSLSocket_Type - -Modules/_threadmodule.c - Locktype - -Modules/_threadmodule.c - RLocktype - -Modules/_threadmodule.c - localdummytype - -Modules/_threadmodule.c - localtype - Modules/_xxsubinterpretersmodule.c - ChannelIDtype - Modules/_zoneinfo.c - PyZoneInfo_ZoneInfoType - Modules/arraymodule.c - Arraytype - @@ -907,27 +867,6 @@ Modules/cjkcodecs/multibytecodec.c - MultibyteIncrementalDecoder_Type - Modules/cjkcodecs/multibytecodec.c - MultibyteIncrementalEncoder_Type - Modules/cjkcodecs/multibytecodec.c - MultibyteStreamReader_Type - Modules/cjkcodecs/multibytecodec.c - MultibyteStreamWriter_Type - -Modules/itertoolsmodule.c - _grouper_type - -Modules/itertoolsmodule.c - accumulate_type - -Modules/itertoolsmodule.c - chain_type - -Modules/itertoolsmodule.c - combinations_type - -Modules/itertoolsmodule.c - compress_type - -Modules/itertoolsmodule.c - count_type - -Modules/itertoolsmodule.c - cwr_type - -Modules/itertoolsmodule.c - cycle_type - -Modules/itertoolsmodule.c - dropwhile_type - -Modules/itertoolsmodule.c - filterfalse_type - -Modules/itertoolsmodule.c - groupby_type - -Modules/itertoolsmodule.c - islice_type - -Modules/itertoolsmodule.c - pairwise_type - -Modules/itertoolsmodule.c - permutations_type - -Modules/itertoolsmodule.c - product_type - -Modules/itertoolsmodule.c - repeat_type - -Modules/itertoolsmodule.c - starmap_type - -Modules/itertoolsmodule.c - takewhile_type - -Modules/itertoolsmodule.c - tee_type - -Modules/itertoolsmodule.c - teedataobject_type - -Modules/itertoolsmodule.c - ziplongest_type - Modules/mmapmodule.c - mmap_object_type - Modules/ossaudiodev.c - OSSAudioType - Modules/ossaudiodev.c - OSSMixerType - @@ -945,11 +884,8 @@ Modules/xxsubtype.c - spamlist_type - # structseq types Modules/_cursesmodule.c - NcursesVersionType - -Modules/_threadmodule.c - ExceptHookArgsType - Modules/resource.c - StructRUsageType - -Modules/signalmodule.c - SiginfoType - Modules/spwdmodule.c - StructSpwdType - -Modules/timemodule.c - StructTimeType - # heap types Modules/_decimal/_decimal.c - DecimalTuple - @@ -971,7 +907,6 @@ Modules/_ssl.c - PySSLWantReadErrorObject - Modules/_ssl.c - PySSLWantWriteErrorObject - Modules/_ssl.c - PySSLSyscallErrorObject - Modules/_ssl.c - PySSLEOFErrorObject - -Modules/_threadmodule.c - ThreadError - Modules/_tkinter.c - Tkinter_TclError - Modules/_xxsubinterpretersmodule.c - ChannelError - Modules/_xxsubinterpretersmodule.c - ChannelNotFoundError - @@ -981,7 +916,6 @@ Modules/_xxsubinterpretersmodule.c - ChannelNotEmptyError - Modules/_xxsubinterpretersmodule.c - RunFailedError - Modules/ossaudiodev.c - OSSAudioError - Modules/pyexpat.c - ErrorObject - -Modules/signalmodule.c - ItimerError - Modules/socketmodule.c - socket_herror - Modules/socketmodule.c - socket_gaierror - Modules/socketmodule.c - socket_timeout - @@ -989,17 +923,9 @@ Modules/xxlimited.c - ErrorObject - Modules/xxmodule.c - ErrorObject - #----------------------- -# cached PyUnicodeObject - initialized once +# cached - initialized once # _Py_IDENTIFIER (global) -Modules/_abc.c - PyId___abc_tpflags__ - -Modules/_abc.c - PyId___abstractmethods__ - -Modules/_abc.c - PyId___bases__ - -Modules/_abc.c - PyId___class__ - -Modules/_abc.c - PyId___dict__ - -Modules/_abc.c - PyId___subclasscheck__ - -Modules/_abc.c - PyId___subclasshook__ - -Modules/_abc.c - PyId__abc_impl - Modules/_asynciomodule.c - PyId___asyncio_running_event_loop__ - Modules/_asynciomodule.c - PyId__asyncio_future_blocking - Modules/_asynciomodule.c - PyId_add_done_callback - @@ -1012,53 +938,8 @@ Modules/_datetimemodule.c - PyId_as_integer_ratio - Modules/_datetimemodule.c - PyId_fromutc - Modules/_datetimemodule.c - PyId_isoformat - Modules/_datetimemodule.c - PyId_strftime - -Modules/_io/bufferedio.c - PyId__dealloc_warn - -Modules/_io/bufferedio.c - PyId_close - -Modules/_io/bufferedio.c - PyId_flush - -Modules/_io/bufferedio.c - PyId_isatty - -Modules/_io/bufferedio.c - PyId_mode - -Modules/_io/bufferedio.c - PyId_name - -Modules/_io/bufferedio.c - PyId_peek - -Modules/_io/bufferedio.c - PyId_read - -Modules/_io/bufferedio.c - PyId_read1 - -Modules/_io/bufferedio.c - PyId_readable - -Modules/_io/bufferedio.c - PyId_readinto - -Modules/_io/bufferedio.c - PyId_readinto1 - -Modules/_io/bufferedio.c - PyId_writable - -Modules/_io/bufferedio.c - PyId_write - -Modules/_io/fileio.c - PyId_name - -Modules/_io/iobase.c - PyId___IOBase_closed - -Modules/_io/iobase.c - PyId_read - -Modules/_io/textio.c - PyId__dealloc_warn - -Modules/_io/textio.c - PyId_close - -Modules/_io/textio.c - PyId_decode - -Modules/_io/textio.c - PyId_fileno - -Modules/_io/textio.c - PyId_flush - -Modules/_io/textio.c - PyId_getpreferredencoding - -Modules/_io/textio.c - PyId_isatty - -Modules/_io/textio.c - PyId_mode - -Modules/_io/textio.c - PyId_name - -Modules/_io/textio.c - PyId_raw - -Modules/_io/textio.c - PyId_read - -Modules/_io/textio.c - PyId_readable - -Modules/_io/textio.c - PyId_replace - -Modules/_io/textio.c - PyId_reset - -Modules/_io/textio.c - PyId_seek - -Modules/_io/textio.c - PyId_seekable - -Modules/_io/textio.c - PyId_setstate - -Modules/_io/textio.c - PyId_strict - -Modules/_io/textio.c - PyId_tell - -Modules/_io/textio.c - PyId_writable - Modules/_sqlite/connection.c - PyId_cursor - -Modules/_threadmodule.c - PyId___dict__ - -Modules/_threadmodule.c - PyId_flush - -Modules/_threadmodule.c - PyId_stderr - Modules/cjkcodecs/multibytecodec.c - PyId_write - -Modules/faulthandler.c - PyId_enable - -Modules/faulthandler.c - PyId_fileno - -Modules/faulthandler.c - PyId_flush - -Modules/faulthandler.c - PyId_stderr - -Modules/posixmodule.c - PyId___fspath__ - Modules/unicodedata.c - PyId_NFC - Modules/unicodedata.c - PyId_NFD - Modules/unicodedata.c - PyId_NFKC - @@ -1101,10 +982,6 @@ Modules/_asynciomodule.c get_future_loop PyId__loop - Modules/_asynciomodule.c get_future_loop PyId_get_loop - Modules/_asynciomodule.c register_task PyId_add - Modules/_asynciomodule.c unregister_task PyId_discard - -Modules/_collectionsmodule.c _collections__count_elements_impl PyId___setitem__ - -Modules/_collectionsmodule.c _collections__count_elements_impl PyId_get - -Modules/_collectionsmodule.c defdict_reduce PyId_items - -Modules/_collectionsmodule.c deque_reduce PyId___dict__ - Modules/_csv.c csv_writer PyId_write - Modules/_ctypes/_ctypes.c CDataType_from_param PyId__as_parameter_ - Modules/_ctypes/_ctypes.c PyCArrayType_new PyId__length_ - @@ -1155,15 +1032,6 @@ Modules/_elementtree.c treebuilder_add_subelement PyId_append - Modules/_elementtree.c treebuilder_flush_data PyId_tail - Modules/_elementtree.c treebuilder_flush_data PyId_text - Modules/_gdbmmodule.c gdbm__exit__ PyId_close - -Modules/_io/_iomodule.c _io_open_impl PyId__blksize - -Modules/_io/_iomodule.c _io_open_impl PyId_close - -Modules/_io/_iomodule.c _io_open_impl PyId_isatty - -Modules/_io/_iomodule.c _io_open_impl PyId_mode - -Modules/_io/fileio.c _io_FileIO_close_impl PyId_close - -Modules/_io/iobase.c _io__IOBase_readlines_impl PyId_extend - -Modules/_io/iobase.c _io__IOBase_tell_impl PyId_seek - -Modules/_io/iobase.c _io__RawIOBase_read_impl PyId_readall - -Modules/_io/iobase.c iobase_finalize PyId__finalizing - Modules/_lzmamodule.c build_filter_spec PyId_dict_size - Modules/_lzmamodule.c build_filter_spec PyId_dist - Modules/_lzmamodule.c build_filter_spec PyId_id - @@ -1216,24 +1084,17 @@ Modules/_ssl.c fill_and_set_sslerror PyId_library - Modules/_ssl.c fill_and_set_sslerror PyId_reason - Modules/_ssl.c fill_and_set_sslerror PyId_verify_code - Modules/_ssl.c fill_and_set_sslerror PyId_verify_message - -Modules/_threadmodule.c thread_excepthook_file PyId_name - Modules/arraymodule.c array_array___reduce_ex__ PyId___dict__ - Modules/arraymodule.c array_array___reduce_ex__ PyId__array_reconstructor - Modules/arraymodule.c array_array_fromfile_impl PyId_read - Modules/arraymodule.c array_array_tofile PyId_write - Modules/arraymodule.c array_arrayiterator___reduce___impl PyId_iter - -Modules/itertoolsmodule.c _grouper_reduce PyId_iter - -Modules/itertoolsmodule.c cycle_reduce PyId___setstate__ - -Modules/itertoolsmodule.c itertools_tee_impl PyId___copy__ - -Modules/itertoolsmodule.c zip_longest_new PyId_fillvalue - -Modules/main.c pymain_sys_path_add_path0 PyId_path - Modules/mathmodule.c math_ceil PyId___ceil__ - Modules/mathmodule.c math_floor PyId___floor__ - Modules/mathmodule.c math_trunc PyId___trunc__ - Modules/mmapmodule.c mmap__exit__method PyId_close - Modules/ossaudiodev.c oss_exit PyId_close - Modules/pyexpat.c pyexpat_xmlparser_ParseFile PyId_read - -Modules/timemodule.c time_strptime PyId__strptime_time - # _Py_static_string Modules/_pickle.c get_dotted_path PyId_dot - @@ -1242,33 +1103,6 @@ Modules/_pickle.c get_dotted_path PyId_dot - Modules/_asynciomodule.c - context_kwname - Modules/_ctypes/callproc.c _ctypes_get_errobj error_object_name - Modules/_ctypes/_ctypes.c CreateSwappedType suffix - -Modules/_io/_iomodule.c - _PyIO_str_close - -Modules/_io/_iomodule.c - _PyIO_str_closed - -Modules/_io/_iomodule.c - _PyIO_str_decode - -Modules/_io/_iomodule.c - _PyIO_str_encode - -Modules/_io/_iomodule.c - _PyIO_str_fileno - -Modules/_io/_iomodule.c - _PyIO_str_flush - -Modules/_io/_iomodule.c - _PyIO_str_getstate - -Modules/_io/_iomodule.c - _PyIO_str_isatty - -Modules/_io/_iomodule.c - _PyIO_str_locale - -Modules/_io/_iomodule.c - _PyIO_str_newlines - -Modules/_io/_iomodule.c - _PyIO_str_nl - -Modules/_io/_iomodule.c - _PyIO_str_peek - -Modules/_io/_iomodule.c - _PyIO_str_read - -Modules/_io/_iomodule.c - _PyIO_str_read1 - -Modules/_io/_iomodule.c - _PyIO_str_readable - -Modules/_io/_iomodule.c - _PyIO_str_readall - -Modules/_io/_iomodule.c - _PyIO_str_readinto - -Modules/_io/_iomodule.c - _PyIO_str_readline - -Modules/_io/_iomodule.c - _PyIO_str_reset - -Modules/_io/_iomodule.c - _PyIO_str_seek - -Modules/_io/_iomodule.c - _PyIO_str_seekable - -Modules/_io/_iomodule.c - _PyIO_str_setstate - -Modules/_io/_iomodule.c - _PyIO_str_tell - -Modules/_io/_iomodule.c - _PyIO_str_truncate - -Modules/_io/_iomodule.c - _PyIO_str_writable - -Modules/_io/_iomodule.c - _PyIO_str_write - -Modules/_io/_iomodule.c - _PyIO_empty_str - Modules/_json.c _encoded_const s_null - Modules/_json.c _encoded_const s_true - Modules/_json.c _encoded_const s_false - @@ -1278,12 +1112,8 @@ Modules/_json.c encoder_listencode_dict empty_dict - Modules/_json.c encoder_listencode_list open_array - Modules/_json.c encoder_listencode_list close_array - Modules/_json.c encoder_listencode_list empty_array - -Modules/_threadmodule.c - str_dict - -Modules/_tracemalloc.c - unknown_filename - -#----------------------- # _PyArg_Parser - Modules/clinic/_asynciomodule.c.h _asyncio_Future___init__ _parser - Modules/clinic/_asynciomodule.c.h _asyncio_Future_add_done_callback _parser - Modules/clinic/_asynciomodule.c.h _asyncio_Future_cancel _parser - @@ -1301,8 +1131,6 @@ Modules/clinic/_bisectmodule.c.h _bisect_bisect_right _parser - Modules/clinic/_bisectmodule.c.h _bisect_insort_left _parser - Modules/clinic/_bisectmodule.c.h _bisect_insort_right _parser - Modules/clinic/_bz2module.c.h _bz2_BZ2Decompressor_decompress _parser - -Modules/clinic/_codecsmodule.c.h _codecs_decode _parser - -Modules/clinic/_codecsmodule.c.h _codecs_encode _parser - Modules/clinic/_curses_panel.c.h _curses_panel_panel_bottom _parser - Modules/clinic/_curses_panel.c.h _curses_panel_panel_hide _parser - Modules/clinic/_curses_panel.c.h _curses_panel_panel_move _parser - @@ -1355,21 +1183,6 @@ Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_get _parser - Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_get_nowait _parser - Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_put _parser - Modules/clinic/_queuemodule.c.h _queue_SimpleQueue_put_nowait _parser - -Modules/clinic/_sre.c.h _sre_SRE_Match_expand _parser - -Modules/clinic/_sre.c.h _sre_SRE_Match_groupdict _parser - -Modules/clinic/_sre.c.h _sre_SRE_Match_groups _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_findall _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_finditer _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_fullmatch _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_match _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_scanner _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_search _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_split _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_sub _parser - -Modules/clinic/_sre.c.h _sre_SRE_Pattern_subn _parser - -Modules/clinic/_sre.c.h _sre_SRE_Scanner_match _parser - -Modules/clinic/_sre.c.h _sre_SRE_Scanner_search _parser - -Modules/clinic/_sre.c.h _sre_compile _parser - Modules/clinic/_ssl.c.h _ssl__SSLContext__wrap_bio _parser - Modules/clinic/_ssl.c.h _ssl__SSLContext__wrap_socket _parser - Modules/clinic/_ssl.c.h _ssl__SSLContext_get_ca_certs _parser - @@ -1398,48 +1211,13 @@ Modules/clinic/binascii.c.h binascii_b2a_qp _parser - Modules/clinic/binascii.c.h binascii_b2a_uu _parser - Modules/clinic/binascii.c.h binascii_hexlify _parser - Modules/clinic/cmathmodule.c.h cmath_isclose _parser - -Modules/clinic/gcmodule.c.h gc_collect _parser - -Modules/clinic/gcmodule.c.h gc_get_objects _parser - Modules/clinic/grpmodule.c.h grp_getgrgid _parser - Modules/clinic/grpmodule.c.h grp_getgrnam _parser - -Modules/clinic/itertoolsmodule.c.h itertools_accumulate _parser - -Modules/clinic/itertoolsmodule.c.h itertools_combinations _parser - -Modules/clinic/itertoolsmodule.c.h itertools_combinations_with_replacement _parser - -Modules/clinic/itertoolsmodule.c.h itertools_compress _parser - -Modules/clinic/itertoolsmodule.c.h itertools_count _parser - -Modules/clinic/itertoolsmodule.c.h itertools_groupby _parser - -Modules/clinic/itertoolsmodule.c.h itertools_permutations _parser - Modules/clinic/mathmodule.c.h math_isclose _parser - Modules/clinic/mathmodule.c.h math_prod _parser - Modules/clinic/md5module.c.h MD5Type_copy _parser - Modules/clinic/md5module.c.h _md5_md5 _parser - Modules/clinic/overlapped.c.h _overlapped_Overlapped _parser - -Modules/clinic/posixmodule.c.h os_DirEntry_is_dir _parser - -Modules/clinic/posixmodule.c.h os_DirEntry_is_file _parser - -Modules/clinic/posixmodule.c.h os_DirEntry_is_symlink _parser - -Modules/clinic/posixmodule.c.h os_DirEntry_stat _parser - -Modules/clinic/posixmodule.c.h os__exit _parser - -Modules/clinic/posixmodule.c.h os__path_normpath _parser - -Modules/clinic/posixmodule.c.h os_access _parser - -Modules/clinic/posixmodule.c.h os_chdir _parser - -Modules/clinic/posixmodule.c.h os_chmod _parser - -Modules/clinic/posixmodule.c.h os_close _parser - -Modules/clinic/posixmodule.c.h os_device_encoding _parser - -Modules/clinic/posixmodule.c.h os_dup2 _parser - -Modules/clinic/posixmodule.c.h os_fspath _parser - -Modules/clinic/posixmodule.c.h os_fstat _parser - -Modules/clinic/posixmodule.c.h os_listdir _parser - -Modules/clinic/posixmodule.c.h os_lstat _parser - -Modules/clinic/posixmodule.c.h os_mkdir _parser - -Modules/clinic/posixmodule.c.h os_open _parser - -Modules/clinic/posixmodule.c.h os_remove _parser - -Modules/clinic/posixmodule.c.h os_rename _parser - -Modules/clinic/posixmodule.c.h os_replace _parser - -Modules/clinic/posixmodule.c.h os_rmdir _parser - -Modules/clinic/posixmodule.c.h os_scandir _parser - -Modules/clinic/posixmodule.c.h os_stat _parser - -Modules/clinic/posixmodule.c.h os_unlink _parser - -Modules/clinic/posixmodule.c.h os_utime _parser - Modules/clinic/pyexpat.c.h pyexpat_ParserCreate _parser - Modules/clinic/pyexpat.c.h pyexpat_xmlparser_ExternalEntityParserCreate _parser - Modules/clinic/pyexpat.c.h pyexpat_xmlparser_Parse _parser - @@ -1461,10 +1239,7 @@ Modules/clinic/zlibmodule.c.h zlib_compressobj _parser - Modules/clinic/zlibmodule.c.h zlib_decompress _parser - Modules/clinic/zlibmodule.c.h zlib_decompressobj _parser - -#----------------------- -# other - -# cached during module init - initialized once +# other - during module init Modules/_asynciomodule.c - asyncio_mod - Modules/_asynciomodule.c - traceback_extract_stack - Modules/_asynciomodule.c - asyncio_get_event_loop_policy - @@ -1479,7 +1254,10 @@ Modules/_zoneinfo.c - io_open - Modules/_zoneinfo.c - _tzpath_find_tzfile - Modules/_zoneinfo.c - _common_mod - -# other - initialized once +#----------------------- +# other + +# initialized once Modules/_ctypes/_ctypes.c - _unpickle - Modules/_ctypes/_ctypes.c PyCArrayType_from_ctype cache - Modules/_cursesmodule.c - ModDict - @@ -1501,8 +1279,6 @@ Modules/_decimal/_decimal.c - extended_context_template - Modules/_decimal/_decimal.c - round_map - Modules/_decimal/_decimal.c - Rational - Modules/_decimal/_decimal.c - SignalTuple - -Modules/_functoolsmodule.c - kwd_mark - -Modules/_io/_iomodule.c - _PyIO_empty_bytes - Modules/_json.c raise_errmsg JSONDecodeError - Modules/_sqlite/microprotocols.c - psyco_adapters - Modules/_sqlite/module.h - _pysqlite_converters - @@ -1512,12 +1288,8 @@ Modules/_ssl.c - lib_codes_to_names - # XXX This should have been found by the analyzer but wasn't: Modules/_ssl.c - _ssl_locks - Modules/_struct.c - cache - -Modules/_tracemalloc.c - tracemalloc_empty_traceback - Modules/arraymodule.c array_array___reduce_ex__ array_reconstructor - Modules/cjkcodecs/cjkcodecs.h getmultibytecodec cofunc - -Modules/signalmodule.c - DefaultHandler - -Modules/signalmodule.c - IgnoreHandler - -Modules/signalmodule.c - IntHandler - # state Modules/_asynciomodule.c - cached_running_holder - @@ -1533,12 +1305,6 @@ Modules/_tkinter.c - valInCmd - Modules/_tkinter.c - trbInCmd - Modules/_zoneinfo.c - TIMEDELTA_CACHE - Modules/_zoneinfo.c - ZONEINFO_WEAK_CACHE - -Modules/faulthandler.c - fatal_error - -Modules/faulthandler.c - thread - -Modules/faulthandler.c - user_signals - -Modules/faulthandler.c - stack - -Modules/faulthandler.c - old_stack - -Modules/signalmodule.c - Handlers - Modules/syslogmodule.c - S_ident_o - Modules/xxlimited_35.c - ErrorObject - @@ -1550,7 +1316,6 @@ Modules/xxlimited_35.c - ErrorObject - # initialized once # pre-allocated buffer -Modules/getbuildinfo.c Py_GetBuildInfo buildinfo - Modules/nismodule.c nisproc_maplist_2 res - Modules/pyexpat.c PyUnknownEncodingHandler template_buffer - @@ -1574,29 +1339,21 @@ Modules/_decimal/_decimal.c - _py_long_bit_length - Modules/_decimal/_decimal.c - _py_float_as_integer_ratio - Modules/_decimal/_decimal.c - _decimal_api - Modules/_elementtree.c - expat_capi - -Modules/_io/bufferedio.c _PyIO_trap_eintr eintr_int - Modules/_sqlite/module.h - _pysqlite_enable_callback_tracebacks - Modules/_sqlite/module.h - pysqlite_BaseTypeAdapted - Modules/_ssl.c - _ssl_locks_count - Modules/cjkcodecs/cjkcodecs.h - codec_list - Modules/cjkcodecs/cjkcodecs.h - mapping_list - Modules/getaddrinfo.c - gai_afdl - -Modules/posixmodule.c os_dup2_impl dup3_works - -Modules/posixmodule.c - structseq_new - -Modules/posixmodule.c - ticks_per_second - Modules/pyexpat.c PyInit_pyexpat capi - Modules/readline.c - libedit_append_replace_history_offset - Modules/readline.c - using_libedit_emulation - Modules/readline.c - libedit_history_start - Modules/resource.c - initialized - -Modules/signalmodule.c - initialized - Modules/socketmodule.c - accept4_works - Modules/socketmodule.c - sock_cloexec_works - Modules/socketmodule.c - PySocketModuleAPI - Modules/spwdmodule.c - initialized - -Modules/timemodule.c - initialized - -Modules/timemodule.c _PyTime_GetClockWithInfo initialized - -Modules/timemodule.c _PyTime_GetProcessTimeWithInfo ticks_per_second - #----------------------- # state @@ -1616,22 +1373,10 @@ Modules/_tkinter.c - command_mutex - Modules/_tkinter.c - HeadFHCD - Modules/_tkinter.c - stdin_ready - Modules/_tkinter.c - event_tstate - -Modules/_tracemalloc.c - allocators - -Modules/_tracemalloc.c - tables_lock - -Modules/_tracemalloc.c - tracemalloc_traced_memory - -Modules/_tracemalloc.c - tracemalloc_peak_traced_memory - -Modules/_tracemalloc.c - tracemalloc_filenames - -Modules/_tracemalloc.c - tracemalloc_traceback - -Modules/_tracemalloc.c - tracemalloc_tracebacks - -Modules/_tracemalloc.c - tracemalloc_traces - -Modules/_tracemalloc.c - tracemalloc_domains - -Modules/_tracemalloc.c - tracemalloc_reentrant_key - Modules/_xxsubinterpretersmodule.c - _globals - Modules/_zoneinfo.c - ZONEINFO_STRONG_CACHE - Modules/_zoneinfo.c - ZONEINFO_STRONG_CACHE_MAX_SIZE - Modules/_zoneinfo.c - NO_TTINFO - -Modules/faulthandler.c faulthandler_dump_traceback reentrant - -Modules/posixmodule.c - environ - Modules/readline.c - completer_word_break_characters - Modules/readline.c - _history_length - Modules/readline.c - should_auto_add_history - @@ -1640,7 +1385,5 @@ Modules/readline.c - sigwinch_ohandler - Modules/readline.c - completed_input_string - Modules/rotatingtree.c - random_stream - Modules/rotatingtree.c - random_value - -Modules/signalmodule.c - is_tripped - -Modules/signalmodule.c - wakeup - Modules/socketmodule.c - defaulttimeout - Modules/syslogmodule.c - S_log_open - diff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsv index 2eda605913ea3..9b8eff3685a68 100644 --- a/Tools/c-analyzer/cpython/ignored.tsv +++ b/Tools/c-analyzer/cpython/ignored.tsv @@ -1460,6 +1460,7 @@ Objects/obmalloc.c - _PyMem_Debug - Objects/obmalloc.c - _PyMem_Raw - Objects/obmalloc.c - _PyObject - Objects/obmalloc.c - usedpools - +Objects/typeobject.c - name_op - Objects/unicodeobject.c - stripfuncnames - Objects/unicodeobject.c - utf7_category - Objects/unicodeobject.c unicode_decode_call_errorhandler_wchar argparse - diff --git a/Tools/c-analyzer/table-file.py b/Tools/c-analyzer/table-file.py index ba3dd29055e03..3cc05cc9de777 100644 --- a/Tools/c-analyzer/table-file.py +++ b/Tools/c-analyzer/table-file.py @@ -92,6 +92,7 @@ def collate_sections(lines): # the commands def cmd_count_by_section(lines): + div = ' ' + '-' * 50 total = 0 def render_tree(root, depth=0): nonlocal total @@ -100,11 +101,14 @@ def render_tree(root, depth=0): subroot, rows, totalrows = data sectotal = f'({len(totalrows)})' if totalrows != rows else '' count = len(rows) if rows else '' + if depth == 0: + yield div yield f'{sectotal:>7} {count:>4} {indent}{name}' yield from render_tree(subroot, depth+1) total += len(rows) sections = collate_sections(lines) yield from render_tree(sections) + yield div yield f'(total: {total})' From webhook-mailer at python.org Wed Feb 9 22:37:27 2022 From: webhook-mailer at python.org (pablogsal) Date: Thu, 10 Feb 2022 03:37:27 -0000 Subject: [Python-checkins] bpo-46707: Avoid potential exponential backtracking in some syntax errors (GH-31241) Message-ID: https://github.com/python/cpython/commit/b71dc71905ab674ccaa4a56230d17a28f61c325c commit: b71dc71905ab674ccaa4a56230d17a28f61c325c branch: main author: Pablo Galindo Salgado committer: pablogsal date: 2022-02-10T03:37:17Z summary: bpo-46707: Avoid potential exponential backtracking in some syntax errors (GH-31241) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-10-03-13-18.bpo-46707.xeSEh0.rst M Grammar/python.gram M Lib/test/test_syntax.py M Parser/parser.c diff --git a/Grammar/python.gram b/Grammar/python.gram index c5a5f1b7fe20e..230d6ca6208e0 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -1095,13 +1095,15 @@ invalid_expression: RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Perhaps you forgot a comma?") } | a=disjunction 'if' b=disjunction !('else'|':') { RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "expected 'else' after 'if' expression") } +invalid_left_assignment_prefixes(memo): list|tuple|genexp|'True'|'None'|'False' + invalid_named_expression: | a=expression ':=' expression { RAISE_SYNTAX_ERROR_KNOWN_LOCATION( a, "cannot use assignment expressions with %s", _PyPegen_get_expr_name(a)) } | a=NAME '=' b=bitwise_or !('='|':=') { p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?") } - | !(list|tuple|genexp|'True'|'None'|'False') a=bitwise_or b='=' bitwise_or !('='|':=') { + | !invalid_left_assignment_prefixes a=bitwise_or b='=' bitwise_or !('='|':=') { p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "cannot assign to %s here. Maybe you meant '==' instead of '='?", _PyPegen_get_expr_name(a)) } diff --git a/Lib/test/test_syntax.py b/Lib/test/test_syntax.py index a6ff319af2ac8..8ddfe91a6e127 100644 --- a/Lib/test/test_syntax.py +++ b/Lib/test/test_syntax.py @@ -1770,6 +1770,14 @@ def test_error_on_parser_stack_overflow(self): with self.assertRaises(MemoryError): compile(source, "", mode) + @support.cpython_only + def test_deep_invalid_rule(self): + # Check that a very deep invalid rule in the PEG + # parser doesn't have exponential backtracking. + source = "d{{{{{{{{{{{{{{{{{{{{{{{{{```{{{{{{{ef f():y" + with self.assertRaises(SyntaxError): + compile(source, "", "exec") + def load_tests(loader, tests, pattern): tests.addTest(doctest.DocTestSuite()) diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-10-03-13-18.bpo-46707.xeSEh0.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-10-03-13-18.bpo-46707.xeSEh0.rst new file mode 100644 index 0000000000000..4b156c4d5f68b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-10-03-13-18.bpo-46707.xeSEh0.rst @@ -0,0 +1,2 @@ +Avoid potential exponential backtracking when producing some syntax errors +involving lots of brackets. Patch by Pablo Galindo. diff --git a/Parser/parser.c b/Parser/parser.c index 07a04c917430c..cf1a19dc23d89 100644 --- a/Parser/parser.c +++ b/Parser/parser.c @@ -257,267 +257,267 @@ static char *soft_keywords[] = { #define expression_without_invalid_type 1181 #define invalid_legacy_expression_type 1182 #define invalid_expression_type 1183 -#define invalid_named_expression_type 1184 -#define invalid_assignment_type 1185 -#define invalid_ann_assign_target_type 1186 -#define invalid_del_stmt_type 1187 -#define invalid_block_type 1188 -#define invalid_comprehension_type 1189 -#define invalid_dict_comprehension_type 1190 -#define invalid_parameters_type 1191 -#define invalid_parameters_helper_type 1192 -#define invalid_lambda_parameters_type 1193 -#define invalid_lambda_parameters_helper_type 1194 -#define invalid_star_etc_type 1195 -#define invalid_lambda_star_etc_type 1196 -#define invalid_double_type_comments_type 1197 -#define invalid_with_item_type 1198 -#define invalid_for_target_type 1199 -#define invalid_group_type 1200 -#define invalid_import_from_targets_type 1201 -#define invalid_with_stmt_type 1202 -#define invalid_with_stmt_indent_type 1203 -#define invalid_try_stmt_type 1204 -#define invalid_except_stmt_type 1205 -#define invalid_finally_stmt_type 1206 -#define invalid_except_stmt_indent_type 1207 -#define invalid_except_star_stmt_indent_type 1208 -#define invalid_match_stmt_type 1209 -#define invalid_case_block_type 1210 -#define invalid_as_pattern_type 1211 -#define invalid_class_pattern_type 1212 -#define invalid_class_argument_pattern_type 1213 -#define invalid_if_stmt_type 1214 -#define invalid_elif_stmt_type 1215 -#define invalid_else_stmt_type 1216 -#define invalid_while_stmt_type 1217 -#define invalid_for_stmt_type 1218 -#define invalid_def_raw_type 1219 -#define invalid_class_def_raw_type 1220 -#define invalid_double_starred_kvpairs_type 1221 -#define invalid_kvpair_type 1222 -#define _loop0_1_type 1223 -#define _loop0_2_type 1224 -#define _loop1_3_type 1225 -#define _loop0_5_type 1226 -#define _gather_4_type 1227 -#define _tmp_6_type 1228 -#define _tmp_7_type 1229 -#define _tmp_8_type 1230 -#define _tmp_9_type 1231 -#define _tmp_10_type 1232 -#define _tmp_11_type 1233 -#define _tmp_12_type 1234 -#define _tmp_13_type 1235 -#define _loop1_14_type 1236 -#define _tmp_15_type 1237 -#define _tmp_16_type 1238 -#define _tmp_17_type 1239 -#define _loop0_19_type 1240 -#define _gather_18_type 1241 -#define _loop0_21_type 1242 -#define _gather_20_type 1243 -#define _tmp_22_type 1244 -#define _tmp_23_type 1245 -#define _loop0_24_type 1246 -#define _loop1_25_type 1247 -#define _loop0_27_type 1248 -#define _gather_26_type 1249 -#define _tmp_28_type 1250 -#define _loop0_30_type 1251 -#define _gather_29_type 1252 -#define _tmp_31_type 1253 -#define _loop1_32_type 1254 -#define _tmp_33_type 1255 -#define _tmp_34_type 1256 -#define _tmp_35_type 1257 -#define _loop0_36_type 1258 -#define _loop0_37_type 1259 -#define _loop0_38_type 1260 -#define _loop1_39_type 1261 -#define _loop0_40_type 1262 -#define _loop1_41_type 1263 -#define _loop1_42_type 1264 -#define _loop1_43_type 1265 -#define _loop0_44_type 1266 -#define _loop1_45_type 1267 -#define _loop0_46_type 1268 -#define _loop1_47_type 1269 -#define _loop0_48_type 1270 -#define _loop1_49_type 1271 -#define _loop0_51_type 1272 -#define _gather_50_type 1273 -#define _loop0_53_type 1274 -#define _gather_52_type 1275 -#define _loop0_55_type 1276 -#define _gather_54_type 1277 -#define _loop0_57_type 1278 -#define _gather_56_type 1279 -#define _tmp_58_type 1280 -#define _loop1_59_type 1281 -#define _loop1_60_type 1282 -#define _tmp_61_type 1283 -#define _tmp_62_type 1284 -#define _loop1_63_type 1285 -#define _loop0_65_type 1286 -#define _gather_64_type 1287 -#define _tmp_66_type 1288 -#define _tmp_67_type 1289 -#define _tmp_68_type 1290 -#define _tmp_69_type 1291 -#define _loop0_71_type 1292 -#define _gather_70_type 1293 -#define _loop0_73_type 1294 -#define _gather_72_type 1295 -#define _tmp_74_type 1296 -#define _loop0_76_type 1297 -#define _gather_75_type 1298 -#define _loop0_78_type 1299 -#define _gather_77_type 1300 -#define _loop1_79_type 1301 -#define _loop1_80_type 1302 -#define _loop0_82_type 1303 -#define _gather_81_type 1304 -#define _loop1_83_type 1305 -#define _loop1_84_type 1306 -#define _loop1_85_type 1307 -#define _tmp_86_type 1308 -#define _loop0_88_type 1309 -#define _gather_87_type 1310 -#define _tmp_89_type 1311 -#define _tmp_90_type 1312 -#define _tmp_91_type 1313 -#define _tmp_92_type 1314 -#define _tmp_93_type 1315 -#define _loop0_94_type 1316 -#define _loop0_95_type 1317 -#define _loop0_96_type 1318 -#define _loop1_97_type 1319 -#define _loop0_98_type 1320 -#define _loop1_99_type 1321 -#define _loop1_100_type 1322 -#define _loop1_101_type 1323 -#define _loop0_102_type 1324 -#define _loop1_103_type 1325 -#define _loop0_104_type 1326 -#define _loop1_105_type 1327 -#define _loop0_106_type 1328 -#define _loop1_107_type 1329 -#define _loop1_108_type 1330 -#define _tmp_109_type 1331 -#define _loop0_111_type 1332 -#define _gather_110_type 1333 -#define _loop1_112_type 1334 -#define _loop0_113_type 1335 -#define _loop0_114_type 1336 -#define _tmp_115_type 1337 -#define _loop0_117_type 1338 -#define _gather_116_type 1339 -#define _tmp_118_type 1340 -#define _loop0_120_type 1341 -#define _gather_119_type 1342 -#define _loop0_122_type 1343 -#define _gather_121_type 1344 -#define _loop0_124_type 1345 -#define _gather_123_type 1346 -#define _loop0_126_type 1347 -#define _gather_125_type 1348 -#define _loop0_127_type 1349 -#define _loop0_129_type 1350 -#define _gather_128_type 1351 -#define _loop1_130_type 1352 -#define _tmp_131_type 1353 -#define _loop0_133_type 1354 -#define _gather_132_type 1355 -#define _loop0_135_type 1356 -#define _gather_134_type 1357 -#define _loop0_137_type 1358 -#define _gather_136_type 1359 -#define _loop0_139_type 1360 -#define _gather_138_type 1361 -#define _loop0_141_type 1362 -#define _gather_140_type 1363 -#define _tmp_142_type 1364 -#define _tmp_143_type 1365 -#define _tmp_144_type 1366 -#define _tmp_145_type 1367 -#define _tmp_146_type 1368 -#define _tmp_147_type 1369 -#define _tmp_148_type 1370 -#define _tmp_149_type 1371 -#define _tmp_150_type 1372 -#define _loop0_151_type 1373 -#define _loop0_152_type 1374 -#define _loop0_153_type 1375 -#define _tmp_154_type 1376 -#define _tmp_155_type 1377 -#define _tmp_156_type 1378 -#define _tmp_157_type 1379 -#define _loop0_158_type 1380 -#define _loop0_159_type 1381 -#define _loop1_160_type 1382 -#define _loop1_161_type 1383 -#define _loop0_162_type 1384 -#define _loop0_163_type 1385 -#define _loop0_165_type 1386 -#define _gather_164_type 1387 -#define _loop1_166_type 1388 -#define _tmp_167_type 1389 -#define _tmp_168_type 1390 -#define _tmp_169_type 1391 -#define _loop0_171_type 1392 -#define _gather_170_type 1393 -#define _loop0_173_type 1394 -#define _gather_172_type 1395 -#define _loop0_175_type 1396 -#define _gather_174_type 1397 -#define _loop0_177_type 1398 -#define _gather_176_type 1399 -#define _tmp_178_type 1400 -#define _loop0_179_type 1401 -#define _tmp_180_type 1402 -#define _loop0_181_type 1403 -#define _tmp_182_type 1404 -#define _tmp_183_type 1405 -#define _tmp_184_type 1406 -#define _tmp_185_type 1407 -#define _tmp_186_type 1408 -#define _tmp_187_type 1409 -#define _tmp_188_type 1410 -#define _tmp_189_type 1411 -#define _loop0_191_type 1412 -#define _gather_190_type 1413 -#define _tmp_192_type 1414 -#define _tmp_193_type 1415 -#define _tmp_194_type 1416 -#define _tmp_195_type 1417 -#define _tmp_196_type 1418 -#define _tmp_197_type 1419 -#define _tmp_198_type 1420 -#define _tmp_199_type 1421 -#define _tmp_200_type 1422 -#define _tmp_201_type 1423 -#define _tmp_202_type 1424 -#define _tmp_203_type 1425 -#define _tmp_204_type 1426 -#define _tmp_205_type 1427 -#define _tmp_206_type 1428 -#define _tmp_207_type 1429 -#define _tmp_208_type 1430 -#define _tmp_209_type 1431 -#define _tmp_210_type 1432 -#define _tmp_211_type 1433 -#define _tmp_212_type 1434 -#define _tmp_213_type 1435 -#define _tmp_214_type 1436 -#define _tmp_215_type 1437 -#define _tmp_216_type 1438 -#define _tmp_217_type 1439 -#define _tmp_218_type 1440 -#define _tmp_219_type 1441 -#define _tmp_220_type 1442 -#define _loop1_221_type 1443 -#define _loop1_222_type 1444 +#define invalid_left_assignment_prefixes_type 1184 +#define invalid_named_expression_type 1185 +#define invalid_assignment_type 1186 +#define invalid_ann_assign_target_type 1187 +#define invalid_del_stmt_type 1188 +#define invalid_block_type 1189 +#define invalid_comprehension_type 1190 +#define invalid_dict_comprehension_type 1191 +#define invalid_parameters_type 1192 +#define invalid_parameters_helper_type 1193 +#define invalid_lambda_parameters_type 1194 +#define invalid_lambda_parameters_helper_type 1195 +#define invalid_star_etc_type 1196 +#define invalid_lambda_star_etc_type 1197 +#define invalid_double_type_comments_type 1198 +#define invalid_with_item_type 1199 +#define invalid_for_target_type 1200 +#define invalid_group_type 1201 +#define invalid_import_from_targets_type 1202 +#define invalid_with_stmt_type 1203 +#define invalid_with_stmt_indent_type 1204 +#define invalid_try_stmt_type 1205 +#define invalid_except_stmt_type 1206 +#define invalid_finally_stmt_type 1207 +#define invalid_except_stmt_indent_type 1208 +#define invalid_except_star_stmt_indent_type 1209 +#define invalid_match_stmt_type 1210 +#define invalid_case_block_type 1211 +#define invalid_as_pattern_type 1212 +#define invalid_class_pattern_type 1213 +#define invalid_class_argument_pattern_type 1214 +#define invalid_if_stmt_type 1215 +#define invalid_elif_stmt_type 1216 +#define invalid_else_stmt_type 1217 +#define invalid_while_stmt_type 1218 +#define invalid_for_stmt_type 1219 +#define invalid_def_raw_type 1220 +#define invalid_class_def_raw_type 1221 +#define invalid_double_starred_kvpairs_type 1222 +#define invalid_kvpair_type 1223 +#define _loop0_1_type 1224 +#define _loop0_2_type 1225 +#define _loop1_3_type 1226 +#define _loop0_5_type 1227 +#define _gather_4_type 1228 +#define _tmp_6_type 1229 +#define _tmp_7_type 1230 +#define _tmp_8_type 1231 +#define _tmp_9_type 1232 +#define _tmp_10_type 1233 +#define _tmp_11_type 1234 +#define _tmp_12_type 1235 +#define _tmp_13_type 1236 +#define _loop1_14_type 1237 +#define _tmp_15_type 1238 +#define _tmp_16_type 1239 +#define _tmp_17_type 1240 +#define _loop0_19_type 1241 +#define _gather_18_type 1242 +#define _loop0_21_type 1243 +#define _gather_20_type 1244 +#define _tmp_22_type 1245 +#define _tmp_23_type 1246 +#define _loop0_24_type 1247 +#define _loop1_25_type 1248 +#define _loop0_27_type 1249 +#define _gather_26_type 1250 +#define _tmp_28_type 1251 +#define _loop0_30_type 1252 +#define _gather_29_type 1253 +#define _tmp_31_type 1254 +#define _loop1_32_type 1255 +#define _tmp_33_type 1256 +#define _tmp_34_type 1257 +#define _tmp_35_type 1258 +#define _loop0_36_type 1259 +#define _loop0_37_type 1260 +#define _loop0_38_type 1261 +#define _loop1_39_type 1262 +#define _loop0_40_type 1263 +#define _loop1_41_type 1264 +#define _loop1_42_type 1265 +#define _loop1_43_type 1266 +#define _loop0_44_type 1267 +#define _loop1_45_type 1268 +#define _loop0_46_type 1269 +#define _loop1_47_type 1270 +#define _loop0_48_type 1271 +#define _loop1_49_type 1272 +#define _loop0_51_type 1273 +#define _gather_50_type 1274 +#define _loop0_53_type 1275 +#define _gather_52_type 1276 +#define _loop0_55_type 1277 +#define _gather_54_type 1278 +#define _loop0_57_type 1279 +#define _gather_56_type 1280 +#define _tmp_58_type 1281 +#define _loop1_59_type 1282 +#define _loop1_60_type 1283 +#define _tmp_61_type 1284 +#define _tmp_62_type 1285 +#define _loop1_63_type 1286 +#define _loop0_65_type 1287 +#define _gather_64_type 1288 +#define _tmp_66_type 1289 +#define _tmp_67_type 1290 +#define _tmp_68_type 1291 +#define _tmp_69_type 1292 +#define _loop0_71_type 1293 +#define _gather_70_type 1294 +#define _loop0_73_type 1295 +#define _gather_72_type 1296 +#define _tmp_74_type 1297 +#define _loop0_76_type 1298 +#define _gather_75_type 1299 +#define _loop0_78_type 1300 +#define _gather_77_type 1301 +#define _loop1_79_type 1302 +#define _loop1_80_type 1303 +#define _loop0_82_type 1304 +#define _gather_81_type 1305 +#define _loop1_83_type 1306 +#define _loop1_84_type 1307 +#define _loop1_85_type 1308 +#define _tmp_86_type 1309 +#define _loop0_88_type 1310 +#define _gather_87_type 1311 +#define _tmp_89_type 1312 +#define _tmp_90_type 1313 +#define _tmp_91_type 1314 +#define _tmp_92_type 1315 +#define _tmp_93_type 1316 +#define _loop0_94_type 1317 +#define _loop0_95_type 1318 +#define _loop0_96_type 1319 +#define _loop1_97_type 1320 +#define _loop0_98_type 1321 +#define _loop1_99_type 1322 +#define _loop1_100_type 1323 +#define _loop1_101_type 1324 +#define _loop0_102_type 1325 +#define _loop1_103_type 1326 +#define _loop0_104_type 1327 +#define _loop1_105_type 1328 +#define _loop0_106_type 1329 +#define _loop1_107_type 1330 +#define _loop1_108_type 1331 +#define _tmp_109_type 1332 +#define _loop0_111_type 1333 +#define _gather_110_type 1334 +#define _loop1_112_type 1335 +#define _loop0_113_type 1336 +#define _loop0_114_type 1337 +#define _tmp_115_type 1338 +#define _loop0_117_type 1339 +#define _gather_116_type 1340 +#define _tmp_118_type 1341 +#define _loop0_120_type 1342 +#define _gather_119_type 1343 +#define _loop0_122_type 1344 +#define _gather_121_type 1345 +#define _loop0_124_type 1346 +#define _gather_123_type 1347 +#define _loop0_126_type 1348 +#define _gather_125_type 1349 +#define _loop0_127_type 1350 +#define _loop0_129_type 1351 +#define _gather_128_type 1352 +#define _loop1_130_type 1353 +#define _tmp_131_type 1354 +#define _loop0_133_type 1355 +#define _gather_132_type 1356 +#define _loop0_135_type 1357 +#define _gather_134_type 1358 +#define _loop0_137_type 1359 +#define _gather_136_type 1360 +#define _loop0_139_type 1361 +#define _gather_138_type 1362 +#define _loop0_141_type 1363 +#define _gather_140_type 1364 +#define _tmp_142_type 1365 +#define _tmp_143_type 1366 +#define _tmp_144_type 1367 +#define _tmp_145_type 1368 +#define _tmp_146_type 1369 +#define _tmp_147_type 1370 +#define _tmp_148_type 1371 +#define _tmp_149_type 1372 +#define _loop0_150_type 1373 +#define _loop0_151_type 1374 +#define _loop0_152_type 1375 +#define _tmp_153_type 1376 +#define _tmp_154_type 1377 +#define _tmp_155_type 1378 +#define _tmp_156_type 1379 +#define _loop0_157_type 1380 +#define _loop0_158_type 1381 +#define _loop1_159_type 1382 +#define _loop1_160_type 1383 +#define _loop0_161_type 1384 +#define _loop0_162_type 1385 +#define _loop0_164_type 1386 +#define _gather_163_type 1387 +#define _loop1_165_type 1388 +#define _tmp_166_type 1389 +#define _tmp_167_type 1390 +#define _tmp_168_type 1391 +#define _loop0_170_type 1392 +#define _gather_169_type 1393 +#define _loop0_172_type 1394 +#define _gather_171_type 1395 +#define _loop0_174_type 1396 +#define _gather_173_type 1397 +#define _loop0_176_type 1398 +#define _gather_175_type 1399 +#define _tmp_177_type 1400 +#define _loop0_178_type 1401 +#define _tmp_179_type 1402 +#define _loop0_180_type 1403 +#define _tmp_181_type 1404 +#define _tmp_182_type 1405 +#define _tmp_183_type 1406 +#define _tmp_184_type 1407 +#define _tmp_185_type 1408 +#define _tmp_186_type 1409 +#define _tmp_187_type 1410 +#define _tmp_188_type 1411 +#define _loop0_190_type 1412 +#define _gather_189_type 1413 +#define _tmp_191_type 1414 +#define _tmp_192_type 1415 +#define _tmp_193_type 1416 +#define _tmp_194_type 1417 +#define _tmp_195_type 1418 +#define _tmp_196_type 1419 +#define _tmp_197_type 1420 +#define _tmp_198_type 1421 +#define _tmp_199_type 1422 +#define _tmp_200_type 1423 +#define _tmp_201_type 1424 +#define _tmp_202_type 1425 +#define _tmp_203_type 1426 +#define _tmp_204_type 1427 +#define _tmp_205_type 1428 +#define _tmp_206_type 1429 +#define _tmp_207_type 1430 +#define _tmp_208_type 1431 +#define _tmp_209_type 1432 +#define _tmp_210_type 1433 +#define _tmp_211_type 1434 +#define _tmp_212_type 1435 +#define _tmp_213_type 1436 +#define _tmp_214_type 1437 +#define _tmp_215_type 1438 +#define _tmp_216_type 1439 +#define _tmp_217_type 1440 +#define _tmp_218_type 1441 +#define _tmp_219_type 1442 +#define _loop1_220_type 1443 +#define _loop1_221_type 1444 static mod_ty file_rule(Parser *p); static mod_ty interactive_rule(Parser *p); @@ -703,6 +703,7 @@ static void *invalid_kwarg_rule(Parser *p); static expr_ty expression_without_invalid_rule(Parser *p); static void *invalid_legacy_expression_rule(Parser *p); static void *invalid_expression_rule(Parser *p); +static void *invalid_left_assignment_prefixes_rule(Parser *p); static void *invalid_named_expression_rule(Parser *p); static void *invalid_assignment_rule(Parser *p); static expr_ty invalid_ann_assign_target_rule(Parser *p); @@ -891,38 +892,38 @@ static void *_tmp_146_rule(Parser *p); static void *_tmp_147_rule(Parser *p); static void *_tmp_148_rule(Parser *p); static void *_tmp_149_rule(Parser *p); -static void *_tmp_150_rule(Parser *p); +static asdl_seq *_loop0_150_rule(Parser *p); static asdl_seq *_loop0_151_rule(Parser *p); static asdl_seq *_loop0_152_rule(Parser *p); -static asdl_seq *_loop0_153_rule(Parser *p); +static void *_tmp_153_rule(Parser *p); static void *_tmp_154_rule(Parser *p); static void *_tmp_155_rule(Parser *p); static void *_tmp_156_rule(Parser *p); -static void *_tmp_157_rule(Parser *p); +static asdl_seq *_loop0_157_rule(Parser *p); static asdl_seq *_loop0_158_rule(Parser *p); -static asdl_seq *_loop0_159_rule(Parser *p); +static asdl_seq *_loop1_159_rule(Parser *p); static asdl_seq *_loop1_160_rule(Parser *p); -static asdl_seq *_loop1_161_rule(Parser *p); +static asdl_seq *_loop0_161_rule(Parser *p); static asdl_seq *_loop0_162_rule(Parser *p); -static asdl_seq *_loop0_163_rule(Parser *p); -static asdl_seq *_loop0_165_rule(Parser *p); -static asdl_seq *_gather_164_rule(Parser *p); -static asdl_seq *_loop1_166_rule(Parser *p); +static asdl_seq *_loop0_164_rule(Parser *p); +static asdl_seq *_gather_163_rule(Parser *p); +static asdl_seq *_loop1_165_rule(Parser *p); +static void *_tmp_166_rule(Parser *p); static void *_tmp_167_rule(Parser *p); static void *_tmp_168_rule(Parser *p); -static void *_tmp_169_rule(Parser *p); -static asdl_seq *_loop0_171_rule(Parser *p); -static asdl_seq *_gather_170_rule(Parser *p); -static asdl_seq *_loop0_173_rule(Parser *p); -static asdl_seq *_gather_172_rule(Parser *p); -static asdl_seq *_loop0_175_rule(Parser *p); -static asdl_seq *_gather_174_rule(Parser *p); -static asdl_seq *_loop0_177_rule(Parser *p); -static asdl_seq *_gather_176_rule(Parser *p); -static void *_tmp_178_rule(Parser *p); -static asdl_seq *_loop0_179_rule(Parser *p); -static void *_tmp_180_rule(Parser *p); -static asdl_seq *_loop0_181_rule(Parser *p); +static asdl_seq *_loop0_170_rule(Parser *p); +static asdl_seq *_gather_169_rule(Parser *p); +static asdl_seq *_loop0_172_rule(Parser *p); +static asdl_seq *_gather_171_rule(Parser *p); +static asdl_seq *_loop0_174_rule(Parser *p); +static asdl_seq *_gather_173_rule(Parser *p); +static asdl_seq *_loop0_176_rule(Parser *p); +static asdl_seq *_gather_175_rule(Parser *p); +static void *_tmp_177_rule(Parser *p); +static asdl_seq *_loop0_178_rule(Parser *p); +static void *_tmp_179_rule(Parser *p); +static asdl_seq *_loop0_180_rule(Parser *p); +static void *_tmp_181_rule(Parser *p); static void *_tmp_182_rule(Parser *p); static void *_tmp_183_rule(Parser *p); static void *_tmp_184_rule(Parser *p); @@ -930,9 +931,9 @@ static void *_tmp_185_rule(Parser *p); static void *_tmp_186_rule(Parser *p); static void *_tmp_187_rule(Parser *p); static void *_tmp_188_rule(Parser *p); -static void *_tmp_189_rule(Parser *p); -static asdl_seq *_loop0_191_rule(Parser *p); -static asdl_seq *_gather_190_rule(Parser *p); +static asdl_seq *_loop0_190_rule(Parser *p); +static asdl_seq *_gather_189_rule(Parser *p); +static void *_tmp_191_rule(Parser *p); static void *_tmp_192_rule(Parser *p); static void *_tmp_193_rule(Parser *p); static void *_tmp_194_rule(Parser *p); @@ -961,9 +962,8 @@ static void *_tmp_216_rule(Parser *p); static void *_tmp_217_rule(Parser *p); static void *_tmp_218_rule(Parser *p); static void *_tmp_219_rule(Parser *p); -static void *_tmp_220_rule(Parser *p); +static asdl_seq *_loop1_220_rule(Parser *p); static asdl_seq *_loop1_221_rule(Parser *p); -static asdl_seq *_loop1_222_rule(Parser *p); // file: statements? $ @@ -19142,10 +19142,149 @@ invalid_expression_rule(Parser *p) return _res; } +// invalid_left_assignment_prefixes: list | tuple | genexp | 'True' | 'None' | 'False' +static void * +invalid_left_assignment_prefixes_rule(Parser *p) +{ + if (p->level++ == MAXSTACK) { + p->error_indicator = 1; + PyErr_NoMemory(); + } + if (p->error_indicator) { + p->level--; + return NULL; + } + void * _res = NULL; + if (_PyPegen_is_memoized(p, invalid_left_assignment_prefixes_type, &_res)) { + p->level--; + return _res; + } + int _mark = p->mark; + { // list + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "list")); + expr_ty list_var; + if ( + (list_var = list_rule(p)) // list + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "list")); + _res = list_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "list")); + } + { // tuple + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "tuple")); + expr_ty tuple_var; + if ( + (tuple_var = tuple_rule(p)) // tuple + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "tuple")); + _res = tuple_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "tuple")); + } + { // genexp + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "genexp")); + expr_ty genexp_var; + if ( + (genexp_var = genexp_rule(p)) // genexp + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "genexp")); + _res = genexp_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "genexp")); + } + { // 'True' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 600)) // token='True' + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'True'")); + } + { // 'None' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 601)) // token='None' + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'None'")); + } + { // 'False' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 602)) // token='False' + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'False'")); + } + _res = NULL; + done: + _PyPegen_insert_memo(p, _mark, invalid_left_assignment_prefixes_type, _res); + p->level--; + return _res; +} + // invalid_named_expression: // | expression ':=' expression // | NAME '=' bitwise_or !('=' | ':=') -// | !(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=') +// | !invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=') static void * invalid_named_expression_rule(Parser *p) { @@ -19221,17 +19360,17 @@ invalid_named_expression_rule(Parser *p) D(fprintf(stderr, "%*c%s invalid_named_expression[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "NAME '=' bitwise_or !('=' | ':=')")); } - { // !(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=') + { // !invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=') if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_named_expression[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); + D(fprintf(stderr, "%*c> invalid_named_expression[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); expr_ty a; Token * b; expr_ty bitwise_or_var; if ( - _PyPegen_lookahead(0, _tmp_149_rule, p) + _PyPegen_lookahead(0, invalid_left_assignment_prefixes_rule, p) && (a = bitwise_or_rule(p)) // bitwise_or && @@ -19239,10 +19378,10 @@ invalid_named_expression_rule(Parser *p) && (bitwise_or_var = bitwise_or_rule(p)) // bitwise_or && - _PyPegen_lookahead(0, _tmp_150_rule, p) + _PyPegen_lookahead(0, _tmp_149_rule, p) ) { - D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); + D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); _res = p -> in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_LOCATION ( a , "cannot assign to %s here. Maybe you meant '==' instead of '='?" , _PyPegen_get_expr_name ( a ) ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -19253,7 +19392,7 @@ invalid_named_expression_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_named_expression[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); } _res = NULL; done: @@ -19319,7 +19458,7 @@ invalid_assignment_rule(Parser *p) D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expression ',' star_named_expressions* ':' expression")); Token * _literal; Token * _literal_1; - asdl_seq * _loop0_151_var; + asdl_seq * _loop0_150_var; expr_ty a; expr_ty expression_var; if ( @@ -19327,7 +19466,7 @@ invalid_assignment_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_loop0_151_var = _loop0_151_rule(p)) // star_named_expressions* + (_loop0_150_var = _loop0_150_rule(p)) // star_named_expressions* && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -19384,10 +19523,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* star_expressions '='")); Token * _literal; - asdl_seq * _loop0_152_var; + asdl_seq * _loop0_151_var; expr_ty a; if ( - (_loop0_152_var = _loop0_152_rule(p)) // ((star_targets '='))* + (_loop0_151_var = _loop0_151_rule(p)) // ((star_targets '='))* && (a = star_expressions_rule(p)) // star_expressions && @@ -19414,10 +19553,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* yield_expr '='")); Token * _literal; - asdl_seq * _loop0_153_var; + asdl_seq * _loop0_152_var; expr_ty a; if ( - (_loop0_153_var = _loop0_153_rule(p)) // ((star_targets '='))* + (_loop0_152_var = _loop0_152_rule(p)) // ((star_targets '='))* && (a = yield_expr_rule(p)) // yield_expr && @@ -19443,7 +19582,7 @@ invalid_assignment_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); - void *_tmp_154_var; + void *_tmp_153_var; expr_ty a; AugOperator* augassign_var; if ( @@ -19451,7 +19590,7 @@ invalid_assignment_rule(Parser *p) && (augassign_var = augassign_rule(p)) // augassign && - (_tmp_154_var = _tmp_154_rule(p)) // yield_expr | star_expressions + (_tmp_153_var = _tmp_153_rule(p)) // yield_expr | star_expressions ) { D(fprintf(stderr, "%*c+ invalid_assignment[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); @@ -19677,11 +19816,11 @@ invalid_comprehension_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '(' | '{') starred_expression for_if_clauses")); - void *_tmp_155_var; + void *_tmp_154_var; expr_ty a; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_155_var = _tmp_155_rule(p)) // '[' | '(' | '{' + (_tmp_154_var = _tmp_154_rule(p)) // '[' | '(' | '{' && (a = starred_expression_rule(p)) // starred_expression && @@ -19708,12 +19847,12 @@ invalid_comprehension_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '{') star_named_expression ',' star_named_expressions for_if_clauses")); Token * _literal; - void *_tmp_156_var; + void *_tmp_155_var; expr_ty a; asdl_expr_seq* b; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_156_var = _tmp_156_rule(p)) // '[' | '{' + (_tmp_155_var = _tmp_155_rule(p)) // '[' | '{' && (a = star_named_expression_rule(p)) // star_named_expression && @@ -19743,12 +19882,12 @@ invalid_comprehension_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '{') star_named_expression ',' for_if_clauses")); - void *_tmp_157_var; + void *_tmp_156_var; expr_ty a; Token * b; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_157_var = _tmp_157_rule(p)) // '[' | '{' + (_tmp_156_var = _tmp_156_rule(p)) // '[' | '{' && (a = star_named_expression_rule(p)) // star_named_expression && @@ -19854,11 +19993,11 @@ invalid_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default* invalid_parameters_helper param_no_default")); - asdl_seq * _loop0_158_var; + asdl_seq * _loop0_157_var; arg_ty a; void *invalid_parameters_helper_var; if ( - (_loop0_158_var = _loop0_158_rule(p)) // param_no_default* + (_loop0_157_var = _loop0_157_rule(p)) // param_no_default* && (invalid_parameters_helper_var = invalid_parameters_helper_rule(p)) // invalid_parameters_helper && @@ -19884,18 +20023,18 @@ invalid_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default* '(' param_no_default+ ','? ')'")); - asdl_seq * _loop0_159_var; - asdl_seq * _loop1_160_var; + asdl_seq * _loop0_158_var; + asdl_seq * _loop1_159_var; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings Token * a; Token * b; if ( - (_loop0_159_var = _loop0_159_rule(p)) // param_no_default* + (_loop0_158_var = _loop0_158_rule(p)) // param_no_default* && (a = _PyPegen_expect_token(p, 7)) // token='(' && - (_loop1_160_var = _loop1_160_rule(p)) // param_no_default+ + (_loop1_159_var = _loop1_159_rule(p)) // param_no_default+ && (_opt_var = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -19965,13 +20104,13 @@ invalid_parameters_helper_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters_helper[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - asdl_seq * _loop1_161_var; + asdl_seq * _loop1_160_var; if ( - (_loop1_161_var = _loop1_161_rule(p)) // param_with_default+ + (_loop1_160_var = _loop1_160_rule(p)) // param_with_default+ ) { D(fprintf(stderr, "%*c+ invalid_parameters_helper[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - _res = _loop1_161_var; + _res = _loop1_160_var; goto done; } p->mark = _mark; @@ -20006,11 +20145,11 @@ invalid_lambda_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default* invalid_lambda_parameters_helper lambda_param_no_default")); - asdl_seq * _loop0_162_var; + asdl_seq * _loop0_161_var; arg_ty a; void *invalid_lambda_parameters_helper_var; if ( - (_loop0_162_var = _loop0_162_rule(p)) // lambda_param_no_default* + (_loop0_161_var = _loop0_161_rule(p)) // lambda_param_no_default* && (invalid_lambda_parameters_helper_var = invalid_lambda_parameters_helper_rule(p)) // invalid_lambda_parameters_helper && @@ -20036,18 +20175,18 @@ invalid_lambda_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default* '(' ','.lambda_param+ ','? ')'")); - asdl_seq * _gather_164_var; - asdl_seq * _loop0_163_var; + asdl_seq * _gather_163_var; + asdl_seq * _loop0_162_var; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings Token * a; Token * b; if ( - (_loop0_163_var = _loop0_163_rule(p)) // lambda_param_no_default* + (_loop0_162_var = _loop0_162_rule(p)) // lambda_param_no_default* && (a = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_164_var = _gather_164_rule(p)) // ','.lambda_param+ + (_gather_163_var = _gather_163_rule(p)) // ','.lambda_param+ && (_opt_var = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20119,13 +20258,13 @@ invalid_lambda_parameters_helper_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters_helper[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - asdl_seq * _loop1_166_var; + asdl_seq * _loop1_165_var; if ( - (_loop1_166_var = _loop1_166_rule(p)) // lambda_param_with_default+ + (_loop1_165_var = _loop1_165_rule(p)) // lambda_param_with_default+ ) { D(fprintf(stderr, "%*c+ invalid_lambda_parameters_helper[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - _res = _loop1_166_var; + _res = _loop1_165_var; goto done; } p->mark = _mark; @@ -20158,12 +20297,12 @@ invalid_star_etc_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); - void *_tmp_167_var; + void *_tmp_166_var; Token * a; if ( (a = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_167_var = _tmp_167_rule(p)) // ')' | ',' (')' | '**') + (_tmp_166_var = _tmp_166_rule(p)) // ')' | ',' (')' | '**') ) { D(fprintf(stderr, "%*c+ invalid_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); @@ -20236,11 +20375,11 @@ invalid_lambda_star_etc_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_lambda_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); Token * _literal; - void *_tmp_168_var; + void *_tmp_167_var; if ( (_literal = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_168_var = _tmp_168_rule(p)) // ':' | ',' (':' | '**') + (_tmp_167_var = _tmp_167_rule(p)) // ':' | ',' (':' | '**') ) { D(fprintf(stderr, "%*c+ invalid_lambda_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); @@ -20348,7 +20487,7 @@ invalid_with_item_rule(Parser *p) && (a = expression_rule(p)) // expression && - _PyPegen_lookahead(1, _tmp_169_rule, p) + _PyPegen_lookahead(1, _tmp_168_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_with_item[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression 'as' expression &(',' | ')' | ':')")); @@ -20576,7 +20715,7 @@ invalid_with_stmt_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ &&':'")); - asdl_seq * _gather_170_var; + asdl_seq * _gather_169_var; Token * _keyword; Token * _literal; void *_opt_var; @@ -20586,13 +20725,13 @@ invalid_with_stmt_rule(Parser *p) && (_keyword = _PyPegen_expect_token(p, 612)) // token='with' && - (_gather_170_var = _gather_170_rule(p)) // ','.(expression ['as' star_target])+ + (_gather_169_var = _gather_169_rule(p)) // ','.(expression ['as' star_target])+ && (_literal = _PyPegen_expect_forced_token(p, 11, ":")) // forced_token=':' ) { D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ &&':'")); - _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _gather_170_var, _literal); + _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _gather_169_var, _literal); goto done; } p->mark = _mark; @@ -20605,7 +20744,7 @@ invalid_with_stmt_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' &&':'")); - asdl_seq * _gather_172_var; + asdl_seq * _gather_171_var; Token * _keyword; Token * _literal; Token * _literal_1; @@ -20621,7 +20760,7 @@ invalid_with_stmt_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_172_var = _gather_172_rule(p)) // ','.(expressions ['as' star_target])+ + (_gather_171_var = _gather_171_rule(p)) // ','.(expressions ['as' star_target])+ && (_opt_var_1 = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20631,7 +20770,7 @@ invalid_with_stmt_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' &&':'")); - _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _literal, _gather_172_var, _opt_var_1, _literal_1, _literal_2); + _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _literal, _gather_171_var, _opt_var_1, _literal_1, _literal_2); goto done; } p->mark = _mark; @@ -20666,7 +20805,7 @@ invalid_with_stmt_indent_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); - asdl_seq * _gather_174_var; + asdl_seq * _gather_173_var; Token * _literal; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings @@ -20677,7 +20816,7 @@ invalid_with_stmt_indent_rule(Parser *p) && (a = _PyPegen_expect_token(p, 612)) // token='with' && - (_gather_174_var = _gather_174_rule(p)) // ','.(expression ['as' star_target])+ + (_gather_173_var = _gather_173_rule(p)) // ','.(expression ['as' star_target])+ && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -20705,7 +20844,7 @@ invalid_with_stmt_indent_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); - asdl_seq * _gather_176_var; + asdl_seq * _gather_175_var; Token * _literal; Token * _literal_1; Token * _literal_2; @@ -20722,7 +20861,7 @@ invalid_with_stmt_indent_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_176_var = _gather_176_rule(p)) // ','.(expressions ['as' star_target])+ + (_gather_175_var = _gather_175_rule(p)) // ','.(expressions ['as' star_target])+ && (_opt_var_1 = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20819,7 +20958,7 @@ invalid_try_stmt_rule(Parser *p) && (block_var = block_rule(p)) // block && - _PyPegen_lookahead(0, _tmp_178_rule, p) + _PyPegen_lookahead(0, _tmp_177_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_try_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'try' ':' block !('except' | 'finally')")); @@ -20843,19 +20982,19 @@ invalid_try_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> invalid_try_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'try' ':' block* ((except_block+ except_star_block) | (except_star_block+ except_block)) block*")); Token * _keyword; Token * _literal; - asdl_seq * _loop0_179_var; - asdl_seq * _loop0_181_var; - void *_tmp_180_var; + asdl_seq * _loop0_178_var; + asdl_seq * _loop0_180_var; + void *_tmp_179_var; if ( (_keyword = _PyPegen_expect_token(p, 618)) // token='try' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && - (_loop0_179_var = _loop0_179_rule(p)) // block* + (_loop0_178_var = _loop0_178_rule(p)) // block* && - (_tmp_180_var = _tmp_180_rule(p)) // (except_block+ except_star_block) | (except_star_block+ except_block) + (_tmp_179_var = _tmp_179_rule(p)) // (except_block+ except_star_block) | (except_star_block+ except_block) && - (_loop0_181_var = _loop0_181_rule(p)) // block* + (_loop0_180_var = _loop0_180_rule(p)) // block* ) { D(fprintf(stderr, "%*c+ invalid_try_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'try' ':' block* ((except_block+ except_star_block) | (except_star_block+ except_block)) block*")); @@ -20921,7 +21060,7 @@ invalid_except_stmt_rule(Parser *p) && (expressions_var = expressions_rule(p)) // expressions && - (_opt_var_1 = _tmp_182_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var_1 = _tmp_181_rule(p), !p->error_indicator) // ['as' NAME] && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' ) @@ -20959,7 +21098,7 @@ invalid_except_stmt_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var_1 = _tmp_183_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var_1 = _tmp_182_rule(p), !p->error_indicator) // ['as' NAME] && (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) @@ -21011,14 +21150,14 @@ invalid_except_stmt_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_except_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except' '*' (NEWLINE | ':')")); Token * _literal; - void *_tmp_184_var; + void *_tmp_183_var; Token * a; if ( (a = _PyPegen_expect_token(p, 629)) // token='except' && (_literal = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_184_var = _tmp_184_rule(p)) // NEWLINE | ':' + (_tmp_183_var = _tmp_183_rule(p)) // NEWLINE | ':' ) { D(fprintf(stderr, "%*c+ invalid_except_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except' '*' (NEWLINE | ':')")); @@ -21125,7 +21264,7 @@ invalid_except_stmt_indent_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_185_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_184_rule(p), !p->error_indicator) // ['as' NAME] && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21220,7 +21359,7 @@ invalid_except_star_stmt_indent_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_186_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_185_rule(p), !p->error_indicator) // ['as' NAME] && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21587,7 +21726,7 @@ invalid_class_argument_pattern_rule(Parser *p) asdl_pattern_seq* a; asdl_seq* keyword_patterns_var; if ( - (_opt_var = _tmp_187_rule(p), !p->error_indicator) // [positional_patterns ','] + (_opt_var = _tmp_186_rule(p), !p->error_indicator) // [positional_patterns ','] && (keyword_patterns_var = keyword_patterns_rule(p)) // keyword_patterns && @@ -22039,7 +22178,7 @@ invalid_def_raw_rule(Parser *p) && (_literal_1 = _PyPegen_expect_token(p, 8)) // token=')' && - (_opt_var_2 = _tmp_188_rule(p), !p->error_indicator) // ['->' expression] + (_opt_var_2 = _tmp_187_rule(p), !p->error_indicator) // ['->' expression] && (_literal_2 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -22098,7 +22237,7 @@ invalid_class_def_raw_rule(Parser *p) && (name_var = _PyPegen_name_token(p)) // NAME && - (_opt_var = _tmp_189_rule(p), !p->error_indicator) // ['(' arguments? ')'] + (_opt_var = _tmp_188_rule(p), !p->error_indicator) // ['(' arguments? ')'] && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -22149,11 +22288,11 @@ invalid_double_starred_kvpairs_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_double_starred_kvpairs[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.double_starred_kvpair+ ',' invalid_kvpair")); - asdl_seq * _gather_190_var; + asdl_seq * _gather_189_var; Token * _literal; void *invalid_kvpair_var; if ( - (_gather_190_var = _gather_190_rule(p)) // ','.double_starred_kvpair+ + (_gather_189_var = _gather_189_rule(p)) // ','.double_starred_kvpair+ && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && @@ -22161,7 +22300,7 @@ invalid_double_starred_kvpairs_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_double_starred_kvpairs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.double_starred_kvpair+ ',' invalid_kvpair")); - _res = _PyPegen_dummy_name(p, _gather_190_var, _literal, invalid_kvpair_var); + _res = _PyPegen_dummy_name(p, _gather_189_var, _literal, invalid_kvpair_var); goto done; } p->mark = _mark; @@ -22214,7 +22353,7 @@ invalid_double_starred_kvpairs_rule(Parser *p) && (a = _PyPegen_expect_token(p, 11)) // token=':' && - _PyPegen_lookahead(1, _tmp_192_rule, p) + _PyPegen_lookahead(1, _tmp_191_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_double_starred_kvpairs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ':' &('}' | ',')")); @@ -23176,12 +23315,12 @@ _loop1_14_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_14[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_193_var; + void *_tmp_192_var; while ( - (_tmp_193_var = _tmp_193_rule(p)) // star_targets '=' + (_tmp_192_var = _tmp_192_rule(p)) // star_targets '=' ) { - _res = _tmp_193_var; + _res = _tmp_192_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23758,12 +23897,12 @@ _loop0_24_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_24[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_194_var; + void *_tmp_193_var; while ( - (_tmp_194_var = _tmp_194_rule(p)) // '.' | '...' + (_tmp_193_var = _tmp_193_rule(p)) // '.' | '...' ) { - _res = _tmp_194_var; + _res = _tmp_193_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23827,12 +23966,12 @@ _loop1_25_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_25[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_195_var; + void *_tmp_194_var; while ( - (_tmp_195_var = _tmp_195_rule(p)) // '.' | '...' + (_tmp_194_var = _tmp_194_rule(p)) // '.' | '...' ) { - _res = _tmp_195_var; + _res = _tmp_194_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -24235,12 +24374,12 @@ _loop1_32_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_32[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('@' named_expression NEWLINE)")); - void *_tmp_196_var; + void *_tmp_195_var; while ( - (_tmp_196_var = _tmp_196_rule(p)) // '@' named_expression NEWLINE + (_tmp_195_var = _tmp_195_rule(p)) // '@' named_expression NEWLINE ) { - _res = _tmp_196_var; + _res = _tmp_195_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27255,12 +27394,12 @@ _loop1_79_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_79[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' expression)")); - void *_tmp_197_var; + void *_tmp_196_var; while ( - (_tmp_197_var = _tmp_197_rule(p)) // ',' expression + (_tmp_196_var = _tmp_196_rule(p)) // ',' expression ) { - _res = _tmp_197_var; + _res = _tmp_196_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27329,12 +27468,12 @@ _loop1_80_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_80[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_expression)")); - void *_tmp_198_var; + void *_tmp_197_var; while ( - (_tmp_198_var = _tmp_198_rule(p)) // ',' star_expression + (_tmp_197_var = _tmp_197_rule(p)) // ',' star_expression ) { - _res = _tmp_198_var; + _res = _tmp_197_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27523,12 +27662,12 @@ _loop1_83_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_83[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('or' conjunction)")); - void *_tmp_199_var; + void *_tmp_198_var; while ( - (_tmp_199_var = _tmp_199_rule(p)) // 'or' conjunction + (_tmp_198_var = _tmp_198_rule(p)) // 'or' conjunction ) { - _res = _tmp_199_var; + _res = _tmp_198_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27597,12 +27736,12 @@ _loop1_84_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_84[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('and' inversion)")); - void *_tmp_200_var; + void *_tmp_199_var; while ( - (_tmp_200_var = _tmp_200_rule(p)) // 'and' inversion + (_tmp_199_var = _tmp_199_rule(p)) // 'and' inversion ) { - _res = _tmp_200_var; + _res = _tmp_199_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29564,12 +29703,12 @@ _loop0_113_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_113[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_201_var; + void *_tmp_200_var; while ( - (_tmp_201_var = _tmp_201_rule(p)) // 'if' disjunction + (_tmp_200_var = _tmp_200_rule(p)) // 'if' disjunction ) { - _res = _tmp_201_var; + _res = _tmp_200_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29633,12 +29772,12 @@ _loop0_114_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_114[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_202_var; + void *_tmp_201_var; while ( - (_tmp_202_var = _tmp_202_rule(p)) // 'if' disjunction + (_tmp_201_var = _tmp_201_rule(p)) // 'if' disjunction ) { - _res = _tmp_202_var; + _res = _tmp_201_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29767,7 +29906,7 @@ _loop0_117_rule(Parser *p) while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_203_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' + (elem = _tmp_202_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' ) { _res = elem; @@ -29834,7 +29973,7 @@ _gather_116_rule(Parser *p) void *elem; asdl_seq * seq; if ( - (elem = _tmp_203_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' + (elem = _tmp_202_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' && (seq = _loop0_117_rule(p)) // _loop0_117 ) @@ -30410,12 +30549,12 @@ _loop0_127_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_127[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); - void *_tmp_204_var; + void *_tmp_203_var; while ( - (_tmp_204_var = _tmp_204_rule(p)) // ',' star_target + (_tmp_203_var = _tmp_203_rule(p)) // ',' star_target ) { - _res = _tmp_204_var; + _res = _tmp_203_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -30599,12 +30738,12 @@ _loop1_130_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_130[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); - void *_tmp_205_var; + void *_tmp_204_var; while ( - (_tmp_205_var = _tmp_205_rule(p)) // ',' star_target + (_tmp_204_var = _tmp_204_rule(p)) // ',' star_target ) { - _res = _tmp_205_var; + _res = _tmp_204_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -31683,143 +31822,9 @@ _tmp_148_rule(Parser *p) return _res; } -// _tmp_149: list | tuple | genexp | 'True' | 'None' | 'False' +// _tmp_149: '=' | ':=' static void * _tmp_149_rule(Parser *p) -{ - if (p->level++ == MAXSTACK) { - p->error_indicator = 1; - PyErr_NoMemory(); - } - if (p->error_indicator) { - p->level--; - return NULL; - } - void * _res = NULL; - int _mark = p->mark; - { // list - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "list")); - expr_ty list_var; - if ( - (list_var = list_rule(p)) // list - ) - { - D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "list")); - _res = list_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "list")); - } - { // tuple - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "tuple")); - expr_ty tuple_var; - if ( - (tuple_var = tuple_rule(p)) // tuple - ) - { - D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "tuple")); - _res = tuple_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "tuple")); - } - { // genexp - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "genexp")); - expr_ty genexp_var; - if ( - (genexp_var = genexp_rule(p)) // genexp - ) - { - D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "genexp")); - _res = genexp_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "genexp")); - } - { // 'True' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 600)) // token='True' - ) - { - D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'True'")); - } - { // 'None' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 601)) // token='None' - ) - { - D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'None'")); - } - { // 'False' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 602)) // token='False' - ) - { - D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'False'")); - } - _res = NULL; - done: - p->level--; - return _res; -} - -// _tmp_150: '=' | ':=' -static void * -_tmp_150_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31836,18 +31841,18 @@ _tmp_150_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'='")); + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'='")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_150[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'='")); + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'='")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_150[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'='")); } { // ':=' @@ -31855,18 +31860,18 @@ _tmp_150_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':='")); + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':='")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 53)) // token=':=' ) { - D(fprintf(stderr, "%*c+ _tmp_150[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':='")); + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':='")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_150[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':='")); } _res = NULL; @@ -31875,9 +31880,9 @@ _tmp_150_rule(Parser *p) return _res; } -// _loop0_151: star_named_expressions +// _loop0_150: star_named_expressions static asdl_seq * -_loop0_151_rule(Parser *p) +_loop0_150_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31904,7 +31909,7 @@ _loop0_151_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); + D(fprintf(stderr, "%*c> _loop0_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); asdl_expr_seq* star_named_expressions_var; while ( (star_named_expressions_var = star_named_expressions_rule(p)) // star_named_expressions @@ -31926,7 +31931,7 @@ _loop0_151_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_151[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_150[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_named_expressions")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31939,14 +31944,14 @@ _loop0_151_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_151_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_150_type, _seq); p->level--; return _seq; } -// _loop0_152: (star_targets '=') +// _loop0_151: (star_targets '=') static asdl_seq * -_loop0_152_rule(Parser *p) +_loop0_151_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31973,13 +31978,13 @@ _loop0_152_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_206_var; + D(fprintf(stderr, "%*c> _loop0_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_205_var; while ( - (_tmp_206_var = _tmp_206_rule(p)) // star_targets '=' + (_tmp_205_var = _tmp_205_rule(p)) // star_targets '=' ) { - _res = _tmp_206_var; + _res = _tmp_205_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -31995,7 +32000,7 @@ _loop0_152_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_152[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_151[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32008,14 +32013,14 @@ _loop0_152_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_152_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_151_type, _seq); p->level--; return _seq; } -// _loop0_153: (star_targets '=') +// _loop0_152: (star_targets '=') static asdl_seq * -_loop0_153_rule(Parser *p) +_loop0_152_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32042,13 +32047,13 @@ _loop0_153_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_207_var; + D(fprintf(stderr, "%*c> _loop0_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_206_var; while ( - (_tmp_207_var = _tmp_207_rule(p)) // star_targets '=' + (_tmp_206_var = _tmp_206_rule(p)) // star_targets '=' ) { - _res = _tmp_207_var; + _res = _tmp_206_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -32064,7 +32069,7 @@ _loop0_153_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_152[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32077,14 +32082,14 @@ _loop0_153_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_153_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_152_type, _seq); p->level--; return _seq; } -// _tmp_154: yield_expr | star_expressions +// _tmp_153: yield_expr | star_expressions static void * -_tmp_154_rule(Parser *p) +_tmp_153_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32101,18 +32106,18 @@ _tmp_154_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); expr_ty yield_expr_var; if ( (yield_expr_var = yield_expr_rule(p)) // yield_expr ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); _res = yield_expr_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "yield_expr")); } { // star_expressions @@ -32120,18 +32125,18 @@ _tmp_154_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); expr_ty star_expressions_var; if ( (star_expressions_var = star_expressions_rule(p)) // star_expressions ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); _res = star_expressions_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_expressions")); } _res = NULL; @@ -32140,9 +32145,9 @@ _tmp_154_rule(Parser *p) return _res; } -// _tmp_155: '[' | '(' | '{' +// _tmp_154: '[' | '(' | '{' static void * -_tmp_155_rule(Parser *p) +_tmp_154_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32159,18 +32164,18 @@ _tmp_155_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '(' @@ -32178,18 +32183,18 @@ _tmp_155_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 7)) // token='(' ) { - D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'('")); } { // '{' @@ -32197,18 +32202,18 @@ _tmp_155_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -32217,9 +32222,9 @@ _tmp_155_rule(Parser *p) return _res; } -// _tmp_156: '[' | '{' +// _tmp_155: '[' | '{' static void * -_tmp_156_rule(Parser *p) +_tmp_155_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32236,18 +32241,18 @@ _tmp_156_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '{' @@ -32255,18 +32260,18 @@ _tmp_156_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -32275,9 +32280,9 @@ _tmp_156_rule(Parser *p) return _res; } -// _tmp_157: '[' | '{' +// _tmp_156: '[' | '{' static void * -_tmp_157_rule(Parser *p) +_tmp_156_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32294,18 +32299,18 @@ _tmp_157_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_157[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_157[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '{' @@ -32313,18 +32318,18 @@ _tmp_157_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_157[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_157[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -32333,9 +32338,9 @@ _tmp_157_rule(Parser *p) return _res; } -// _loop0_158: param_no_default +// _loop0_157: param_no_default static asdl_seq * -_loop0_158_rule(Parser *p) +_loop0_157_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32362,7 +32367,7 @@ _loop0_158_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_158[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); + D(fprintf(stderr, "%*c> _loop0_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); arg_ty param_no_default_var; while ( (param_no_default_var = param_no_default_rule(p)) // param_no_default @@ -32384,7 +32389,7 @@ _loop0_158_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_158[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_157[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32397,14 +32402,14 @@ _loop0_158_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_158_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_157_type, _seq); p->level--; return _seq; } -// _loop0_159: param_no_default +// _loop0_158: param_no_default static asdl_seq * -_loop0_159_rule(Parser *p) +_loop0_158_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32431,7 +32436,7 @@ _loop0_159_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); + D(fprintf(stderr, "%*c> _loop0_158[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); arg_ty param_no_default_var; while ( (param_no_default_var = param_no_default_rule(p)) // param_no_default @@ -32453,7 +32458,7 @@ _loop0_159_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_159[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_158[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32466,14 +32471,14 @@ _loop0_159_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_159_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_158_type, _seq); p->level--; return _seq; } -// _loop1_160: param_no_default +// _loop1_159: param_no_default static asdl_seq * -_loop1_160_rule(Parser *p) +_loop1_159_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32500,7 +32505,7 @@ _loop1_160_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); + D(fprintf(stderr, "%*c> _loop1_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); arg_ty param_no_default_var; while ( (param_no_default_var = param_no_default_rule(p)) // param_no_default @@ -32522,7 +32527,7 @@ _loop1_160_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_160[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_159[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } if (_n == 0 || p->error_indicator) { @@ -32540,14 +32545,14 @@ _loop1_160_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_160_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_159_type, _seq); p->level--; return _seq; } -// _loop1_161: param_with_default +// _loop1_160: param_with_default static asdl_seq * -_loop1_161_rule(Parser *p) +_loop1_160_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32574,7 +32579,7 @@ _loop1_161_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); + D(fprintf(stderr, "%*c> _loop1_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); NameDefaultPair* param_with_default_var; while ( (param_with_default_var = param_with_default_rule(p)) // param_with_default @@ -32596,7 +32601,7 @@ _loop1_161_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_161[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_160[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -32614,14 +32619,14 @@ _loop1_161_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_161_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_160_type, _seq); p->level--; return _seq; } -// _loop0_162: lambda_param_no_default +// _loop0_161: lambda_param_no_default static asdl_seq * -_loop0_162_rule(Parser *p) +_loop0_161_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32648,7 +32653,7 @@ _loop0_162_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); + D(fprintf(stderr, "%*c> _loop0_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); arg_ty lambda_param_no_default_var; while ( (lambda_param_no_default_var = lambda_param_no_default_rule(p)) // lambda_param_no_default @@ -32670,7 +32675,7 @@ _loop0_162_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_162[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_161[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32683,14 +32688,14 @@ _loop0_162_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_162_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_161_type, _seq); p->level--; return _seq; } -// _loop0_163: lambda_param_no_default +// _loop0_162: lambda_param_no_default static asdl_seq * -_loop0_163_rule(Parser *p) +_loop0_162_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32717,7 +32722,7 @@ _loop0_163_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_163[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); + D(fprintf(stderr, "%*c> _loop0_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); arg_ty lambda_param_no_default_var; while ( (lambda_param_no_default_var = lambda_param_no_default_rule(p)) // lambda_param_no_default @@ -32739,7 +32744,7 @@ _loop0_163_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_163[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_162[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32752,14 +32757,14 @@ _loop0_163_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_163_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_162_type, _seq); p->level--; return _seq; } -// _loop0_165: ',' lambda_param +// _loop0_164: ',' lambda_param static asdl_seq * -_loop0_165_rule(Parser *p) +_loop0_164_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32786,7 +32791,7 @@ _loop0_165_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_165[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' lambda_param")); + D(fprintf(stderr, "%*c> _loop0_164[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' lambda_param")); Token * _literal; arg_ty elem; while ( @@ -32817,7 +32822,7 @@ _loop0_165_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_165[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_164[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' lambda_param")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32830,14 +32835,14 @@ _loop0_165_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_165_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_164_type, _seq); p->level--; return _seq; } -// _gather_164: lambda_param _loop0_165 +// _gather_163: lambda_param _loop0_164 static asdl_seq * -_gather_164_rule(Parser *p) +_gather_163_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32849,27 +32854,27 @@ _gather_164_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // lambda_param _loop0_165 + { // lambda_param _loop0_164 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_164[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param _loop0_165")); + D(fprintf(stderr, "%*c> _gather_163[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param _loop0_164")); arg_ty elem; asdl_seq * seq; if ( (elem = lambda_param_rule(p)) // lambda_param && - (seq = _loop0_165_rule(p)) // _loop0_165 + (seq = _loop0_164_rule(p)) // _loop0_164 ) { - D(fprintf(stderr, "%*c+ _gather_164[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param _loop0_165")); + D(fprintf(stderr, "%*c+ _gather_163[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param _loop0_164")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_164[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param _loop0_165")); + D(fprintf(stderr, "%*c%s _gather_163[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param _loop0_164")); } _res = NULL; done: @@ -32877,9 +32882,9 @@ _gather_164_rule(Parser *p) return _res; } -// _loop1_166: lambda_param_with_default +// _loop1_165: lambda_param_with_default static asdl_seq * -_loop1_166_rule(Parser *p) +_loop1_165_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32906,7 +32911,7 @@ _loop1_166_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); + D(fprintf(stderr, "%*c> _loop1_165[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); NameDefaultPair* lambda_param_with_default_var; while ( (lambda_param_with_default_var = lambda_param_with_default_rule(p)) // lambda_param_with_default @@ -32928,7 +32933,7 @@ _loop1_166_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_166[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_165[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -32946,14 +32951,14 @@ _loop1_166_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_166_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_165_type, _seq); p->level--; return _seq; } -// _tmp_167: ')' | ',' (')' | '**') +// _tmp_166: ')' | ',' (')' | '**') static void * -_tmp_167_rule(Parser *p) +_tmp_166_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32970,18 +32975,18 @@ _tmp_167_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_166[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_167[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_166[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // ',' (')' | '**') @@ -32989,21 +32994,21 @@ _tmp_167_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + D(fprintf(stderr, "%*c> _tmp_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); Token * _literal; - void *_tmp_208_var; + void *_tmp_207_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_208_var = _tmp_208_rule(p)) // ')' | '**' + (_tmp_207_var = _tmp_207_rule(p)) // ')' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_208_var); + D(fprintf(stderr, "%*c+ _tmp_166[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_207_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_167[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_166[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (')' | '**')")); } _res = NULL; @@ -33012,9 +33017,9 @@ _tmp_167_rule(Parser *p) return _res; } -// _tmp_168: ':' | ',' (':' | '**') +// _tmp_167: ':' | ',' (':' | '**') static void * -_tmp_168_rule(Parser *p) +_tmp_167_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33031,18 +33036,18 @@ _tmp_168_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_167[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // ',' (':' | '**') @@ -33050,21 +33055,21 @@ _tmp_168_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + D(fprintf(stderr, "%*c> _tmp_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); Token * _literal; - void *_tmp_209_var; + void *_tmp_208_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_209_var = _tmp_209_rule(p)) // ':' | '**' + (_tmp_208_var = _tmp_208_rule(p)) // ':' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_209_var); + D(fprintf(stderr, "%*c+ _tmp_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_208_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_167[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (':' | '**')")); } _res = NULL; @@ -33073,9 +33078,9 @@ _tmp_168_rule(Parser *p) return _res; } -// _tmp_169: ',' | ')' | ':' +// _tmp_168: ',' | ')' | ':' static void * -_tmp_169_rule(Parser *p) +_tmp_168_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33092,18 +33097,18 @@ _tmp_169_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_169[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "','")); } { // ')' @@ -33111,18 +33116,18 @@ _tmp_169_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_169[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // ':' @@ -33130,18 +33135,18 @@ _tmp_169_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_169[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } _res = NULL; @@ -33150,9 +33155,9 @@ _tmp_169_rule(Parser *p) return _res; } -// _loop0_171: ',' (expression ['as' star_target]) +// _loop0_170: ',' (expression ['as' star_target]) static asdl_seq * -_loop0_171_rule(Parser *p) +_loop0_170_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33179,13 +33184,13 @@ _loop0_171_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_210_rule(p)) // expression ['as' star_target] + (elem = _tmp_209_rule(p)) // expression ['as' star_target] ) { _res = elem; @@ -33210,7 +33215,7 @@ _loop0_171_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_171[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_170[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expression ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33223,14 +33228,14 @@ _loop0_171_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_171_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_170_type, _seq); p->level--; return _seq; } -// _gather_170: (expression ['as' star_target]) _loop0_171 +// _gather_169: (expression ['as' star_target]) _loop0_170 static asdl_seq * -_gather_170_rule(Parser *p) +_gather_169_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33242,27 +33247,27 @@ _gather_170_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expression ['as' star_target]) _loop0_171 + { // (expression ['as' star_target]) _loop0_170 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_171")); + D(fprintf(stderr, "%*c> _gather_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_170")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_210_rule(p)) // expression ['as' star_target] + (elem = _tmp_209_rule(p)) // expression ['as' star_target] && - (seq = _loop0_171_rule(p)) // _loop0_171 + (seq = _loop0_170_rule(p)) // _loop0_170 ) { - D(fprintf(stderr, "%*c+ _gather_170[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_171")); + D(fprintf(stderr, "%*c+ _gather_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_170")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_170[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_171")); + D(fprintf(stderr, "%*c%s _gather_169[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_170")); } _res = NULL; done: @@ -33270,9 +33275,9 @@ _gather_170_rule(Parser *p) return _res; } -// _loop0_173: ',' (expressions ['as' star_target]) +// _loop0_172: ',' (expressions ['as' star_target]) static asdl_seq * -_loop0_173_rule(Parser *p) +_loop0_172_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33299,13 +33304,13 @@ _loop0_173_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_173[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_172[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_211_rule(p)) // expressions ['as' star_target] + (elem = _tmp_210_rule(p)) // expressions ['as' star_target] ) { _res = elem; @@ -33330,7 +33335,7 @@ _loop0_173_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_173[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_172[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expressions ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33343,14 +33348,14 @@ _loop0_173_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_173_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_172_type, _seq); p->level--; return _seq; } -// _gather_172: (expressions ['as' star_target]) _loop0_173 +// _gather_171: (expressions ['as' star_target]) _loop0_172 static asdl_seq * -_gather_172_rule(Parser *p) +_gather_171_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33362,27 +33367,27 @@ _gather_172_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expressions ['as' star_target]) _loop0_173 + { // (expressions ['as' star_target]) _loop0_172 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_172[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_173")); + D(fprintf(stderr, "%*c> _gather_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_172")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_211_rule(p)) // expressions ['as' star_target] + (elem = _tmp_210_rule(p)) // expressions ['as' star_target] && - (seq = _loop0_173_rule(p)) // _loop0_173 + (seq = _loop0_172_rule(p)) // _loop0_172 ) { - D(fprintf(stderr, "%*c+ _gather_172[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_173")); + D(fprintf(stderr, "%*c+ _gather_171[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_172")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_172[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_173")); + D(fprintf(stderr, "%*c%s _gather_171[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_172")); } _res = NULL; done: @@ -33390,9 +33395,9 @@ _gather_172_rule(Parser *p) return _res; } -// _loop0_175: ',' (expression ['as' star_target]) +// _loop0_174: ',' (expression ['as' star_target]) static asdl_seq * -_loop0_175_rule(Parser *p) +_loop0_174_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33419,13 +33424,13 @@ _loop0_175_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_175[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_174[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_212_rule(p)) // expression ['as' star_target] + (elem = _tmp_211_rule(p)) // expression ['as' star_target] ) { _res = elem; @@ -33450,7 +33455,7 @@ _loop0_175_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_175[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_174[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expression ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33463,14 +33468,14 @@ _loop0_175_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_175_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_174_type, _seq); p->level--; return _seq; } -// _gather_174: (expression ['as' star_target]) _loop0_175 +// _gather_173: (expression ['as' star_target]) _loop0_174 static asdl_seq * -_gather_174_rule(Parser *p) +_gather_173_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33482,27 +33487,27 @@ _gather_174_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expression ['as' star_target]) _loop0_175 + { // (expression ['as' star_target]) _loop0_174 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_174[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_175")); + D(fprintf(stderr, "%*c> _gather_173[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_174")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_212_rule(p)) // expression ['as' star_target] + (elem = _tmp_211_rule(p)) // expression ['as' star_target] && - (seq = _loop0_175_rule(p)) // _loop0_175 + (seq = _loop0_174_rule(p)) // _loop0_174 ) { - D(fprintf(stderr, "%*c+ _gather_174[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_175")); + D(fprintf(stderr, "%*c+ _gather_173[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_174")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_174[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_175")); + D(fprintf(stderr, "%*c%s _gather_173[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_174")); } _res = NULL; done: @@ -33510,9 +33515,9 @@ _gather_174_rule(Parser *p) return _res; } -// _loop0_177: ',' (expressions ['as' star_target]) +// _loop0_176: ',' (expressions ['as' star_target]) static asdl_seq * -_loop0_177_rule(Parser *p) +_loop0_176_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33539,13 +33544,13 @@ _loop0_177_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_176[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_213_rule(p)) // expressions ['as' star_target] + (elem = _tmp_212_rule(p)) // expressions ['as' star_target] ) { _res = elem; @@ -33570,7 +33575,7 @@ _loop0_177_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_177[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_176[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expressions ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33583,14 +33588,14 @@ _loop0_177_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_177_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_176_type, _seq); p->level--; return _seq; } -// _gather_176: (expressions ['as' star_target]) _loop0_177 +// _gather_175: (expressions ['as' star_target]) _loop0_176 static asdl_seq * -_gather_176_rule(Parser *p) +_gather_175_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33602,27 +33607,27 @@ _gather_176_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expressions ['as' star_target]) _loop0_177 + { // (expressions ['as' star_target]) _loop0_176 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_176[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_177")); + D(fprintf(stderr, "%*c> _gather_175[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_176")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_213_rule(p)) // expressions ['as' star_target] + (elem = _tmp_212_rule(p)) // expressions ['as' star_target] && - (seq = _loop0_177_rule(p)) // _loop0_177 + (seq = _loop0_176_rule(p)) // _loop0_176 ) { - D(fprintf(stderr, "%*c+ _gather_176[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_177")); + D(fprintf(stderr, "%*c+ _gather_175[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_176")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_176[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_177")); + D(fprintf(stderr, "%*c%s _gather_175[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_176")); } _res = NULL; done: @@ -33630,9 +33635,9 @@ _gather_176_rule(Parser *p) return _res; } -// _tmp_178: 'except' | 'finally' +// _tmp_177: 'except' | 'finally' static void * -_tmp_178_rule(Parser *p) +_tmp_177_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33649,18 +33654,18 @@ _tmp_178_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except'")); + D(fprintf(stderr, "%*c> _tmp_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except'")); Token * _keyword; if ( (_keyword = _PyPegen_expect_token(p, 629)) // token='except' ) { - D(fprintf(stderr, "%*c+ _tmp_178[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except'")); + D(fprintf(stderr, "%*c+ _tmp_177[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except'")); _res = _keyword; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_178[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_177[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'except'")); } { // 'finally' @@ -33668,18 +33673,18 @@ _tmp_178_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally'")); + D(fprintf(stderr, "%*c> _tmp_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally'")); Token * _keyword; if ( (_keyword = _PyPegen_expect_token(p, 625)) // token='finally' ) { - D(fprintf(stderr, "%*c+ _tmp_178[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'finally'")); + D(fprintf(stderr, "%*c+ _tmp_177[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'finally'")); _res = _keyword; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_178[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_177[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'finally'")); } _res = NULL; @@ -33688,9 +33693,9 @@ _tmp_178_rule(Parser *p) return _res; } -// _loop0_179: block +// _loop0_178: block static asdl_seq * -_loop0_179_rule(Parser *p) +_loop0_178_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33717,7 +33722,7 @@ _loop0_179_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "block")); + D(fprintf(stderr, "%*c> _loop0_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "block")); asdl_stmt_seq* block_var; while ( (block_var = block_rule(p)) // block @@ -33739,7 +33744,7 @@ _loop0_179_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_179[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_178[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "block")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33752,14 +33757,14 @@ _loop0_179_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_179_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_178_type, _seq); p->level--; return _seq; } -// _tmp_180: (except_block+ except_star_block) | (except_star_block+ except_block) +// _tmp_179: (except_block+ except_star_block) | (except_star_block+ except_block) static void * -_tmp_180_rule(Parser *p) +_tmp_179_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33776,18 +33781,18 @@ _tmp_180_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(except_block+ except_star_block)")); - void *_tmp_214_var; + D(fprintf(stderr, "%*c> _tmp_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(except_block+ except_star_block)")); + void *_tmp_213_var; if ( - (_tmp_214_var = _tmp_214_rule(p)) // except_block+ except_star_block + (_tmp_213_var = _tmp_213_rule(p)) // except_block+ except_star_block ) { - D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(except_block+ except_star_block)")); - _res = _tmp_214_var; + D(fprintf(stderr, "%*c+ _tmp_179[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(except_block+ except_star_block)")); + _res = _tmp_213_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_179[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(except_block+ except_star_block)")); } { // (except_star_block+ except_block) @@ -33795,18 +33800,18 @@ _tmp_180_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(except_star_block+ except_block)")); - void *_tmp_215_var; + D(fprintf(stderr, "%*c> _tmp_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(except_star_block+ except_block)")); + void *_tmp_214_var; if ( - (_tmp_215_var = _tmp_215_rule(p)) // except_star_block+ except_block + (_tmp_214_var = _tmp_214_rule(p)) // except_star_block+ except_block ) { - D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(except_star_block+ except_block)")); - _res = _tmp_215_var; + D(fprintf(stderr, "%*c+ _tmp_179[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(except_star_block+ except_block)")); + _res = _tmp_214_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_179[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(except_star_block+ except_block)")); } _res = NULL; @@ -33815,9 +33820,9 @@ _tmp_180_rule(Parser *p) return _res; } -// _loop0_181: block +// _loop0_180: block static asdl_seq * -_loop0_181_rule(Parser *p) +_loop0_180_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33844,7 +33849,7 @@ _loop0_181_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "block")); + D(fprintf(stderr, "%*c> _loop0_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "block")); asdl_stmt_seq* block_var; while ( (block_var = block_rule(p)) // block @@ -33866,7 +33871,7 @@ _loop0_181_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_181[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_180[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "block")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33879,11 +33884,53 @@ _loop0_181_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_181_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_180_type, _seq); p->level--; return _seq; } +// _tmp_181: 'as' NAME +static void * +_tmp_181_rule(Parser *p) +{ + if (p->level++ == MAXSTACK) { + p->error_indicator = 1; + PyErr_NoMemory(); + } + if (p->error_indicator) { + p->level--; + return NULL; + } + void * _res = NULL; + int _mark = p->mark; + { // 'as' NAME + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + Token * _keyword; + expr_ty name_var; + if ( + (_keyword = _PyPegen_expect_token(p, 632)) // token='as' + && + (name_var = _PyPegen_name_token(p)) // NAME + ) + { + D(fprintf(stderr, "%*c+ _tmp_181[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + _res = _PyPegen_dummy_name(p, _keyword, name_var); + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_181[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); + } + _res = NULL; + done: + p->level--; + return _res; +} + // _tmp_182: 'as' NAME static void * _tmp_182_rule(Parser *p) @@ -33926,51 +33973,9 @@ _tmp_182_rule(Parser *p) return _res; } -// _tmp_183: 'as' NAME +// _tmp_183: NEWLINE | ':' static void * _tmp_183_rule(Parser *p) -{ - if (p->level++ == MAXSTACK) { - p->error_indicator = 1; - PyErr_NoMemory(); - } - if (p->error_indicator) { - p->level--; - return NULL; - } - void * _res = NULL; - int _mark = p->mark; - { // 'as' NAME - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); - Token * _keyword; - expr_ty name_var; - if ( - (_keyword = _PyPegen_expect_token(p, 632)) // token='as' - && - (name_var = _PyPegen_name_token(p)) // NAME - ) - { - D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); - _res = _PyPegen_dummy_name(p, _keyword, name_var); - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); - } - _res = NULL; - done: - p->level--; - return _res; -} - -// _tmp_184: NEWLINE | ':' -static void * -_tmp_184_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33987,18 +33992,18 @@ _tmp_184_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "NEWLINE")); + D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "NEWLINE")); Token * newline_var; if ( (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "NEWLINE")); + D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "NEWLINE")); _res = newline_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "NEWLINE")); } { // ':' @@ -34006,18 +34011,18 @@ _tmp_184_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } _res = NULL; @@ -34026,9 +34031,9 @@ _tmp_184_rule(Parser *p) return _res; } -// _tmp_185: 'as' NAME +// _tmp_184: 'as' NAME static void * -_tmp_185_rule(Parser *p) +_tmp_184_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34045,7 +34050,7 @@ _tmp_185_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_185[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -34054,12 +34059,12 @@ _tmp_185_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_185[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_185[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -34068,9 +34073,9 @@ _tmp_185_rule(Parser *p) return _res; } -// _tmp_186: 'as' NAME +// _tmp_185: 'as' NAME static void * -_tmp_186_rule(Parser *p) +_tmp_185_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34087,7 +34092,7 @@ _tmp_186_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_186[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_185[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -34096,12 +34101,12 @@ _tmp_186_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_186[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_185[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_186[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_185[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -34110,9 +34115,9 @@ _tmp_186_rule(Parser *p) return _res; } -// _tmp_187: positional_patterns ',' +// _tmp_186: positional_patterns ',' static void * -_tmp_187_rule(Parser *p) +_tmp_186_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34129,7 +34134,7 @@ _tmp_187_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_187[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); + D(fprintf(stderr, "%*c> _tmp_186[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); Token * _literal; asdl_pattern_seq* positional_patterns_var; if ( @@ -34138,12 +34143,12 @@ _tmp_187_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_187[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); + D(fprintf(stderr, "%*c+ _tmp_186[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); _res = _PyPegen_dummy_name(p, positional_patterns_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_187[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_186[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "positional_patterns ','")); } _res = NULL; @@ -34152,9 +34157,9 @@ _tmp_187_rule(Parser *p) return _res; } -// _tmp_188: '->' expression +// _tmp_187: '->' expression static void * -_tmp_188_rule(Parser *p) +_tmp_187_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34171,7 +34176,7 @@ _tmp_188_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_188[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'->' expression")); + D(fprintf(stderr, "%*c> _tmp_187[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'->' expression")); Token * _literal; expr_ty expression_var; if ( @@ -34180,12 +34185,12 @@ _tmp_188_rule(Parser *p) (expression_var = expression_rule(p)) // expression ) { - D(fprintf(stderr, "%*c+ _tmp_188[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'->' expression")); + D(fprintf(stderr, "%*c+ _tmp_187[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'->' expression")); _res = _PyPegen_dummy_name(p, _literal, expression_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_188[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_187[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'->' expression")); } _res = NULL; @@ -34194,9 +34199,9 @@ _tmp_188_rule(Parser *p) return _res; } -// _tmp_189: '(' arguments? ')' +// _tmp_188: '(' arguments? ')' static void * -_tmp_189_rule(Parser *p) +_tmp_188_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34213,7 +34218,7 @@ _tmp_189_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_189[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); + D(fprintf(stderr, "%*c> _tmp_188[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); Token * _literal; Token * _literal_1; void *_opt_var; @@ -34226,12 +34231,12 @@ _tmp_189_rule(Parser *p) (_literal_1 = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_189[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); + D(fprintf(stderr, "%*c+ _tmp_188[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); _res = _PyPegen_dummy_name(p, _literal, _opt_var, _literal_1); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_189[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_188[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'(' arguments? ')'")); } _res = NULL; @@ -34240,9 +34245,9 @@ _tmp_189_rule(Parser *p) return _res; } -// _loop0_191: ',' double_starred_kvpair +// _loop0_190: ',' double_starred_kvpair static asdl_seq * -_loop0_191_rule(Parser *p) +_loop0_190_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34269,7 +34274,7 @@ _loop0_191_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' double_starred_kvpair")); + D(fprintf(stderr, "%*c> _loop0_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' double_starred_kvpair")); Token * _literal; KeyValuePair* elem; while ( @@ -34300,7 +34305,7 @@ _loop0_191_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_191[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_190[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' double_starred_kvpair")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -34313,14 +34318,14 @@ _loop0_191_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_191_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_190_type, _seq); p->level--; return _seq; } -// _gather_190: double_starred_kvpair _loop0_191 +// _gather_189: double_starred_kvpair _loop0_190 static asdl_seq * -_gather_190_rule(Parser *p) +_gather_189_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34332,27 +34337,27 @@ _gather_190_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // double_starred_kvpair _loop0_191 + { // double_starred_kvpair _loop0_190 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_191")); + D(fprintf(stderr, "%*c> _gather_189[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_190")); KeyValuePair* elem; asdl_seq * seq; if ( (elem = double_starred_kvpair_rule(p)) // double_starred_kvpair && - (seq = _loop0_191_rule(p)) // _loop0_191 + (seq = _loop0_190_rule(p)) // _loop0_190 ) { - D(fprintf(stderr, "%*c+ _gather_190[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_191")); + D(fprintf(stderr, "%*c+ _gather_189[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_190")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_190[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "double_starred_kvpair _loop0_191")); + D(fprintf(stderr, "%*c%s _gather_189[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "double_starred_kvpair _loop0_190")); } _res = NULL; done: @@ -34360,9 +34365,9 @@ _gather_190_rule(Parser *p) return _res; } -// _tmp_192: '}' | ',' +// _tmp_191: '}' | ',' static void * -_tmp_192_rule(Parser *p) +_tmp_191_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34379,18 +34384,18 @@ _tmp_192_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'}'")); + D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'}'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 26)) // token='}' ) { - D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'}'")); + D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'}'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'}'")); } { // ',' @@ -34398,18 +34403,18 @@ _tmp_192_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "','")); } _res = NULL; @@ -34418,9 +34423,9 @@ _tmp_192_rule(Parser *p) return _res; } -// _tmp_193: star_targets '=' +// _tmp_192: star_targets '=' static void * -_tmp_193_rule(Parser *p) +_tmp_192_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34437,7 +34442,7 @@ _tmp_193_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty z; if ( @@ -34446,7 +34451,7 @@ _tmp_193_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34456,7 +34461,7 @@ _tmp_193_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -34465,9 +34470,9 @@ _tmp_193_rule(Parser *p) return _res; } -// _tmp_194: '.' | '...' +// _tmp_193: '.' | '...' static void * -_tmp_194_rule(Parser *p) +_tmp_193_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34484,18 +34489,18 @@ _tmp_194_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -34503,18 +34508,18 @@ _tmp_194_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -34523,9 +34528,9 @@ _tmp_194_rule(Parser *p) return _res; } -// _tmp_195: '.' | '...' +// _tmp_194: '.' | '...' static void * -_tmp_195_rule(Parser *p) +_tmp_194_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34542,18 +34547,18 @@ _tmp_195_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -34561,18 +34566,18 @@ _tmp_195_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -34581,9 +34586,9 @@ _tmp_195_rule(Parser *p) return _res; } -// _tmp_196: '@' named_expression NEWLINE +// _tmp_195: '@' named_expression NEWLINE static void * -_tmp_196_rule(Parser *p) +_tmp_195_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34600,7 +34605,7 @@ _tmp_196_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); Token * _literal; expr_ty f; Token * newline_var; @@ -34612,7 +34617,7 @@ _tmp_196_rule(Parser *p) (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); _res = f; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34622,7 +34627,7 @@ _tmp_196_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'@' named_expression NEWLINE")); } _res = NULL; @@ -34631,9 +34636,9 @@ _tmp_196_rule(Parser *p) return _res; } -// _tmp_197: ',' expression +// _tmp_196: ',' expression static void * -_tmp_197_rule(Parser *p) +_tmp_196_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34650,7 +34655,7 @@ _tmp_197_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); Token * _literal; expr_ty c; if ( @@ -34659,7 +34664,7 @@ _tmp_197_rule(Parser *p) (c = expression_rule(p)) // expression ) { - D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34669,7 +34674,7 @@ _tmp_197_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' expression")); } _res = NULL; @@ -34678,9 +34683,9 @@ _tmp_197_rule(Parser *p) return _res; } -// _tmp_198: ',' star_expression +// _tmp_197: ',' star_expression static void * -_tmp_198_rule(Parser *p) +_tmp_197_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34697,7 +34702,7 @@ _tmp_198_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_198[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); Token * _literal; expr_ty c; if ( @@ -34706,7 +34711,7 @@ _tmp_198_rule(Parser *p) (c = star_expression_rule(p)) // star_expression ) { - D(fprintf(stderr, "%*c+ _tmp_198[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34716,7 +34721,7 @@ _tmp_198_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_198[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_expression")); } _res = NULL; @@ -34725,9 +34730,9 @@ _tmp_198_rule(Parser *p) return _res; } -// _tmp_199: 'or' conjunction +// _tmp_198: 'or' conjunction static void * -_tmp_199_rule(Parser *p) +_tmp_198_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34744,7 +34749,7 @@ _tmp_199_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_199[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c> _tmp_198[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); Token * _keyword; expr_ty c; if ( @@ -34753,7 +34758,7 @@ _tmp_199_rule(Parser *p) (c = conjunction_rule(p)) // conjunction ) { - D(fprintf(stderr, "%*c+ _tmp_199[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c+ _tmp_198[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34763,7 +34768,7 @@ _tmp_199_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_199[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_198[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'or' conjunction")); } _res = NULL; @@ -34772,9 +34777,9 @@ _tmp_199_rule(Parser *p) return _res; } -// _tmp_200: 'and' inversion +// _tmp_199: 'and' inversion static void * -_tmp_200_rule(Parser *p) +_tmp_199_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34791,7 +34796,7 @@ _tmp_200_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_200[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c> _tmp_199[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); Token * _keyword; expr_ty c; if ( @@ -34800,7 +34805,7 @@ _tmp_200_rule(Parser *p) (c = inversion_rule(p)) // inversion ) { - D(fprintf(stderr, "%*c+ _tmp_200[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c+ _tmp_199[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34810,7 +34815,7 @@ _tmp_200_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_200[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_199[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'and' inversion")); } _res = NULL; @@ -34819,9 +34824,9 @@ _tmp_200_rule(Parser *p) return _res; } -// _tmp_201: 'if' disjunction +// _tmp_200: 'if' disjunction static void * -_tmp_201_rule(Parser *p) +_tmp_200_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34838,7 +34843,7 @@ _tmp_201_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_200[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -34847,7 +34852,7 @@ _tmp_201_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_200[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34857,7 +34862,7 @@ _tmp_201_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_200[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -34866,9 +34871,9 @@ _tmp_201_rule(Parser *p) return _res; } -// _tmp_202: 'if' disjunction +// _tmp_201: 'if' disjunction static void * -_tmp_202_rule(Parser *p) +_tmp_201_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34885,7 +34890,7 @@ _tmp_202_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -34894,7 +34899,7 @@ _tmp_202_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34904,7 +34909,7 @@ _tmp_202_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -34913,9 +34918,9 @@ _tmp_202_rule(Parser *p) return _res; } -// _tmp_203: starred_expression | (assignment_expression | expression !':=') !'=' +// _tmp_202: starred_expression | (assignment_expression | expression !':=') !'=' static void * -_tmp_203_rule(Parser *p) +_tmp_202_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34932,18 +34937,18 @@ _tmp_203_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression")); + D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression")); expr_ty starred_expression_var; if ( (starred_expression_var = starred_expression_rule(p)) // starred_expression ) { - D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression")); + D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression")); _res = starred_expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "starred_expression")); } { // (assignment_expression | expression !':=') !'=' @@ -34951,20 +34956,20 @@ _tmp_203_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); - void *_tmp_216_var; + D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); + void *_tmp_215_var; if ( - (_tmp_216_var = _tmp_216_rule(p)) // assignment_expression | expression !':=' + (_tmp_215_var = _tmp_215_rule(p)) // assignment_expression | expression !':=' && _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 22) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); - _res = _tmp_216_var; + D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); + _res = _tmp_215_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(assignment_expression | expression !':=') !'='")); } _res = NULL; @@ -34973,9 +34978,9 @@ _tmp_203_rule(Parser *p) return _res; } -// _tmp_204: ',' star_target +// _tmp_203: ',' star_target static void * -_tmp_204_rule(Parser *p) +_tmp_203_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34992,7 +34997,7 @@ _tmp_204_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_204[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty c; if ( @@ -35001,7 +35006,7 @@ _tmp_204_rule(Parser *p) (c = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_204[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -35011,7 +35016,7 @@ _tmp_204_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_204[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } _res = NULL; @@ -35020,9 +35025,9 @@ _tmp_204_rule(Parser *p) return _res; } -// _tmp_205: ',' star_target +// _tmp_204: ',' star_target static void * -_tmp_205_rule(Parser *p) +_tmp_204_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35039,7 +35044,7 @@ _tmp_205_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_205[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _tmp_204[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty c; if ( @@ -35048,7 +35053,7 @@ _tmp_205_rule(Parser *p) (c = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_205[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c+ _tmp_204[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -35058,7 +35063,7 @@ _tmp_205_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_205[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_204[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } _res = NULL; @@ -35067,9 +35072,9 @@ _tmp_205_rule(Parser *p) return _res; } -// _tmp_206: star_targets '=' +// _tmp_205: star_targets '=' static void * -_tmp_206_rule(Parser *p) +_tmp_205_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35086,7 +35091,7 @@ _tmp_206_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_206[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_205[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -35095,12 +35100,12 @@ _tmp_206_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_206[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_205[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_206[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_205[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -35109,9 +35114,9 @@ _tmp_206_rule(Parser *p) return _res; } -// _tmp_207: star_targets '=' +// _tmp_206: star_targets '=' static void * -_tmp_207_rule(Parser *p) +_tmp_206_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35128,7 +35133,7 @@ _tmp_207_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_207[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_206[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -35137,12 +35142,12 @@ _tmp_207_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_207[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_206[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_207[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_206[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -35151,9 +35156,9 @@ _tmp_207_rule(Parser *p) return _res; } -// _tmp_208: ')' | '**' +// _tmp_207: ')' | '**' static void * -_tmp_208_rule(Parser *p) +_tmp_207_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35170,18 +35175,18 @@ _tmp_208_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_208[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_207[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_208[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_207[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_208[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_207[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // '**' @@ -35189,18 +35194,18 @@ _tmp_208_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_208[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_207[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_208[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_207[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_208[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_207[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; @@ -35209,9 +35214,9 @@ _tmp_208_rule(Parser *p) return _res; } -// _tmp_209: ':' | '**' +// _tmp_208: ':' | '**' static void * -_tmp_209_rule(Parser *p) +_tmp_208_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35228,18 +35233,18 @@ _tmp_209_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_209[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_208[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_209[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_208[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_209[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_208[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // '**' @@ -35247,18 +35252,18 @@ _tmp_209_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_209[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_208[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_209[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_208[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_209[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_208[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; @@ -35267,9 +35272,9 @@ _tmp_209_rule(Parser *p) return _res; } -// _tmp_210: expression ['as' star_target] +// _tmp_209: expression ['as' star_target] static void * -_tmp_210_rule(Parser *p) +_tmp_209_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35286,22 +35291,22 @@ _tmp_210_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_210[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_209[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_217_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_216_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_210[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_209[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); _res = _PyPegen_dummy_name(p, expression_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_210[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_209[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression ['as' star_target]")); } _res = NULL; @@ -35310,9 +35315,9 @@ _tmp_210_rule(Parser *p) return _res; } -// _tmp_211: expressions ['as' star_target] +// _tmp_210: expressions ['as' star_target] static void * -_tmp_211_rule(Parser *p) +_tmp_210_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35329,22 +35334,22 @@ _tmp_211_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_211[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_210[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expressions_var; if ( (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_218_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_217_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_211[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_210[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); _res = _PyPegen_dummy_name(p, expressions_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_211[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_210[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expressions ['as' star_target]")); } _res = NULL; @@ -35353,9 +35358,9 @@ _tmp_211_rule(Parser *p) return _res; } -// _tmp_212: expression ['as' star_target] +// _tmp_211: expression ['as' star_target] static void * -_tmp_212_rule(Parser *p) +_tmp_211_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35372,22 +35377,22 @@ _tmp_212_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_212[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_211[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_219_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_218_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_212[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_211[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); _res = _PyPegen_dummy_name(p, expression_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_212[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_211[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression ['as' star_target]")); } _res = NULL; @@ -35396,9 +35401,9 @@ _tmp_212_rule(Parser *p) return _res; } -// _tmp_213: expressions ['as' star_target] +// _tmp_212: expressions ['as' star_target] static void * -_tmp_213_rule(Parser *p) +_tmp_212_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35415,22 +35420,22 @@ _tmp_213_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_213[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_212[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expressions_var; if ( (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_220_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_219_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_213[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_212[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); _res = _PyPegen_dummy_name(p, expressions_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_213[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_212[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expressions ['as' star_target]")); } _res = NULL; @@ -35439,9 +35444,9 @@ _tmp_213_rule(Parser *p) return _res; } -// _tmp_214: except_block+ except_star_block +// _tmp_213: except_block+ except_star_block static void * -_tmp_214_rule(Parser *p) +_tmp_213_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35458,21 +35463,21 @@ _tmp_214_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_214[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_block+ except_star_block")); - asdl_seq * _loop1_221_var; + D(fprintf(stderr, "%*c> _tmp_213[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_block+ except_star_block")); + asdl_seq * _loop1_220_var; excepthandler_ty except_star_block_var; if ( - (_loop1_221_var = _loop1_221_rule(p)) // except_block+ + (_loop1_220_var = _loop1_220_rule(p)) // except_block+ && (except_star_block_var = except_star_block_rule(p)) // except_star_block ) { - D(fprintf(stderr, "%*c+ _tmp_214[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "except_block+ except_star_block")); - _res = _PyPegen_dummy_name(p, _loop1_221_var, except_star_block_var); + D(fprintf(stderr, "%*c+ _tmp_213[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "except_block+ except_star_block")); + _res = _PyPegen_dummy_name(p, _loop1_220_var, except_star_block_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_214[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_213[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "except_block+ except_star_block")); } _res = NULL; @@ -35481,9 +35486,9 @@ _tmp_214_rule(Parser *p) return _res; } -// _tmp_215: except_star_block+ except_block +// _tmp_214: except_star_block+ except_block static void * -_tmp_215_rule(Parser *p) +_tmp_214_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35500,21 +35505,21 @@ _tmp_215_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_215[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_star_block+ except_block")); - asdl_seq * _loop1_222_var; + D(fprintf(stderr, "%*c> _tmp_214[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_star_block+ except_block")); + asdl_seq * _loop1_221_var; excepthandler_ty except_block_var; if ( - (_loop1_222_var = _loop1_222_rule(p)) // except_star_block+ + (_loop1_221_var = _loop1_221_rule(p)) // except_star_block+ && (except_block_var = except_block_rule(p)) // except_block ) { - D(fprintf(stderr, "%*c+ _tmp_215[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "except_star_block+ except_block")); - _res = _PyPegen_dummy_name(p, _loop1_222_var, except_block_var); + D(fprintf(stderr, "%*c+ _tmp_214[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "except_star_block+ except_block")); + _res = _PyPegen_dummy_name(p, _loop1_221_var, except_block_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_215[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_214[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "except_star_block+ except_block")); } _res = NULL; @@ -35523,9 +35528,9 @@ _tmp_215_rule(Parser *p) return _res; } -// _tmp_216: assignment_expression | expression !':=' +// _tmp_215: assignment_expression | expression !':=' static void * -_tmp_216_rule(Parser *p) +_tmp_215_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35542,18 +35547,18 @@ _tmp_216_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_216[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "assignment_expression")); + D(fprintf(stderr, "%*c> _tmp_215[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "assignment_expression")); expr_ty assignment_expression_var; if ( (assignment_expression_var = assignment_expression_rule(p)) // assignment_expression ) { - D(fprintf(stderr, "%*c+ _tmp_216[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "assignment_expression")); + D(fprintf(stderr, "%*c+ _tmp_215[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "assignment_expression")); _res = assignment_expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_216[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_215[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "assignment_expression")); } { // expression !':=' @@ -35561,7 +35566,7 @@ _tmp_216_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_216[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression !':='")); + D(fprintf(stderr, "%*c> _tmp_215[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression !':='")); expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression @@ -35569,12 +35574,12 @@ _tmp_216_rule(Parser *p) _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 53) // token=':=' ) { - D(fprintf(stderr, "%*c+ _tmp_216[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression !':='")); + D(fprintf(stderr, "%*c+ _tmp_215[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression !':='")); _res = expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_216[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_215[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression !':='")); } _res = NULL; @@ -35583,9 +35588,9 @@ _tmp_216_rule(Parser *p) return _res; } -// _tmp_217: 'as' star_target +// _tmp_216: 'as' star_target static void * -_tmp_217_rule(Parser *p) +_tmp_216_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35602,7 +35607,7 @@ _tmp_217_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_217[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_216[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -35611,12 +35616,12 @@ _tmp_217_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_217[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_216[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_217[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_216[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -35625,9 +35630,9 @@ _tmp_217_rule(Parser *p) return _res; } -// _tmp_218: 'as' star_target +// _tmp_217: 'as' star_target static void * -_tmp_218_rule(Parser *p) +_tmp_217_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35644,7 +35649,7 @@ _tmp_218_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_218[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_217[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -35653,12 +35658,12 @@ _tmp_218_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_218[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_217[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_218[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_217[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -35667,9 +35672,9 @@ _tmp_218_rule(Parser *p) return _res; } -// _tmp_219: 'as' star_target +// _tmp_218: 'as' star_target static void * -_tmp_219_rule(Parser *p) +_tmp_218_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35686,7 +35691,7 @@ _tmp_219_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_219[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_218[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -35695,12 +35700,12 @@ _tmp_219_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_219[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_218[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_219[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_218[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -35709,9 +35714,9 @@ _tmp_219_rule(Parser *p) return _res; } -// _tmp_220: 'as' star_target +// _tmp_219: 'as' star_target static void * -_tmp_220_rule(Parser *p) +_tmp_219_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35728,7 +35733,7 @@ _tmp_220_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_220[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_219[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -35737,12 +35742,12 @@ _tmp_220_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_220[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_219[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_220[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_219[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -35751,9 +35756,9 @@ _tmp_220_rule(Parser *p) return _res; } -// _loop1_221: except_block +// _loop1_220: except_block static asdl_seq * -_loop1_221_rule(Parser *p) +_loop1_220_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35780,7 +35785,7 @@ _loop1_221_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_221[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_block")); + D(fprintf(stderr, "%*c> _loop1_220[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_block")); excepthandler_ty except_block_var; while ( (except_block_var = except_block_rule(p)) // except_block @@ -35802,7 +35807,7 @@ _loop1_221_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_221[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_220[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "except_block")); } if (_n == 0 || p->error_indicator) { @@ -35820,14 +35825,14 @@ _loop1_221_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_221_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_220_type, _seq); p->level--; return _seq; } -// _loop1_222: except_star_block +// _loop1_221: except_star_block static asdl_seq * -_loop1_222_rule(Parser *p) +_loop1_221_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35854,7 +35859,7 @@ _loop1_222_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_222[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_star_block")); + D(fprintf(stderr, "%*c> _loop1_221[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_star_block")); excepthandler_ty except_star_block_var; while ( (except_star_block_var = except_star_block_rule(p)) // except_star_block @@ -35876,7 +35881,7 @@ _loop1_222_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_222[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_221[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "except_star_block")); } if (_n == 0 || p->error_indicator) { @@ -35894,7 +35899,7 @@ _loop1_222_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_222_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_221_type, _seq); p->level--; return _seq; } From webhook-mailer at python.org Wed Feb 9 22:54:56 2022 From: webhook-mailer at python.org (pablogsal) Date: Thu, 10 Feb 2022 03:54:56 -0000 Subject: [Python-checkins] [3.10] bpo-46707: Avoid potential exponential backtracking in some syntax errors (GH-31241). (GH-31242) Message-ID: https://github.com/python/cpython/commit/9b23f8f78fdb0d7eba016616ae7a97abbfc65aa6 commit: 9b23f8f78fdb0d7eba016616ae7a97abbfc65aa6 branch: 3.10 author: Pablo Galindo Salgado committer: pablogsal date: 2022-02-10T03:54:47Z summary: [3.10] bpo-46707: Avoid potential exponential backtracking in some syntax errors (GH-31241). (GH-31242) (cherry picked from commit b71dc71905ab674ccaa4a56230d17a28f61c325c) Co-authored-by: Pablo Galindo Salgado files: A Misc/NEWS.d/next/Core and Builtins/2022-02-10-03-13-18.bpo-46707.xeSEh0.rst M Grammar/python.gram M Lib/test/test_syntax.py M Parser/parser.c diff --git a/Grammar/python.gram b/Grammar/python.gram index f730658462147..69c20bddf46eb 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -863,13 +863,15 @@ invalid_expression: RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Perhaps you forgot a comma?") } | a=disjunction 'if' b=disjunction !('else'|':') { RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "expected 'else' after 'if' expression") } +invalid_left_assignment_prefixes(memo): list|tuple|genexp|'True'|'None'|'False' + invalid_named_expression: | a=expression ':=' expression { RAISE_SYNTAX_ERROR_KNOWN_LOCATION( a, "cannot use assignment expressions with %s", _PyPegen_get_expr_name(a)) } | a=NAME '=' b=bitwise_or !('='|':=') { p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?") } - | !(list|tuple|genexp|'True'|'None'|'False') a=bitwise_or b='=' bitwise_or !('='|':=') { + | !invalid_left_assignment_prefixes a=bitwise_or b='=' bitwise_or !('='|':=') { p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "cannot assign to %s here. Maybe you meant '==' instead of '='?", _PyPegen_get_expr_name(a)) } diff --git a/Lib/test/test_syntax.py b/Lib/test/test_syntax.py index ac5a41ce4cc67..09c0d5682a7e4 100644 --- a/Lib/test/test_syntax.py +++ b/Lib/test/test_syntax.py @@ -1617,6 +1617,14 @@ def test_error_on_parser_stack_overflow(self): with self.assertRaises(MemoryError): compile(source, "", mode) + @support.cpython_only + def test_deep_invalid_rule(self): + # Check that a very deep invalid rule in the PEG + # parser doesn't have exponential backtracking. + source = "d{{{{{{{{{{{{{{{{{{{{{{{{{```{{{{{{{ef f():y" + with self.assertRaises(SyntaxError): + compile(source, "", "exec") + def test_main(): support.run_unittest(SyntaxTestCase) diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-10-03-13-18.bpo-46707.xeSEh0.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-10-03-13-18.bpo-46707.xeSEh0.rst new file mode 100644 index 0000000000000..4b156c4d5f68b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-10-03-13-18.bpo-46707.xeSEh0.rst @@ -0,0 +1,2 @@ +Avoid potential exponential backtracking when producing some syntax errors +involving lots of brackets. Patch by Pablo Galindo. diff --git a/Parser/parser.c b/Parser/parser.c index 7e6f04898e1e3..763716982ed83 100644 --- a/Parser/parser.c +++ b/Parser/parser.c @@ -256,250 +256,250 @@ static char *soft_keywords[] = { #define expression_without_invalid_type 1180 #define invalid_legacy_expression_type 1181 #define invalid_expression_type 1182 -#define invalid_named_expression_type 1183 -#define invalid_assignment_type 1184 -#define invalid_ann_assign_target_type 1185 -#define invalid_del_stmt_type 1186 -#define invalid_block_type 1187 -#define invalid_comprehension_type 1188 -#define invalid_dict_comprehension_type 1189 -#define invalid_parameters_type 1190 -#define invalid_parameters_helper_type 1191 -#define invalid_lambda_parameters_type 1192 -#define invalid_lambda_parameters_helper_type 1193 -#define invalid_star_etc_type 1194 -#define invalid_lambda_star_etc_type 1195 -#define invalid_double_type_comments_type 1196 -#define invalid_with_item_type 1197 -#define invalid_for_target_type 1198 -#define invalid_group_type 1199 -#define invalid_import_from_targets_type 1200 -#define invalid_with_stmt_type 1201 -#define invalid_with_stmt_indent_type 1202 -#define invalid_try_stmt_type 1203 -#define invalid_except_stmt_type 1204 -#define invalid_finally_stmt_type 1205 -#define invalid_except_stmt_indent_type 1206 -#define invalid_match_stmt_type 1207 -#define invalid_case_block_type 1208 -#define invalid_as_pattern_type 1209 -#define invalid_class_pattern_type 1210 -#define invalid_class_argument_pattern_type 1211 -#define invalid_if_stmt_type 1212 -#define invalid_elif_stmt_type 1213 -#define invalid_else_stmt_type 1214 -#define invalid_while_stmt_type 1215 -#define invalid_for_stmt_type 1216 -#define invalid_def_raw_type 1217 -#define invalid_class_def_raw_type 1218 -#define invalid_double_starred_kvpairs_type 1219 -#define invalid_kvpair_type 1220 -#define _loop0_1_type 1221 -#define _loop0_2_type 1222 -#define _loop0_4_type 1223 -#define _gather_3_type 1224 -#define _loop0_6_type 1225 -#define _gather_5_type 1226 -#define _loop0_8_type 1227 -#define _gather_7_type 1228 -#define _loop0_10_type 1229 -#define _gather_9_type 1230 -#define _loop1_11_type 1231 -#define _loop0_13_type 1232 -#define _gather_12_type 1233 -#define _tmp_14_type 1234 -#define _tmp_15_type 1235 -#define _tmp_16_type 1236 -#define _tmp_17_type 1237 -#define _tmp_18_type 1238 -#define _tmp_19_type 1239 -#define _tmp_20_type 1240 -#define _tmp_21_type 1241 -#define _loop1_22_type 1242 -#define _tmp_23_type 1243 -#define _tmp_24_type 1244 -#define _loop0_26_type 1245 -#define _gather_25_type 1246 -#define _loop0_28_type 1247 -#define _gather_27_type 1248 -#define _tmp_29_type 1249 -#define _tmp_30_type 1250 -#define _loop0_31_type 1251 -#define _loop1_32_type 1252 -#define _loop0_34_type 1253 -#define _gather_33_type 1254 -#define _tmp_35_type 1255 -#define _loop0_37_type 1256 -#define _gather_36_type 1257 -#define _tmp_38_type 1258 -#define _loop0_40_type 1259 -#define _gather_39_type 1260 -#define _loop0_42_type 1261 -#define _gather_41_type 1262 -#define _loop0_44_type 1263 -#define _gather_43_type 1264 -#define _loop0_46_type 1265 -#define _gather_45_type 1266 -#define _tmp_47_type 1267 -#define _loop1_48_type 1268 -#define _tmp_49_type 1269 -#define _loop1_50_type 1270 -#define _loop0_52_type 1271 -#define _gather_51_type 1272 -#define _tmp_53_type 1273 -#define _tmp_54_type 1274 -#define _tmp_55_type 1275 -#define _tmp_56_type 1276 -#define _loop0_58_type 1277 -#define _gather_57_type 1278 -#define _loop0_60_type 1279 -#define _gather_59_type 1280 -#define _tmp_61_type 1281 -#define _loop0_63_type 1282 -#define _gather_62_type 1283 -#define _loop0_65_type 1284 -#define _gather_64_type 1285 -#define _tmp_66_type 1286 -#define _tmp_67_type 1287 -#define _tmp_68_type 1288 -#define _tmp_69_type 1289 -#define _loop0_70_type 1290 -#define _loop0_71_type 1291 -#define _loop0_72_type 1292 -#define _loop1_73_type 1293 -#define _loop0_74_type 1294 -#define _loop1_75_type 1295 -#define _loop1_76_type 1296 -#define _loop1_77_type 1297 -#define _loop0_78_type 1298 -#define _loop1_79_type 1299 -#define _loop0_80_type 1300 -#define _loop1_81_type 1301 -#define _loop0_82_type 1302 -#define _loop1_83_type 1303 -#define _loop1_84_type 1304 -#define _tmp_85_type 1305 -#define _loop1_86_type 1306 -#define _loop0_88_type 1307 -#define _gather_87_type 1308 -#define _loop1_89_type 1309 -#define _loop0_90_type 1310 -#define _loop0_91_type 1311 -#define _loop0_92_type 1312 -#define _loop1_93_type 1313 -#define _loop0_94_type 1314 -#define _loop1_95_type 1315 -#define _loop1_96_type 1316 -#define _loop1_97_type 1317 -#define _loop0_98_type 1318 -#define _loop1_99_type 1319 -#define _loop0_100_type 1320 -#define _loop1_101_type 1321 -#define _loop0_102_type 1322 -#define _loop1_103_type 1323 -#define _loop1_104_type 1324 -#define _loop1_105_type 1325 -#define _loop1_106_type 1326 -#define _tmp_107_type 1327 -#define _loop0_109_type 1328 -#define _gather_108_type 1329 -#define _tmp_110_type 1330 -#define _tmp_111_type 1331 -#define _tmp_112_type 1332 -#define _tmp_113_type 1333 -#define _loop1_114_type 1334 -#define _tmp_115_type 1335 -#define _tmp_116_type 1336 -#define _tmp_117_type 1337 -#define _loop0_119_type 1338 -#define _gather_118_type 1339 -#define _loop1_120_type 1340 -#define _loop0_121_type 1341 -#define _loop0_122_type 1342 -#define _loop0_124_type 1343 -#define _gather_123_type 1344 -#define _tmp_125_type 1345 -#define _loop0_127_type 1346 -#define _gather_126_type 1347 -#define _loop0_129_type 1348 -#define _gather_128_type 1349 -#define _loop0_131_type 1350 -#define _gather_130_type 1351 -#define _loop0_133_type 1352 -#define _gather_132_type 1353 -#define _loop0_134_type 1354 -#define _loop0_136_type 1355 -#define _gather_135_type 1356 -#define _loop1_137_type 1357 -#define _tmp_138_type 1358 -#define _loop0_140_type 1359 -#define _gather_139_type 1360 -#define _tmp_141_type 1361 -#define _tmp_142_type 1362 -#define _tmp_143_type 1363 -#define _tmp_144_type 1364 -#define _tmp_145_type 1365 -#define _tmp_146_type 1366 -#define _tmp_147_type 1367 -#define _tmp_148_type 1368 -#define _loop0_149_type 1369 -#define _loop0_150_type 1370 -#define _loop0_151_type 1371 -#define _tmp_152_type 1372 -#define _tmp_153_type 1373 -#define _tmp_154_type 1374 -#define _tmp_155_type 1375 -#define _loop0_156_type 1376 -#define _loop1_157_type 1377 -#define _loop0_158_type 1378 -#define _loop1_159_type 1379 -#define _tmp_160_type 1380 -#define _tmp_161_type 1381 -#define _tmp_162_type 1382 -#define _loop0_164_type 1383 -#define _gather_163_type 1384 -#define _loop0_166_type 1385 -#define _gather_165_type 1386 -#define _loop0_168_type 1387 -#define _gather_167_type 1388 -#define _loop0_170_type 1389 -#define _gather_169_type 1390 -#define _tmp_171_type 1391 -#define _tmp_172_type 1392 -#define _tmp_173_type 1393 -#define _tmp_174_type 1394 -#define _tmp_175_type 1395 -#define _tmp_176_type 1396 -#define _tmp_177_type 1397 -#define _loop0_179_type 1398 -#define _gather_178_type 1399 -#define _tmp_180_type 1400 -#define _tmp_181_type 1401 -#define _tmp_182_type 1402 -#define _tmp_183_type 1403 -#define _tmp_184_type 1404 -#define _tmp_185_type 1405 -#define _tmp_186_type 1406 -#define _tmp_187_type 1407 -#define _tmp_188_type 1408 -#define _tmp_189_type 1409 -#define _tmp_190_type 1410 -#define _tmp_191_type 1411 -#define _tmp_192_type 1412 -#define _tmp_193_type 1413 -#define _tmp_194_type 1414 -#define _tmp_195_type 1415 -#define _tmp_196_type 1416 -#define _tmp_197_type 1417 -#define _tmp_198_type 1418 -#define _tmp_199_type 1419 -#define _tmp_200_type 1420 -#define _tmp_201_type 1421 -#define _tmp_202_type 1422 -#define _tmp_203_type 1423 -#define _tmp_204_type 1424 -#define _tmp_205_type 1425 -#define _tmp_206_type 1426 +#define invalid_left_assignment_prefixes_type 1183 +#define invalid_named_expression_type 1184 +#define invalid_assignment_type 1185 +#define invalid_ann_assign_target_type 1186 +#define invalid_del_stmt_type 1187 +#define invalid_block_type 1188 +#define invalid_comprehension_type 1189 +#define invalid_dict_comprehension_type 1190 +#define invalid_parameters_type 1191 +#define invalid_parameters_helper_type 1192 +#define invalid_lambda_parameters_type 1193 +#define invalid_lambda_parameters_helper_type 1194 +#define invalid_star_etc_type 1195 +#define invalid_lambda_star_etc_type 1196 +#define invalid_double_type_comments_type 1197 +#define invalid_with_item_type 1198 +#define invalid_for_target_type 1199 +#define invalid_group_type 1200 +#define invalid_import_from_targets_type 1201 +#define invalid_with_stmt_type 1202 +#define invalid_with_stmt_indent_type 1203 +#define invalid_try_stmt_type 1204 +#define invalid_except_stmt_type 1205 +#define invalid_finally_stmt_type 1206 +#define invalid_except_stmt_indent_type 1207 +#define invalid_match_stmt_type 1208 +#define invalid_case_block_type 1209 +#define invalid_as_pattern_type 1210 +#define invalid_class_pattern_type 1211 +#define invalid_class_argument_pattern_type 1212 +#define invalid_if_stmt_type 1213 +#define invalid_elif_stmt_type 1214 +#define invalid_else_stmt_type 1215 +#define invalid_while_stmt_type 1216 +#define invalid_for_stmt_type 1217 +#define invalid_def_raw_type 1218 +#define invalid_class_def_raw_type 1219 +#define invalid_double_starred_kvpairs_type 1220 +#define invalid_kvpair_type 1221 +#define _loop0_1_type 1222 +#define _loop0_2_type 1223 +#define _loop0_4_type 1224 +#define _gather_3_type 1225 +#define _loop0_6_type 1226 +#define _gather_5_type 1227 +#define _loop0_8_type 1228 +#define _gather_7_type 1229 +#define _loop0_10_type 1230 +#define _gather_9_type 1231 +#define _loop1_11_type 1232 +#define _loop0_13_type 1233 +#define _gather_12_type 1234 +#define _tmp_14_type 1235 +#define _tmp_15_type 1236 +#define _tmp_16_type 1237 +#define _tmp_17_type 1238 +#define _tmp_18_type 1239 +#define _tmp_19_type 1240 +#define _tmp_20_type 1241 +#define _tmp_21_type 1242 +#define _loop1_22_type 1243 +#define _tmp_23_type 1244 +#define _tmp_24_type 1245 +#define _loop0_26_type 1246 +#define _gather_25_type 1247 +#define _loop0_28_type 1248 +#define _gather_27_type 1249 +#define _tmp_29_type 1250 +#define _tmp_30_type 1251 +#define _loop0_31_type 1252 +#define _loop1_32_type 1253 +#define _loop0_34_type 1254 +#define _gather_33_type 1255 +#define _tmp_35_type 1256 +#define _loop0_37_type 1257 +#define _gather_36_type 1258 +#define _tmp_38_type 1259 +#define _loop0_40_type 1260 +#define _gather_39_type 1261 +#define _loop0_42_type 1262 +#define _gather_41_type 1263 +#define _loop0_44_type 1264 +#define _gather_43_type 1265 +#define _loop0_46_type 1266 +#define _gather_45_type 1267 +#define _tmp_47_type 1268 +#define _loop1_48_type 1269 +#define _tmp_49_type 1270 +#define _loop1_50_type 1271 +#define _loop0_52_type 1272 +#define _gather_51_type 1273 +#define _tmp_53_type 1274 +#define _tmp_54_type 1275 +#define _tmp_55_type 1276 +#define _tmp_56_type 1277 +#define _loop0_58_type 1278 +#define _gather_57_type 1279 +#define _loop0_60_type 1280 +#define _gather_59_type 1281 +#define _tmp_61_type 1282 +#define _loop0_63_type 1283 +#define _gather_62_type 1284 +#define _loop0_65_type 1285 +#define _gather_64_type 1286 +#define _tmp_66_type 1287 +#define _tmp_67_type 1288 +#define _tmp_68_type 1289 +#define _tmp_69_type 1290 +#define _loop0_70_type 1291 +#define _loop0_71_type 1292 +#define _loop0_72_type 1293 +#define _loop1_73_type 1294 +#define _loop0_74_type 1295 +#define _loop1_75_type 1296 +#define _loop1_76_type 1297 +#define _loop1_77_type 1298 +#define _loop0_78_type 1299 +#define _loop1_79_type 1300 +#define _loop0_80_type 1301 +#define _loop1_81_type 1302 +#define _loop0_82_type 1303 +#define _loop1_83_type 1304 +#define _loop1_84_type 1305 +#define _tmp_85_type 1306 +#define _loop1_86_type 1307 +#define _loop0_88_type 1308 +#define _gather_87_type 1309 +#define _loop1_89_type 1310 +#define _loop0_90_type 1311 +#define _loop0_91_type 1312 +#define _loop0_92_type 1313 +#define _loop1_93_type 1314 +#define _loop0_94_type 1315 +#define _loop1_95_type 1316 +#define _loop1_96_type 1317 +#define _loop1_97_type 1318 +#define _loop0_98_type 1319 +#define _loop1_99_type 1320 +#define _loop0_100_type 1321 +#define _loop1_101_type 1322 +#define _loop0_102_type 1323 +#define _loop1_103_type 1324 +#define _loop1_104_type 1325 +#define _loop1_105_type 1326 +#define _loop1_106_type 1327 +#define _tmp_107_type 1328 +#define _loop0_109_type 1329 +#define _gather_108_type 1330 +#define _tmp_110_type 1331 +#define _tmp_111_type 1332 +#define _tmp_112_type 1333 +#define _tmp_113_type 1334 +#define _loop1_114_type 1335 +#define _tmp_115_type 1336 +#define _tmp_116_type 1337 +#define _tmp_117_type 1338 +#define _loop0_119_type 1339 +#define _gather_118_type 1340 +#define _loop1_120_type 1341 +#define _loop0_121_type 1342 +#define _loop0_122_type 1343 +#define _loop0_124_type 1344 +#define _gather_123_type 1345 +#define _tmp_125_type 1346 +#define _loop0_127_type 1347 +#define _gather_126_type 1348 +#define _loop0_129_type 1349 +#define _gather_128_type 1350 +#define _loop0_131_type 1351 +#define _gather_130_type 1352 +#define _loop0_133_type 1353 +#define _gather_132_type 1354 +#define _loop0_134_type 1355 +#define _loop0_136_type 1356 +#define _gather_135_type 1357 +#define _loop1_137_type 1358 +#define _tmp_138_type 1359 +#define _loop0_140_type 1360 +#define _gather_139_type 1361 +#define _tmp_141_type 1362 +#define _tmp_142_type 1363 +#define _tmp_143_type 1364 +#define _tmp_144_type 1365 +#define _tmp_145_type 1366 +#define _tmp_146_type 1367 +#define _tmp_147_type 1368 +#define _loop0_148_type 1369 +#define _loop0_149_type 1370 +#define _loop0_150_type 1371 +#define _tmp_151_type 1372 +#define _tmp_152_type 1373 +#define _tmp_153_type 1374 +#define _tmp_154_type 1375 +#define _loop0_155_type 1376 +#define _loop1_156_type 1377 +#define _loop0_157_type 1378 +#define _loop1_158_type 1379 +#define _tmp_159_type 1380 +#define _tmp_160_type 1381 +#define _tmp_161_type 1382 +#define _loop0_163_type 1383 +#define _gather_162_type 1384 +#define _loop0_165_type 1385 +#define _gather_164_type 1386 +#define _loop0_167_type 1387 +#define _gather_166_type 1388 +#define _loop0_169_type 1389 +#define _gather_168_type 1390 +#define _tmp_170_type 1391 +#define _tmp_171_type 1392 +#define _tmp_172_type 1393 +#define _tmp_173_type 1394 +#define _tmp_174_type 1395 +#define _tmp_175_type 1396 +#define _tmp_176_type 1397 +#define _loop0_178_type 1398 +#define _gather_177_type 1399 +#define _tmp_179_type 1400 +#define _tmp_180_type 1401 +#define _tmp_181_type 1402 +#define _tmp_182_type 1403 +#define _tmp_183_type 1404 +#define _tmp_184_type 1405 +#define _tmp_185_type 1406 +#define _tmp_186_type 1407 +#define _tmp_187_type 1408 +#define _tmp_188_type 1409 +#define _tmp_189_type 1410 +#define _tmp_190_type 1411 +#define _tmp_191_type 1412 +#define _tmp_192_type 1413 +#define _tmp_193_type 1414 +#define _tmp_194_type 1415 +#define _tmp_195_type 1416 +#define _tmp_196_type 1417 +#define _tmp_197_type 1418 +#define _tmp_198_type 1419 +#define _tmp_199_type 1420 +#define _tmp_200_type 1421 +#define _tmp_201_type 1422 +#define _tmp_202_type 1423 +#define _tmp_203_type 1424 +#define _tmp_204_type 1425 +#define _tmp_205_type 1426 static mod_ty file_rule(Parser *p); static mod_ty interactive_rule(Parser *p); @@ -684,6 +684,7 @@ static void *invalid_kwarg_rule(Parser *p); static expr_ty expression_without_invalid_rule(Parser *p); static void *invalid_legacy_expression_rule(Parser *p); static void *invalid_expression_rule(Parser *p); +static void *invalid_left_assignment_prefixes_rule(Parser *p); static void *invalid_named_expression_rule(Parser *p); static void *invalid_assignment_rule(Parser *p); static expr_ty invalid_ann_assign_target_rule(Parser *p); @@ -869,38 +870,38 @@ static void *_tmp_144_rule(Parser *p); static void *_tmp_145_rule(Parser *p); static void *_tmp_146_rule(Parser *p); static void *_tmp_147_rule(Parser *p); -static void *_tmp_148_rule(Parser *p); +static asdl_seq *_loop0_148_rule(Parser *p); static asdl_seq *_loop0_149_rule(Parser *p); static asdl_seq *_loop0_150_rule(Parser *p); -static asdl_seq *_loop0_151_rule(Parser *p); +static void *_tmp_151_rule(Parser *p); static void *_tmp_152_rule(Parser *p); static void *_tmp_153_rule(Parser *p); static void *_tmp_154_rule(Parser *p); -static void *_tmp_155_rule(Parser *p); -static asdl_seq *_loop0_156_rule(Parser *p); -static asdl_seq *_loop1_157_rule(Parser *p); -static asdl_seq *_loop0_158_rule(Parser *p); -static asdl_seq *_loop1_159_rule(Parser *p); +static asdl_seq *_loop0_155_rule(Parser *p); +static asdl_seq *_loop1_156_rule(Parser *p); +static asdl_seq *_loop0_157_rule(Parser *p); +static asdl_seq *_loop1_158_rule(Parser *p); +static void *_tmp_159_rule(Parser *p); static void *_tmp_160_rule(Parser *p); static void *_tmp_161_rule(Parser *p); -static void *_tmp_162_rule(Parser *p); -static asdl_seq *_loop0_164_rule(Parser *p); -static asdl_seq *_gather_163_rule(Parser *p); -static asdl_seq *_loop0_166_rule(Parser *p); -static asdl_seq *_gather_165_rule(Parser *p); -static asdl_seq *_loop0_168_rule(Parser *p); -static asdl_seq *_gather_167_rule(Parser *p); -static asdl_seq *_loop0_170_rule(Parser *p); -static asdl_seq *_gather_169_rule(Parser *p); +static asdl_seq *_loop0_163_rule(Parser *p); +static asdl_seq *_gather_162_rule(Parser *p); +static asdl_seq *_loop0_165_rule(Parser *p); +static asdl_seq *_gather_164_rule(Parser *p); +static asdl_seq *_loop0_167_rule(Parser *p); +static asdl_seq *_gather_166_rule(Parser *p); +static asdl_seq *_loop0_169_rule(Parser *p); +static asdl_seq *_gather_168_rule(Parser *p); +static void *_tmp_170_rule(Parser *p); static void *_tmp_171_rule(Parser *p); static void *_tmp_172_rule(Parser *p); static void *_tmp_173_rule(Parser *p); static void *_tmp_174_rule(Parser *p); static void *_tmp_175_rule(Parser *p); static void *_tmp_176_rule(Parser *p); -static void *_tmp_177_rule(Parser *p); -static asdl_seq *_loop0_179_rule(Parser *p); -static asdl_seq *_gather_178_rule(Parser *p); +static asdl_seq *_loop0_178_rule(Parser *p); +static asdl_seq *_gather_177_rule(Parser *p); +static void *_tmp_179_rule(Parser *p); static void *_tmp_180_rule(Parser *p); static void *_tmp_181_rule(Parser *p); static void *_tmp_182_rule(Parser *p); @@ -927,7 +928,6 @@ static void *_tmp_202_rule(Parser *p); static void *_tmp_203_rule(Parser *p); static void *_tmp_204_rule(Parser *p); static void *_tmp_205_rule(Parser *p); -static void *_tmp_206_rule(Parser *p); // file: statements? $ @@ -18939,10 +18939,149 @@ invalid_expression_rule(Parser *p) return _res; } +// invalid_left_assignment_prefixes: list | tuple | genexp | 'True' | 'None' | 'False' +static void * +invalid_left_assignment_prefixes_rule(Parser *p) +{ + if (p->level++ == MAXSTACK) { + p->error_indicator = 1; + PyErr_NoMemory(); + } + if (p->error_indicator) { + p->level--; + return NULL; + } + void * _res = NULL; + if (_PyPegen_is_memoized(p, invalid_left_assignment_prefixes_type, &_res)) { + p->level--; + return _res; + } + int _mark = p->mark; + { // list + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "list")); + expr_ty list_var; + if ( + (list_var = list_rule(p)) // list + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "list")); + _res = list_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "list")); + } + { // tuple + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "tuple")); + expr_ty tuple_var; + if ( + (tuple_var = tuple_rule(p)) // tuple + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "tuple")); + _res = tuple_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "tuple")); + } + { // genexp + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "genexp")); + expr_ty genexp_var; + if ( + (genexp_var = genexp_rule(p)) // genexp + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "genexp")); + _res = genexp_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "genexp")); + } + { // 'True' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 524)) // token='True' + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'True'")); + } + { // 'None' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 523)) // token='None' + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'None'")); + } + { // 'False' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 525)) // token='False' + ) + { + D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'False'")); + } + _res = NULL; + done: + _PyPegen_insert_memo(p, _mark, invalid_left_assignment_prefixes_type, _res); + p->level--; + return _res; +} + // invalid_named_expression: // | expression ':=' expression // | NAME '=' bitwise_or !('=' | ':=') -// | !(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=') +// | !invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=') static void * invalid_named_expression_rule(Parser *p) { @@ -19018,17 +19157,17 @@ invalid_named_expression_rule(Parser *p) D(fprintf(stderr, "%*c%s invalid_named_expression[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "NAME '=' bitwise_or !('=' | ':=')")); } - { // !(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=') + { // !invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=') if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_named_expression[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); + D(fprintf(stderr, "%*c> invalid_named_expression[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); expr_ty a; Token * b; expr_ty bitwise_or_var; if ( - _PyPegen_lookahead(0, _tmp_147_rule, p) + _PyPegen_lookahead(0, invalid_left_assignment_prefixes_rule, p) && (a = bitwise_or_rule(p)) // bitwise_or && @@ -19036,10 +19175,10 @@ invalid_named_expression_rule(Parser *p) && (bitwise_or_var = bitwise_or_rule(p)) // bitwise_or && - _PyPegen_lookahead(0, _tmp_148_rule, p) + _PyPegen_lookahead(0, _tmp_147_rule, p) ) { - D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); + D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); _res = p -> in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_LOCATION ( a , "cannot assign to %s here. Maybe you meant '==' instead of '='?" , _PyPegen_get_expr_name ( a ) ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -19050,7 +19189,7 @@ invalid_named_expression_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_named_expression[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); } _res = NULL; done: @@ -19116,7 +19255,7 @@ invalid_assignment_rule(Parser *p) D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expression ',' star_named_expressions* ':' expression")); Token * _literal; Token * _literal_1; - asdl_seq * _loop0_149_var; + asdl_seq * _loop0_148_var; expr_ty a; expr_ty expression_var; if ( @@ -19124,7 +19263,7 @@ invalid_assignment_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_loop0_149_var = _loop0_149_rule(p)) // star_named_expressions* + (_loop0_148_var = _loop0_148_rule(p)) // star_named_expressions* && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -19181,10 +19320,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* star_expressions '='")); Token * _literal; - asdl_seq * _loop0_150_var; + asdl_seq * _loop0_149_var; expr_ty a; if ( - (_loop0_150_var = _loop0_150_rule(p)) // ((star_targets '='))* + (_loop0_149_var = _loop0_149_rule(p)) // ((star_targets '='))* && (a = star_expressions_rule(p)) // star_expressions && @@ -19211,10 +19350,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* yield_expr '='")); Token * _literal; - asdl_seq * _loop0_151_var; + asdl_seq * _loop0_150_var; expr_ty a; if ( - (_loop0_151_var = _loop0_151_rule(p)) // ((star_targets '='))* + (_loop0_150_var = _loop0_150_rule(p)) // ((star_targets '='))* && (a = yield_expr_rule(p)) // yield_expr && @@ -19240,7 +19379,7 @@ invalid_assignment_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); - void *_tmp_152_var; + void *_tmp_151_var; expr_ty a; AugOperator* augassign_var; if ( @@ -19248,7 +19387,7 @@ invalid_assignment_rule(Parser *p) && (augassign_var = augassign_rule(p)) // augassign && - (_tmp_152_var = _tmp_152_rule(p)) // yield_expr | star_expressions + (_tmp_151_var = _tmp_151_rule(p)) // yield_expr | star_expressions ) { D(fprintf(stderr, "%*c+ invalid_assignment[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); @@ -19474,11 +19613,11 @@ invalid_comprehension_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '(' | '{') starred_expression for_if_clauses")); - void *_tmp_153_var; + void *_tmp_152_var; expr_ty a; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_153_var = _tmp_153_rule(p)) // '[' | '(' | '{' + (_tmp_152_var = _tmp_152_rule(p)) // '[' | '(' | '{' && (a = starred_expression_rule(p)) // starred_expression && @@ -19505,12 +19644,12 @@ invalid_comprehension_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '{') star_named_expression ',' star_named_expressions for_if_clauses")); Token * _literal; - void *_tmp_154_var; + void *_tmp_153_var; expr_ty a; asdl_expr_seq* b; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_154_var = _tmp_154_rule(p)) // '[' | '{' + (_tmp_153_var = _tmp_153_rule(p)) // '[' | '{' && (a = star_named_expression_rule(p)) // star_named_expression && @@ -19540,12 +19679,12 @@ invalid_comprehension_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '{') star_named_expression ',' for_if_clauses")); - void *_tmp_155_var; + void *_tmp_154_var; expr_ty a; Token * b; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_155_var = _tmp_155_rule(p)) // '[' | '{' + (_tmp_154_var = _tmp_154_rule(p)) // '[' | '{' && (a = star_named_expression_rule(p)) // star_named_expression && @@ -19649,11 +19788,11 @@ invalid_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default* invalid_parameters_helper param_no_default")); - asdl_seq * _loop0_156_var; + asdl_seq * _loop0_155_var; arg_ty a; void *invalid_parameters_helper_var; if ( - (_loop0_156_var = _loop0_156_rule(p)) // param_no_default* + (_loop0_155_var = _loop0_155_rule(p)) // param_no_default* && (invalid_parameters_helper_var = invalid_parameters_helper_rule(p)) // invalid_parameters_helper && @@ -19723,13 +19862,13 @@ invalid_parameters_helper_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters_helper[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - asdl_seq * _loop1_157_var; + asdl_seq * _loop1_156_var; if ( - (_loop1_157_var = _loop1_157_rule(p)) // param_with_default+ + (_loop1_156_var = _loop1_156_rule(p)) // param_with_default+ ) { D(fprintf(stderr, "%*c+ invalid_parameters_helper[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - _res = _loop1_157_var; + _res = _loop1_156_var; goto done; } p->mark = _mark; @@ -19763,11 +19902,11 @@ invalid_lambda_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default* invalid_lambda_parameters_helper lambda_param_no_default")); - asdl_seq * _loop0_158_var; + asdl_seq * _loop0_157_var; arg_ty a; void *invalid_lambda_parameters_helper_var; if ( - (_loop0_158_var = _loop0_158_rule(p)) // lambda_param_no_default* + (_loop0_157_var = _loop0_157_rule(p)) // lambda_param_no_default* && (invalid_lambda_parameters_helper_var = invalid_lambda_parameters_helper_rule(p)) // invalid_lambda_parameters_helper && @@ -19839,13 +19978,13 @@ invalid_lambda_parameters_helper_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters_helper[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - asdl_seq * _loop1_159_var; + asdl_seq * _loop1_158_var; if ( - (_loop1_159_var = _loop1_159_rule(p)) // lambda_param_with_default+ + (_loop1_158_var = _loop1_158_rule(p)) // lambda_param_with_default+ ) { D(fprintf(stderr, "%*c+ invalid_lambda_parameters_helper[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - _res = _loop1_159_var; + _res = _loop1_158_var; goto done; } p->mark = _mark; @@ -19878,12 +20017,12 @@ invalid_star_etc_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); - void *_tmp_160_var; + void *_tmp_159_var; Token * a; if ( (a = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_160_var = _tmp_160_rule(p)) // ')' | ',' (')' | '**') + (_tmp_159_var = _tmp_159_rule(p)) // ')' | ',' (')' | '**') ) { D(fprintf(stderr, "%*c+ invalid_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); @@ -19956,11 +20095,11 @@ invalid_lambda_star_etc_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_lambda_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); Token * _literal; - void *_tmp_161_var; + void *_tmp_160_var; if ( (_literal = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_161_var = _tmp_161_rule(p)) // ':' | ',' (':' | '**') + (_tmp_160_var = _tmp_160_rule(p)) // ':' | ',' (':' | '**') ) { D(fprintf(stderr, "%*c+ invalid_lambda_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); @@ -20068,7 +20207,7 @@ invalid_with_item_rule(Parser *p) && (a = expression_rule(p)) // expression && - _PyPegen_lookahead(1, _tmp_162_rule, p) + _PyPegen_lookahead(1, _tmp_161_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_with_item[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression 'as' expression &(',' | ')' | ':')")); @@ -20296,7 +20435,7 @@ invalid_with_stmt_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ &&':'")); - asdl_seq * _gather_163_var; + asdl_seq * _gather_162_var; Token * _keyword; Token * _literal; void *_opt_var; @@ -20306,13 +20445,13 @@ invalid_with_stmt_rule(Parser *p) && (_keyword = _PyPegen_expect_token(p, 519)) // token='with' && - (_gather_163_var = _gather_163_rule(p)) // ','.(expression ['as' star_target])+ + (_gather_162_var = _gather_162_rule(p)) // ','.(expression ['as' star_target])+ && (_literal = _PyPegen_expect_forced_token(p, 11, ":")) // forced_token=':' ) { D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ &&':'")); - _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _gather_163_var, _literal); + _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _gather_162_var, _literal); goto done; } p->mark = _mark; @@ -20325,7 +20464,7 @@ invalid_with_stmt_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' &&':'")); - asdl_seq * _gather_165_var; + asdl_seq * _gather_164_var; Token * _keyword; Token * _literal; Token * _literal_1; @@ -20341,7 +20480,7 @@ invalid_with_stmt_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_165_var = _gather_165_rule(p)) // ','.(expressions ['as' star_target])+ + (_gather_164_var = _gather_164_rule(p)) // ','.(expressions ['as' star_target])+ && (_opt_var_1 = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20351,7 +20490,7 @@ invalid_with_stmt_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' &&':'")); - _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _literal, _gather_165_var, _opt_var_1, _literal_1, _literal_2); + _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _literal, _gather_164_var, _opt_var_1, _literal_1, _literal_2); goto done; } p->mark = _mark; @@ -20386,7 +20525,7 @@ invalid_with_stmt_indent_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); - asdl_seq * _gather_167_var; + asdl_seq * _gather_166_var; Token * _literal; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings @@ -20397,7 +20536,7 @@ invalid_with_stmt_indent_rule(Parser *p) && (a = _PyPegen_expect_token(p, 519)) // token='with' && - (_gather_167_var = _gather_167_rule(p)) // ','.(expression ['as' star_target])+ + (_gather_166_var = _gather_166_rule(p)) // ','.(expression ['as' star_target])+ && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -20425,7 +20564,7 @@ invalid_with_stmt_indent_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); - asdl_seq * _gather_169_var; + asdl_seq * _gather_168_var; Token * _literal; Token * _literal_1; Token * _literal_2; @@ -20442,7 +20581,7 @@ invalid_with_stmt_indent_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_169_var = _gather_169_rule(p)) // ','.(expressions ['as' star_target])+ + (_gather_168_var = _gather_168_rule(p)) // ','.(expressions ['as' star_target])+ && (_opt_var_1 = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20536,7 +20675,7 @@ invalid_try_stmt_rule(Parser *p) && (block_var = block_rule(p)) // block && - _PyPegen_lookahead(0, _tmp_171_rule, p) + _PyPegen_lookahead(0, _tmp_170_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_try_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'try' ':' block !('except' | 'finally')")); @@ -20597,7 +20736,7 @@ invalid_except_stmt_rule(Parser *p) && (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_172_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_171_rule(p), !p->error_indicator) // ['as' NAME] && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' ) @@ -20631,7 +20770,7 @@ invalid_except_stmt_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_173_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_172_rule(p), !p->error_indicator) // ['as' NAME] && (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) @@ -20767,7 +20906,7 @@ invalid_except_stmt_indent_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_174_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_173_rule(p), !p->error_indicator) // ['as' NAME] && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21166,7 +21305,7 @@ invalid_class_argument_pattern_rule(Parser *p) asdl_pattern_seq* a; asdl_seq* keyword_patterns_var; if ( - (_opt_var = _tmp_175_rule(p), !p->error_indicator) // [positional_patterns ','] + (_opt_var = _tmp_174_rule(p), !p->error_indicator) // [positional_patterns ','] && (keyword_patterns_var = keyword_patterns_rule(p)) // keyword_patterns && @@ -21618,7 +21757,7 @@ invalid_def_raw_rule(Parser *p) && (_literal_1 = _PyPegen_expect_token(p, 8)) // token=')' && - (_opt_var_2 = _tmp_176_rule(p), !p->error_indicator) // ['->' expression] + (_opt_var_2 = _tmp_175_rule(p), !p->error_indicator) // ['->' expression] && (_literal_2 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21677,7 +21816,7 @@ invalid_class_def_raw_rule(Parser *p) && (name_var = _PyPegen_name_token(p)) // NAME && - (_opt_var = _tmp_177_rule(p), !p->error_indicator) // ['(' arguments? ')'] + (_opt_var = _tmp_176_rule(p), !p->error_indicator) // ['(' arguments? ')'] && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21728,11 +21867,11 @@ invalid_double_starred_kvpairs_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_double_starred_kvpairs[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.double_starred_kvpair+ ',' invalid_kvpair")); - asdl_seq * _gather_178_var; + asdl_seq * _gather_177_var; Token * _literal; void *invalid_kvpair_var; if ( - (_gather_178_var = _gather_178_rule(p)) // ','.double_starred_kvpair+ + (_gather_177_var = _gather_177_rule(p)) // ','.double_starred_kvpair+ && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && @@ -21740,7 +21879,7 @@ invalid_double_starred_kvpairs_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_double_starred_kvpairs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.double_starred_kvpair+ ',' invalid_kvpair")); - _res = _PyPegen_dummy_name(p, _gather_178_var, _literal, invalid_kvpair_var); + _res = _PyPegen_dummy_name(p, _gather_177_var, _literal, invalid_kvpair_var); goto done; } p->mark = _mark; @@ -21793,7 +21932,7 @@ invalid_double_starred_kvpairs_rule(Parser *p) && (a = _PyPegen_expect_token(p, 11)) // token=':' && - _PyPegen_lookahead(1, _tmp_180_rule, p) + _PyPegen_lookahead(1, _tmp_179_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_double_starred_kvpairs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ':' &('}' | ',')")); @@ -23235,12 +23374,12 @@ _loop1_22_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_22[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_181_var; + void *_tmp_180_var; while ( - (_tmp_181_var = _tmp_181_rule(p)) // star_targets '=' + (_tmp_180_var = _tmp_180_rule(p)) // star_targets '=' ) { - _res = _tmp_181_var; + _res = _tmp_180_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23770,12 +23909,12 @@ _loop0_31_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_31[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_182_var; + void *_tmp_181_var; while ( - (_tmp_182_var = _tmp_182_rule(p)) // '.' | '...' + (_tmp_181_var = _tmp_181_rule(p)) // '.' | '...' ) { - _res = _tmp_182_var; + _res = _tmp_181_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23839,12 +23978,12 @@ _loop1_32_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_32[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_183_var; + void *_tmp_182_var; while ( - (_tmp_183_var = _tmp_183_rule(p)) // '.' | '...' + (_tmp_182_var = _tmp_182_rule(p)) // '.' | '...' ) { - _res = _tmp_183_var; + _res = _tmp_182_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27111,12 +27250,12 @@ _loop1_84_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_84[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('@' named_expression NEWLINE)")); - void *_tmp_184_var; + void *_tmp_183_var; while ( - (_tmp_184_var = _tmp_184_rule(p)) // '@' named_expression NEWLINE + (_tmp_183_var = _tmp_183_rule(p)) // '@' named_expression NEWLINE ) { - _res = _tmp_184_var; + _res = _tmp_183_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27235,12 +27374,12 @@ _loop1_86_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_86[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_expression)")); - void *_tmp_185_var; + void *_tmp_184_var; while ( - (_tmp_185_var = _tmp_185_rule(p)) // ',' star_expression + (_tmp_184_var = _tmp_184_rule(p)) // ',' star_expression ) { - _res = _tmp_185_var; + _res = _tmp_184_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27429,12 +27568,12 @@ _loop1_89_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_89[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' expression)")); - void *_tmp_186_var; + void *_tmp_185_var; while ( - (_tmp_186_var = _tmp_186_rule(p)) // ',' expression + (_tmp_185_var = _tmp_185_rule(p)) // ',' expression ) { - _res = _tmp_186_var; + _res = _tmp_185_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -28504,12 +28643,12 @@ _loop1_104_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_104[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('or' conjunction)")); - void *_tmp_187_var; + void *_tmp_186_var; while ( - (_tmp_187_var = _tmp_187_rule(p)) // 'or' conjunction + (_tmp_186_var = _tmp_186_rule(p)) // 'or' conjunction ) { - _res = _tmp_187_var; + _res = _tmp_186_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -28578,12 +28717,12 @@ _loop1_105_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_105[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('and' inversion)")); - void *_tmp_188_var; + void *_tmp_187_var; while ( - (_tmp_188_var = _tmp_188_rule(p)) // 'and' inversion + (_tmp_187_var = _tmp_187_rule(p)) // 'and' inversion ) { - _res = _tmp_188_var; + _res = _tmp_187_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29604,12 +29743,12 @@ _loop0_121_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_121[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_189_var; + void *_tmp_188_var; while ( - (_tmp_189_var = _tmp_189_rule(p)) // 'if' disjunction + (_tmp_188_var = _tmp_188_rule(p)) // 'if' disjunction ) { - _res = _tmp_189_var; + _res = _tmp_188_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29673,12 +29812,12 @@ _loop0_122_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_122[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_190_var; + void *_tmp_189_var; while ( - (_tmp_190_var = _tmp_190_rule(p)) // 'if' disjunction + (_tmp_189_var = _tmp_189_rule(p)) // 'if' disjunction ) { - _res = _tmp_190_var; + _res = _tmp_189_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29747,7 +29886,7 @@ _loop0_124_rule(Parser *p) while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_191_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' + (elem = _tmp_190_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' ) { _res = elem; @@ -29814,7 +29953,7 @@ _gather_123_rule(Parser *p) void *elem; asdl_seq * seq; if ( - (elem = _tmp_191_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' + (elem = _tmp_190_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' && (seq = _loop0_124_rule(p)) // _loop0_124 ) @@ -30390,12 +30529,12 @@ _loop0_134_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_134[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); - void *_tmp_192_var; + void *_tmp_191_var; while ( - (_tmp_192_var = _tmp_192_rule(p)) // ',' star_target + (_tmp_191_var = _tmp_191_rule(p)) // ',' star_target ) { - _res = _tmp_192_var; + _res = _tmp_191_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -30579,12 +30718,12 @@ _loop1_137_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_137[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); - void *_tmp_193_var; + void *_tmp_192_var; while ( - (_tmp_193_var = _tmp_193_rule(p)) // ',' star_target + (_tmp_192_var = _tmp_192_rule(p)) // ',' star_target ) { - _res = _tmp_193_var; + _res = _tmp_192_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -31141,143 +31280,9 @@ _tmp_146_rule(Parser *p) return _res; } -// _tmp_147: list | tuple | genexp | 'True' | 'None' | 'False' +// _tmp_147: '=' | ':=' static void * _tmp_147_rule(Parser *p) -{ - if (p->level++ == MAXSTACK) { - p->error_indicator = 1; - PyErr_NoMemory(); - } - if (p->error_indicator) { - p->level--; - return NULL; - } - void * _res = NULL; - int _mark = p->mark; - { // list - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "list")); - expr_ty list_var; - if ( - (list_var = list_rule(p)) // list - ) - { - D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "list")); - _res = list_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "list")); - } - { // tuple - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "tuple")); - expr_ty tuple_var; - if ( - (tuple_var = tuple_rule(p)) // tuple - ) - { - D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "tuple")); - _res = tuple_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "tuple")); - } - { // genexp - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "genexp")); - expr_ty genexp_var; - if ( - (genexp_var = genexp_rule(p)) // genexp - ) - { - D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "genexp")); - _res = genexp_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "genexp")); - } - { // 'True' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 524)) // token='True' - ) - { - D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'True'")); - } - { // 'None' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 523)) // token='None' - ) - { - D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'None'")); - } - { // 'False' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 525)) // token='False' - ) - { - D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'False'")); - } - _res = NULL; - done: - p->level--; - return _res; -} - -// _tmp_148: '=' | ':=' -static void * -_tmp_148_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31294,18 +31299,18 @@ _tmp_148_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_148[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'='")); + D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'='")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_148[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'='")); + D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'='")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_148[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'='")); } { // ':=' @@ -31313,18 +31318,18 @@ _tmp_148_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_148[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':='")); + D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':='")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 53)) // token=':=' ) { - D(fprintf(stderr, "%*c+ _tmp_148[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':='")); + D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':='")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_148[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':='")); } _res = NULL; @@ -31333,9 +31338,9 @@ _tmp_148_rule(Parser *p) return _res; } -// _loop0_149: star_named_expressions +// _loop0_148: star_named_expressions static asdl_seq * -_loop0_149_rule(Parser *p) +_loop0_148_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31362,7 +31367,7 @@ _loop0_149_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); + D(fprintf(stderr, "%*c> _loop0_148[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); asdl_expr_seq* star_named_expressions_var; while ( (star_named_expressions_var = star_named_expressions_rule(p)) // star_named_expressions @@ -31384,7 +31389,7 @@ _loop0_149_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_149[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_148[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_named_expressions")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31397,14 +31402,14 @@ _loop0_149_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_149_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_148_type, _seq); p->level--; return _seq; } -// _loop0_150: (star_targets '=') +// _loop0_149: (star_targets '=') static asdl_seq * -_loop0_150_rule(Parser *p) +_loop0_149_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31431,13 +31436,13 @@ _loop0_150_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_194_var; + D(fprintf(stderr, "%*c> _loop0_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_193_var; while ( - (_tmp_194_var = _tmp_194_rule(p)) // star_targets '=' + (_tmp_193_var = _tmp_193_rule(p)) // star_targets '=' ) { - _res = _tmp_194_var; + _res = _tmp_193_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -31453,7 +31458,7 @@ _loop0_150_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_150[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_149[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31466,14 +31471,14 @@ _loop0_150_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_150_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_149_type, _seq); p->level--; return _seq; } -// _loop0_151: (star_targets '=') +// _loop0_150: (star_targets '=') static asdl_seq * -_loop0_151_rule(Parser *p) +_loop0_150_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31500,13 +31505,13 @@ _loop0_151_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_195_var; + D(fprintf(stderr, "%*c> _loop0_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_194_var; while ( - (_tmp_195_var = _tmp_195_rule(p)) // star_targets '=' + (_tmp_194_var = _tmp_194_rule(p)) // star_targets '=' ) { - _res = _tmp_195_var; + _res = _tmp_194_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -31522,7 +31527,7 @@ _loop0_151_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_151[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_150[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31535,14 +31540,14 @@ _loop0_151_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_151_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_150_type, _seq); p->level--; return _seq; } -// _tmp_152: yield_expr | star_expressions +// _tmp_151: yield_expr | star_expressions static void * -_tmp_152_rule(Parser *p) +_tmp_151_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31559,18 +31564,18 @@ _tmp_152_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c> _tmp_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); expr_ty yield_expr_var; if ( (yield_expr_var = yield_expr_rule(p)) // yield_expr ) { - D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c+ _tmp_151[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); _res = yield_expr_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_151[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "yield_expr")); } { // star_expressions @@ -31578,18 +31583,18 @@ _tmp_152_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c> _tmp_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); expr_ty star_expressions_var; if ( (star_expressions_var = star_expressions_rule(p)) // star_expressions ) { - D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c+ _tmp_151[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); _res = star_expressions_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_151[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_expressions")); } _res = NULL; @@ -31598,9 +31603,9 @@ _tmp_152_rule(Parser *p) return _res; } -// _tmp_153: '[' | '(' | '{' +// _tmp_152: '[' | '(' | '{' static void * -_tmp_153_rule(Parser *p) +_tmp_152_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31617,18 +31622,18 @@ _tmp_153_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '(' @@ -31636,18 +31641,18 @@ _tmp_153_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 7)) // token='(' ) { - D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'('")); } { // '{' @@ -31655,18 +31660,18 @@ _tmp_153_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -31675,9 +31680,9 @@ _tmp_153_rule(Parser *p) return _res; } -// _tmp_154: '[' | '{' +// _tmp_153: '[' | '{' static void * -_tmp_154_rule(Parser *p) +_tmp_153_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31694,18 +31699,18 @@ _tmp_154_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '{' @@ -31713,18 +31718,18 @@ _tmp_154_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -31733,9 +31738,9 @@ _tmp_154_rule(Parser *p) return _res; } -// _tmp_155: '[' | '{' +// _tmp_154: '[' | '{' static void * -_tmp_155_rule(Parser *p) +_tmp_154_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31752,18 +31757,18 @@ _tmp_155_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '{' @@ -31771,18 +31776,18 @@ _tmp_155_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -31791,9 +31796,9 @@ _tmp_155_rule(Parser *p) return _res; } -// _loop0_156: param_no_default +// _loop0_155: param_no_default static asdl_seq * -_loop0_156_rule(Parser *p) +_loop0_155_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31820,7 +31825,7 @@ _loop0_156_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); + D(fprintf(stderr, "%*c> _loop0_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); arg_ty param_no_default_var; while ( (param_no_default_var = param_no_default_rule(p)) // param_no_default @@ -31842,7 +31847,7 @@ _loop0_156_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_156[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31855,14 +31860,14 @@ _loop0_156_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_156_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_155_type, _seq); p->level--; return _seq; } -// _loop1_157: param_with_default +// _loop1_156: param_with_default static asdl_seq * -_loop1_157_rule(Parser *p) +_loop1_156_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31889,7 +31894,7 @@ _loop1_157_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); + D(fprintf(stderr, "%*c> _loop1_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); NameDefaultPair* param_with_default_var; while ( (param_with_default_var = param_with_default_rule(p)) // param_with_default @@ -31911,7 +31916,7 @@ _loop1_157_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_157[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_156[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -31929,14 +31934,14 @@ _loop1_157_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_157_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_156_type, _seq); p->level--; return _seq; } -// _loop0_158: lambda_param_no_default +// _loop0_157: lambda_param_no_default static asdl_seq * -_loop0_158_rule(Parser *p) +_loop0_157_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31963,7 +31968,7 @@ _loop0_158_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_158[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); + D(fprintf(stderr, "%*c> _loop0_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); arg_ty lambda_param_no_default_var; while ( (lambda_param_no_default_var = lambda_param_no_default_rule(p)) // lambda_param_no_default @@ -31985,7 +31990,7 @@ _loop0_158_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_158[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_157[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31998,14 +32003,14 @@ _loop0_158_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_158_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_157_type, _seq); p->level--; return _seq; } -// _loop1_159: lambda_param_with_default +// _loop1_158: lambda_param_with_default static asdl_seq * -_loop1_159_rule(Parser *p) +_loop1_158_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32032,7 +32037,7 @@ _loop1_159_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); + D(fprintf(stderr, "%*c> _loop1_158[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); NameDefaultPair* lambda_param_with_default_var; while ( (lambda_param_with_default_var = lambda_param_with_default_rule(p)) // lambda_param_with_default @@ -32054,7 +32059,7 @@ _loop1_159_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_159[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_158[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -32072,14 +32077,14 @@ _loop1_159_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_159_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_158_type, _seq); p->level--; return _seq; } -// _tmp_160: ')' | ',' (')' | '**') +// _tmp_159: ')' | ',' (')' | '**') static void * -_tmp_160_rule(Parser *p) +_tmp_159_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32096,18 +32101,18 @@ _tmp_160_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_160[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_159[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_160[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_159[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // ',' (')' | '**') @@ -32115,21 +32120,21 @@ _tmp_160_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + D(fprintf(stderr, "%*c> _tmp_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); Token * _literal; - void *_tmp_196_var; + void *_tmp_195_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_196_var = _tmp_196_rule(p)) // ')' | '**' + (_tmp_195_var = _tmp_195_rule(p)) // ')' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_160[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_196_var); + D(fprintf(stderr, "%*c+ _tmp_159[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_195_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_160[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_159[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (')' | '**')")); } _res = NULL; @@ -32138,9 +32143,9 @@ _tmp_160_rule(Parser *p) return _res; } -// _tmp_161: ':' | ',' (':' | '**') +// _tmp_160: ':' | ',' (':' | '**') static void * -_tmp_161_rule(Parser *p) +_tmp_160_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32157,18 +32162,18 @@ _tmp_161_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_160[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_160[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // ',' (':' | '**') @@ -32176,21 +32181,21 @@ _tmp_161_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + D(fprintf(stderr, "%*c> _tmp_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); Token * _literal; - void *_tmp_197_var; + void *_tmp_196_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_197_var = _tmp_197_rule(p)) // ':' | '**' + (_tmp_196_var = _tmp_196_rule(p)) // ':' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_197_var); + D(fprintf(stderr, "%*c+ _tmp_160[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_196_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_160[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (':' | '**')")); } _res = NULL; @@ -32199,9 +32204,9 @@ _tmp_161_rule(Parser *p) return _res; } -// _tmp_162: ',' | ')' | ':' +// _tmp_161: ',' | ')' | ':' static void * -_tmp_162_rule(Parser *p) +_tmp_161_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32218,18 +32223,18 @@ _tmp_162_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_162[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_162[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "','")); } { // ')' @@ -32237,18 +32242,18 @@ _tmp_162_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_162[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_162[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // ':' @@ -32256,18 +32261,18 @@ _tmp_162_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_162[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_162[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } _res = NULL; @@ -32276,9 +32281,9 @@ _tmp_162_rule(Parser *p) return _res; } -// _loop0_164: ',' (expression ['as' star_target]) +// _loop0_163: ',' (expression ['as' star_target]) static asdl_seq * -_loop0_164_rule(Parser *p) +_loop0_163_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32305,13 +32310,13 @@ _loop0_164_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_164[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_163[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_198_rule(p)) // expression ['as' star_target] + (elem = _tmp_197_rule(p)) // expression ['as' star_target] ) { _res = elem; @@ -32336,7 +32341,7 @@ _loop0_164_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_164[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_163[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expression ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32349,14 +32354,14 @@ _loop0_164_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_164_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_163_type, _seq); p->level--; return _seq; } -// _gather_163: (expression ['as' star_target]) _loop0_164 +// _gather_162: (expression ['as' star_target]) _loop0_163 static asdl_seq * -_gather_163_rule(Parser *p) +_gather_162_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32368,27 +32373,27 @@ _gather_163_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expression ['as' star_target]) _loop0_164 + { // (expression ['as' star_target]) _loop0_163 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_163[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_164")); + D(fprintf(stderr, "%*c> _gather_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_163")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_198_rule(p)) // expression ['as' star_target] + (elem = _tmp_197_rule(p)) // expression ['as' star_target] && - (seq = _loop0_164_rule(p)) // _loop0_164 + (seq = _loop0_163_rule(p)) // _loop0_163 ) { - D(fprintf(stderr, "%*c+ _gather_163[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_164")); + D(fprintf(stderr, "%*c+ _gather_162[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_163")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_163[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_164")); + D(fprintf(stderr, "%*c%s _gather_162[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_163")); } _res = NULL; done: @@ -32396,9 +32401,9 @@ _gather_163_rule(Parser *p) return _res; } -// _loop0_166: ',' (expressions ['as' star_target]) +// _loop0_165: ',' (expressions ['as' star_target]) static asdl_seq * -_loop0_166_rule(Parser *p) +_loop0_165_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32425,13 +32430,13 @@ _loop0_166_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_165[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_199_rule(p)) // expressions ['as' star_target] + (elem = _tmp_198_rule(p)) // expressions ['as' star_target] ) { _res = elem; @@ -32456,7 +32461,7 @@ _loop0_166_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_166[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_165[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expressions ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32469,14 +32474,14 @@ _loop0_166_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_166_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_165_type, _seq); p->level--; return _seq; } -// _gather_165: (expressions ['as' star_target]) _loop0_166 +// _gather_164: (expressions ['as' star_target]) _loop0_165 static asdl_seq * -_gather_165_rule(Parser *p) +_gather_164_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32488,27 +32493,27 @@ _gather_165_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expressions ['as' star_target]) _loop0_166 + { // (expressions ['as' star_target]) _loop0_165 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_165[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_166")); + D(fprintf(stderr, "%*c> _gather_164[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_165")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_199_rule(p)) // expressions ['as' star_target] + (elem = _tmp_198_rule(p)) // expressions ['as' star_target] && - (seq = _loop0_166_rule(p)) // _loop0_166 + (seq = _loop0_165_rule(p)) // _loop0_165 ) { - D(fprintf(stderr, "%*c+ _gather_165[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_166")); + D(fprintf(stderr, "%*c+ _gather_164[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_165")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_165[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_166")); + D(fprintf(stderr, "%*c%s _gather_164[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_165")); } _res = NULL; done: @@ -32516,9 +32521,9 @@ _gather_165_rule(Parser *p) return _res; } -// _loop0_168: ',' (expression ['as' star_target]) +// _loop0_167: ',' (expression ['as' star_target]) static asdl_seq * -_loop0_168_rule(Parser *p) +_loop0_167_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32545,13 +32550,13 @@ _loop0_168_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_200_rule(p)) // expression ['as' star_target] + (elem = _tmp_199_rule(p)) // expression ['as' star_target] ) { _res = elem; @@ -32576,7 +32581,7 @@ _loop0_168_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_168[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_167[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expression ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32589,14 +32594,14 @@ _loop0_168_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_168_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_167_type, _seq); p->level--; return _seq; } -// _gather_167: (expression ['as' star_target]) _loop0_168 +// _gather_166: (expression ['as' star_target]) _loop0_167 static asdl_seq * -_gather_167_rule(Parser *p) +_gather_166_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32608,27 +32613,27 @@ _gather_167_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expression ['as' star_target]) _loop0_168 + { // (expression ['as' star_target]) _loop0_167 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_168")); + D(fprintf(stderr, "%*c> _gather_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_167")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_200_rule(p)) // expression ['as' star_target] + (elem = _tmp_199_rule(p)) // expression ['as' star_target] && - (seq = _loop0_168_rule(p)) // _loop0_168 + (seq = _loop0_167_rule(p)) // _loop0_167 ) { - D(fprintf(stderr, "%*c+ _gather_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_168")); + D(fprintf(stderr, "%*c+ _gather_166[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_167")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_167[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_168")); + D(fprintf(stderr, "%*c%s _gather_166[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_167")); } _res = NULL; done: @@ -32636,9 +32641,9 @@ _gather_167_rule(Parser *p) return _res; } -// _loop0_170: ',' (expressions ['as' star_target]) +// _loop0_169: ',' (expressions ['as' star_target]) static asdl_seq * -_loop0_170_rule(Parser *p) +_loop0_169_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32665,13 +32670,13 @@ _loop0_170_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_201_rule(p)) // expressions ['as' star_target] + (elem = _tmp_200_rule(p)) // expressions ['as' star_target] ) { _res = elem; @@ -32696,7 +32701,7 @@ _loop0_170_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_170[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_169[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expressions ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32709,14 +32714,14 @@ _loop0_170_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_170_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_169_type, _seq); p->level--; return _seq; } -// _gather_169: (expressions ['as' star_target]) _loop0_170 +// _gather_168: (expressions ['as' star_target]) _loop0_169 static asdl_seq * -_gather_169_rule(Parser *p) +_gather_168_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32728,27 +32733,27 @@ _gather_169_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expressions ['as' star_target]) _loop0_170 + { // (expressions ['as' star_target]) _loop0_169 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_170")); + D(fprintf(stderr, "%*c> _gather_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_169")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_201_rule(p)) // expressions ['as' star_target] + (elem = _tmp_200_rule(p)) // expressions ['as' star_target] && - (seq = _loop0_170_rule(p)) // _loop0_170 + (seq = _loop0_169_rule(p)) // _loop0_169 ) { - D(fprintf(stderr, "%*c+ _gather_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_170")); + D(fprintf(stderr, "%*c+ _gather_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_169")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_169[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_170")); + D(fprintf(stderr, "%*c%s _gather_168[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_169")); } _res = NULL; done: @@ -32756,9 +32761,9 @@ _gather_169_rule(Parser *p) return _res; } -// _tmp_171: 'except' | 'finally' +// _tmp_170: 'except' | 'finally' static void * -_tmp_171_rule(Parser *p) +_tmp_170_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32775,18 +32780,18 @@ _tmp_171_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except'")); + D(fprintf(stderr, "%*c> _tmp_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except'")); Token * _keyword; if ( (_keyword = _PyPegen_expect_token(p, 521)) // token='except' ) { - D(fprintf(stderr, "%*c+ _tmp_171[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except'")); + D(fprintf(stderr, "%*c+ _tmp_170[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except'")); _res = _keyword; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_171[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_170[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'except'")); } { // 'finally' @@ -32794,18 +32799,18 @@ _tmp_171_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally'")); + D(fprintf(stderr, "%*c> _tmp_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally'")); Token * _keyword; if ( (_keyword = _PyPegen_expect_token(p, 522)) // token='finally' ) { - D(fprintf(stderr, "%*c+ _tmp_171[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'finally'")); + D(fprintf(stderr, "%*c+ _tmp_170[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'finally'")); _res = _keyword; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_171[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_170[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'finally'")); } _res = NULL; @@ -32814,9 +32819,9 @@ _tmp_171_rule(Parser *p) return _res; } -// _tmp_172: 'as' NAME +// _tmp_171: 'as' NAME static void * -_tmp_172_rule(Parser *p) +_tmp_171_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32833,7 +32838,7 @@ _tmp_172_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_172[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -32842,12 +32847,12 @@ _tmp_172_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_172[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_171[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_172[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_171[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -32856,9 +32861,9 @@ _tmp_172_rule(Parser *p) return _res; } -// _tmp_173: 'as' NAME +// _tmp_172: 'as' NAME static void * -_tmp_173_rule(Parser *p) +_tmp_172_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32875,7 +32880,7 @@ _tmp_173_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_173[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_172[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -32884,12 +32889,12 @@ _tmp_173_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_173[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_172[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_173[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_172[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -32898,9 +32903,9 @@ _tmp_173_rule(Parser *p) return _res; } -// _tmp_174: 'as' NAME +// _tmp_173: 'as' NAME static void * -_tmp_174_rule(Parser *p) +_tmp_173_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32917,7 +32922,7 @@ _tmp_174_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_174[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_173[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -32926,12 +32931,12 @@ _tmp_174_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_174[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_173[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_174[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_173[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -32940,9 +32945,9 @@ _tmp_174_rule(Parser *p) return _res; } -// _tmp_175: positional_patterns ',' +// _tmp_174: positional_patterns ',' static void * -_tmp_175_rule(Parser *p) +_tmp_174_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32959,7 +32964,7 @@ _tmp_175_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_175[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); + D(fprintf(stderr, "%*c> _tmp_174[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); Token * _literal; asdl_pattern_seq* positional_patterns_var; if ( @@ -32968,12 +32973,12 @@ _tmp_175_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_175[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); + D(fprintf(stderr, "%*c+ _tmp_174[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); _res = _PyPegen_dummy_name(p, positional_patterns_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_175[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_174[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "positional_patterns ','")); } _res = NULL; @@ -32982,9 +32987,9 @@ _tmp_175_rule(Parser *p) return _res; } -// _tmp_176: '->' expression +// _tmp_175: '->' expression static void * -_tmp_176_rule(Parser *p) +_tmp_175_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33001,7 +33006,7 @@ _tmp_176_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_176[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'->' expression")); + D(fprintf(stderr, "%*c> _tmp_175[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'->' expression")); Token * _literal; expr_ty expression_var; if ( @@ -33010,12 +33015,12 @@ _tmp_176_rule(Parser *p) (expression_var = expression_rule(p)) // expression ) { - D(fprintf(stderr, "%*c+ _tmp_176[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'->' expression")); + D(fprintf(stderr, "%*c+ _tmp_175[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'->' expression")); _res = _PyPegen_dummy_name(p, _literal, expression_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_176[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_175[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'->' expression")); } _res = NULL; @@ -33024,9 +33029,9 @@ _tmp_176_rule(Parser *p) return _res; } -// _tmp_177: '(' arguments? ')' +// _tmp_176: '(' arguments? ')' static void * -_tmp_177_rule(Parser *p) +_tmp_176_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33043,7 +33048,7 @@ _tmp_177_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); + D(fprintf(stderr, "%*c> _tmp_176[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); Token * _literal; Token * _literal_1; void *_opt_var; @@ -33056,12 +33061,12 @@ _tmp_177_rule(Parser *p) (_literal_1 = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_177[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); + D(fprintf(stderr, "%*c+ _tmp_176[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); _res = _PyPegen_dummy_name(p, _literal, _opt_var, _literal_1); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_177[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_176[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'(' arguments? ')'")); } _res = NULL; @@ -33070,9 +33075,9 @@ _tmp_177_rule(Parser *p) return _res; } -// _loop0_179: ',' double_starred_kvpair +// _loop0_178: ',' double_starred_kvpair static asdl_seq * -_loop0_179_rule(Parser *p) +_loop0_178_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33099,7 +33104,7 @@ _loop0_179_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' double_starred_kvpair")); + D(fprintf(stderr, "%*c> _loop0_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' double_starred_kvpair")); Token * _literal; KeyValuePair* elem; while ( @@ -33130,7 +33135,7 @@ _loop0_179_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_179[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_178[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' double_starred_kvpair")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33143,14 +33148,14 @@ _loop0_179_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_179_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_178_type, _seq); p->level--; return _seq; } -// _gather_178: double_starred_kvpair _loop0_179 +// _gather_177: double_starred_kvpair _loop0_178 static asdl_seq * -_gather_178_rule(Parser *p) +_gather_177_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33162,27 +33167,27 @@ _gather_178_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // double_starred_kvpair _loop0_179 + { // double_starred_kvpair _loop0_178 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_179")); + D(fprintf(stderr, "%*c> _gather_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_178")); KeyValuePair* elem; asdl_seq * seq; if ( (elem = double_starred_kvpair_rule(p)) // double_starred_kvpair && - (seq = _loop0_179_rule(p)) // _loop0_179 + (seq = _loop0_178_rule(p)) // _loop0_178 ) { - D(fprintf(stderr, "%*c+ _gather_178[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_179")); + D(fprintf(stderr, "%*c+ _gather_177[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_178")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_178[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "double_starred_kvpair _loop0_179")); + D(fprintf(stderr, "%*c%s _gather_177[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "double_starred_kvpair _loop0_178")); } _res = NULL; done: @@ -33190,9 +33195,9 @@ _gather_178_rule(Parser *p) return _res; } -// _tmp_180: '}' | ',' +// _tmp_179: '}' | ',' static void * -_tmp_180_rule(Parser *p) +_tmp_179_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33209,18 +33214,18 @@ _tmp_180_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'}'")); + D(fprintf(stderr, "%*c> _tmp_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'}'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 26)) // token='}' ) { - D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'}'")); + D(fprintf(stderr, "%*c+ _tmp_179[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'}'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_179[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'}'")); } { // ',' @@ -33228,18 +33233,18 @@ _tmp_180_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c> _tmp_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c+ _tmp_179[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_179[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "','")); } _res = NULL; @@ -33248,9 +33253,9 @@ _tmp_180_rule(Parser *p) return _res; } -// _tmp_181: star_targets '=' +// _tmp_180: star_targets '=' static void * -_tmp_181_rule(Parser *p) +_tmp_180_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33267,7 +33272,7 @@ _tmp_181_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty z; if ( @@ -33276,7 +33281,7 @@ _tmp_181_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_181[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33286,7 +33291,7 @@ _tmp_181_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_181[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -33295,9 +33300,9 @@ _tmp_181_rule(Parser *p) return _res; } -// _tmp_182: '.' | '...' +// _tmp_181: '.' | '...' static void * -_tmp_182_rule(Parser *p) +_tmp_181_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33314,18 +33319,18 @@ _tmp_182_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_182[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_182[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_181[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_182[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_181[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -33333,18 +33338,18 @@ _tmp_182_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_182[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_182[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_181[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_182[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_181[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -33353,9 +33358,9 @@ _tmp_182_rule(Parser *p) return _res; } -// _tmp_183: '.' | '...' +// _tmp_182: '.' | '...' static void * -_tmp_183_rule(Parser *p) +_tmp_182_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33372,18 +33377,18 @@ _tmp_183_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_182[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_182[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_182[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -33391,18 +33396,18 @@ _tmp_183_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_182[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_182[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_182[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -33411,9 +33416,9 @@ _tmp_183_rule(Parser *p) return _res; } -// _tmp_184: '@' named_expression NEWLINE +// _tmp_183: '@' named_expression NEWLINE static void * -_tmp_184_rule(Parser *p) +_tmp_183_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33430,7 +33435,7 @@ _tmp_184_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); Token * _literal; expr_ty f; Token * newline_var; @@ -33442,7 +33447,7 @@ _tmp_184_rule(Parser *p) (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); _res = f; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33452,7 +33457,7 @@ _tmp_184_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'@' named_expression NEWLINE")); } _res = NULL; @@ -33461,9 +33466,9 @@ _tmp_184_rule(Parser *p) return _res; } -// _tmp_185: ',' star_expression +// _tmp_184: ',' star_expression static void * -_tmp_185_rule(Parser *p) +_tmp_184_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33480,7 +33485,7 @@ _tmp_185_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_185[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); Token * _literal; expr_ty c; if ( @@ -33489,7 +33494,7 @@ _tmp_185_rule(Parser *p) (c = star_expression_rule(p)) // star_expression ) { - D(fprintf(stderr, "%*c+ _tmp_185[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33499,7 +33504,7 @@ _tmp_185_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_185[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_expression")); } _res = NULL; @@ -33508,9 +33513,9 @@ _tmp_185_rule(Parser *p) return _res; } -// _tmp_186: ',' expression +// _tmp_185: ',' expression static void * -_tmp_186_rule(Parser *p) +_tmp_185_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33527,7 +33532,7 @@ _tmp_186_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_186[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c> _tmp_185[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); Token * _literal; expr_ty c; if ( @@ -33536,7 +33541,7 @@ _tmp_186_rule(Parser *p) (c = expression_rule(p)) // expression ) { - D(fprintf(stderr, "%*c+ _tmp_186[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c+ _tmp_185[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33546,7 +33551,7 @@ _tmp_186_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_186[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_185[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' expression")); } _res = NULL; @@ -33555,9 +33560,9 @@ _tmp_186_rule(Parser *p) return _res; } -// _tmp_187: 'or' conjunction +// _tmp_186: 'or' conjunction static void * -_tmp_187_rule(Parser *p) +_tmp_186_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33574,7 +33579,7 @@ _tmp_187_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_187[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c> _tmp_186[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); Token * _keyword; expr_ty c; if ( @@ -33583,7 +33588,7 @@ _tmp_187_rule(Parser *p) (c = conjunction_rule(p)) // conjunction ) { - D(fprintf(stderr, "%*c+ _tmp_187[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c+ _tmp_186[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33593,7 +33598,7 @@ _tmp_187_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_187[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_186[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'or' conjunction")); } _res = NULL; @@ -33602,9 +33607,9 @@ _tmp_187_rule(Parser *p) return _res; } -// _tmp_188: 'and' inversion +// _tmp_187: 'and' inversion static void * -_tmp_188_rule(Parser *p) +_tmp_187_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33621,7 +33626,7 @@ _tmp_188_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_188[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c> _tmp_187[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); Token * _keyword; expr_ty c; if ( @@ -33630,7 +33635,7 @@ _tmp_188_rule(Parser *p) (c = inversion_rule(p)) // inversion ) { - D(fprintf(stderr, "%*c+ _tmp_188[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c+ _tmp_187[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33640,7 +33645,7 @@ _tmp_188_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_188[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_187[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'and' inversion")); } _res = NULL; @@ -33649,9 +33654,9 @@ _tmp_188_rule(Parser *p) return _res; } -// _tmp_189: 'if' disjunction +// _tmp_188: 'if' disjunction static void * -_tmp_189_rule(Parser *p) +_tmp_188_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33668,7 +33673,7 @@ _tmp_189_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_189[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_188[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -33677,7 +33682,7 @@ _tmp_189_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_189[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_188[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33687,7 +33692,7 @@ _tmp_189_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_189[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_188[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -33696,9 +33701,9 @@ _tmp_189_rule(Parser *p) return _res; } -// _tmp_190: 'if' disjunction +// _tmp_189: 'if' disjunction static void * -_tmp_190_rule(Parser *p) +_tmp_189_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33715,7 +33720,7 @@ _tmp_190_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_189[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -33724,7 +33729,7 @@ _tmp_190_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_190[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_189[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33734,7 +33739,7 @@ _tmp_190_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_190[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_189[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -33743,9 +33748,9 @@ _tmp_190_rule(Parser *p) return _res; } -// _tmp_191: starred_expression | (assignment_expression | expression !':=') !'=' +// _tmp_190: starred_expression | (assignment_expression | expression !':=') !'=' static void * -_tmp_191_rule(Parser *p) +_tmp_190_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33762,18 +33767,18 @@ _tmp_191_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression")); + D(fprintf(stderr, "%*c> _tmp_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression")); expr_ty starred_expression_var; if ( (starred_expression_var = starred_expression_rule(p)) // starred_expression ) { - D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression")); + D(fprintf(stderr, "%*c+ _tmp_190[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression")); _res = starred_expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_190[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "starred_expression")); } { // (assignment_expression | expression !':=') !'=' @@ -33781,20 +33786,20 @@ _tmp_191_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); - void *_tmp_202_var; + D(fprintf(stderr, "%*c> _tmp_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); + void *_tmp_201_var; if ( - (_tmp_202_var = _tmp_202_rule(p)) // assignment_expression | expression !':=' + (_tmp_201_var = _tmp_201_rule(p)) // assignment_expression | expression !':=' && _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 22) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); - _res = _tmp_202_var; + D(fprintf(stderr, "%*c+ _tmp_190[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); + _res = _tmp_201_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_190[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(assignment_expression | expression !':=') !'='")); } _res = NULL; @@ -33803,9 +33808,9 @@ _tmp_191_rule(Parser *p) return _res; } -// _tmp_192: ',' star_target +// _tmp_191: ',' star_target static void * -_tmp_192_rule(Parser *p) +_tmp_191_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33822,7 +33827,7 @@ _tmp_192_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty c; if ( @@ -33831,7 +33836,7 @@ _tmp_192_rule(Parser *p) (c = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33841,7 +33846,7 @@ _tmp_192_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } _res = NULL; @@ -33850,9 +33855,9 @@ _tmp_192_rule(Parser *p) return _res; } -// _tmp_193: ',' star_target +// _tmp_192: ',' star_target static void * -_tmp_193_rule(Parser *p) +_tmp_192_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33869,7 +33874,7 @@ _tmp_193_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty c; if ( @@ -33878,7 +33883,7 @@ _tmp_193_rule(Parser *p) (c = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33888,7 +33893,7 @@ _tmp_193_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } _res = NULL; @@ -33897,9 +33902,9 @@ _tmp_193_rule(Parser *p) return _res; } -// _tmp_194: star_targets '=' +// _tmp_193: star_targets '=' static void * -_tmp_194_rule(Parser *p) +_tmp_193_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33916,7 +33921,7 @@ _tmp_194_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -33925,12 +33930,12 @@ _tmp_194_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -33939,9 +33944,9 @@ _tmp_194_rule(Parser *p) return _res; } -// _tmp_195: star_targets '=' +// _tmp_194: star_targets '=' static void * -_tmp_195_rule(Parser *p) +_tmp_194_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33958,7 +33963,7 @@ _tmp_195_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -33967,12 +33972,12 @@ _tmp_195_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -33981,9 +33986,9 @@ _tmp_195_rule(Parser *p) return _res; } -// _tmp_196: ')' | '**' +// _tmp_195: ')' | '**' static void * -_tmp_196_rule(Parser *p) +_tmp_195_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34000,18 +34005,18 @@ _tmp_196_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // '**' @@ -34019,18 +34024,18 @@ _tmp_196_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; @@ -34039,9 +34044,9 @@ _tmp_196_rule(Parser *p) return _res; } -// _tmp_197: ':' | '**' +// _tmp_196: ':' | '**' static void * -_tmp_197_rule(Parser *p) +_tmp_196_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34058,18 +34063,18 @@ _tmp_197_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // '**' @@ -34077,18 +34082,18 @@ _tmp_197_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; @@ -34097,9 +34102,9 @@ _tmp_197_rule(Parser *p) return _res; } -// _tmp_198: expression ['as' star_target] +// _tmp_197: expression ['as' star_target] static void * -_tmp_198_rule(Parser *p) +_tmp_197_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34116,22 +34121,22 @@ _tmp_198_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_198[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_203_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_202_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_198[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); _res = _PyPegen_dummy_name(p, expression_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_198[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression ['as' star_target]")); } _res = NULL; @@ -34140,9 +34145,9 @@ _tmp_198_rule(Parser *p) return _res; } -// _tmp_199: expressions ['as' star_target] +// _tmp_198: expressions ['as' star_target] static void * -_tmp_199_rule(Parser *p) +_tmp_198_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34159,22 +34164,22 @@ _tmp_199_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_199[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_198[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expressions_var; if ( (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_204_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_203_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_199[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_198[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); _res = _PyPegen_dummy_name(p, expressions_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_199[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_198[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expressions ['as' star_target]")); } _res = NULL; @@ -34183,9 +34188,9 @@ _tmp_199_rule(Parser *p) return _res; } -// _tmp_200: expression ['as' star_target] +// _tmp_199: expression ['as' star_target] static void * -_tmp_200_rule(Parser *p) +_tmp_199_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34202,22 +34207,22 @@ _tmp_200_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_200[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_199[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_205_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_204_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_200[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_199[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); _res = _PyPegen_dummy_name(p, expression_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_200[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_199[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression ['as' star_target]")); } _res = NULL; @@ -34226,9 +34231,9 @@ _tmp_200_rule(Parser *p) return _res; } -// _tmp_201: expressions ['as' star_target] +// _tmp_200: expressions ['as' star_target] static void * -_tmp_201_rule(Parser *p) +_tmp_200_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34245,22 +34250,22 @@ _tmp_201_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_200[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expressions_var; if ( (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_206_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_205_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_200[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); _res = _PyPegen_dummy_name(p, expressions_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_200[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expressions ['as' star_target]")); } _res = NULL; @@ -34269,9 +34274,9 @@ _tmp_201_rule(Parser *p) return _res; } -// _tmp_202: assignment_expression | expression !':=' +// _tmp_201: assignment_expression | expression !':=' static void * -_tmp_202_rule(Parser *p) +_tmp_201_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34288,18 +34293,18 @@ _tmp_202_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "assignment_expression")); + D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "assignment_expression")); expr_ty assignment_expression_var; if ( (assignment_expression_var = assignment_expression_rule(p)) // assignment_expression ) { - D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "assignment_expression")); + D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "assignment_expression")); _res = assignment_expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "assignment_expression")); } { // expression !':=' @@ -34307,7 +34312,7 @@ _tmp_202_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression !':='")); + D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression !':='")); expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression @@ -34315,12 +34320,12 @@ _tmp_202_rule(Parser *p) _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 53) // token=':=' ) { - D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression !':='")); + D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression !':='")); _res = expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression !':='")); } _res = NULL; @@ -34329,9 +34334,9 @@ _tmp_202_rule(Parser *p) return _res; } -// _tmp_203: 'as' star_target +// _tmp_202: 'as' star_target static void * -_tmp_203_rule(Parser *p) +_tmp_202_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34348,7 +34353,7 @@ _tmp_203_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -34357,12 +34362,12 @@ _tmp_203_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -34371,9 +34376,9 @@ _tmp_203_rule(Parser *p) return _res; } -// _tmp_204: 'as' star_target +// _tmp_203: 'as' star_target static void * -_tmp_204_rule(Parser *p) +_tmp_203_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34390,7 +34395,7 @@ _tmp_204_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_204[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -34399,12 +34404,12 @@ _tmp_204_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_204[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_204[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -34413,9 +34418,9 @@ _tmp_204_rule(Parser *p) return _res; } -// _tmp_205: 'as' star_target +// _tmp_204: 'as' star_target static void * -_tmp_205_rule(Parser *p) +_tmp_204_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34432,7 +34437,7 @@ _tmp_205_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_205[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_204[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -34441,12 +34446,12 @@ _tmp_205_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_205[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_204[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_205[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_204[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -34455,9 +34460,9 @@ _tmp_205_rule(Parser *p) return _res; } -// _tmp_206: 'as' star_target +// _tmp_205: 'as' star_target static void * -_tmp_206_rule(Parser *p) +_tmp_205_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34474,7 +34479,7 @@ _tmp_206_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_206[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_205[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -34483,12 +34488,12 @@ _tmp_206_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_206[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_205[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_206[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_205[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; From webhook-mailer at python.org Thu Feb 10 02:59:13 2022 From: webhook-mailer at python.org (JulienPalard) Date: Thu, 10 Feb 2022 07:59:13 -0000 Subject: [Python-checkins] bpo-42238: [doc] moving from rstlint.py to sphinx-lint. (GH-31097) Message-ID: https://github.com/python/cpython/commit/b878b3af0b3a9e3ab3ffcaf90a4066dfb3bb7cac commit: b878b3af0b3a9e3ab3ffcaf90a4066dfb3bb7cac branch: main author: Julien Palard committer: JulienPalard date: 2022-02-10T08:59:04+01:00 summary: bpo-42238: [doc] moving from rstlint.py to sphinx-lint. (GH-31097) files: A Misc/NEWS.d/next/Documentation/2022-02-03-11-24-59.bpo-42238.yJcMa8.rst M Doc/Makefile M Doc/make.bat M Doc/requirements.txt M Doc/tools/rstlint.py M Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst diff --git a/Doc/Makefile b/Doc/Makefile index e60aa3427fea2..61a7ce0d0981f 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -7,6 +7,7 @@ PYTHON = python3 VENVDIR = ./venv SPHINXBUILD = PATH=$(VENVDIR)/bin:$$PATH sphinx-build +SPHINXLINT = PATH=$(VENVDIR)/bin:$$PATH sphinx-lint BLURB = PATH=$(VENVDIR)/bin:$$PATH blurb PAPER = SOURCES = @@ -214,8 +215,8 @@ dist: rm dist/python-$(DISTVERSION)-docs-texinfo.tar check: - $(PYTHON) tools/rstlint.py -i tools -i $(VENVDIR) -i README.rst - $(PYTHON) tools/rstlint.py ../Misc/NEWS.d/next/ + $(SPHINXLINT) -i tools -i $(VENVDIR) -i README.rst + $(SPHINXLINT) ../Misc/NEWS.d/next/ serve: $(PYTHON) ../Tools/scripts/serve.py build/html $(SERVE_PORT) diff --git a/Doc/make.bat b/Doc/make.bat index 7fde063642771..9eaaa46806829 100644 --- a/Doc/make.bat +++ b/Doc/make.bat @@ -36,6 +36,16 @@ if not defined BLURB ( set BLURB=%PYTHON% -m blurb ) +if not defined SPHINXLINT ( + %PYTHON% -c "import sphinxlint" > nul 2> nul + if errorlevel 1 ( + echo Installing sphinx-lint with %PYTHON% + %PYTHON% -m pip install sphinx-lint + if errorlevel 1 exit /B + ) + set SPHINXLINT=%PYTHON% -m sphinxlint +) + if "%1" NEQ "htmlhelp" goto :skiphhcsearch if exist "%HTMLHELP%" goto :skiphhcsearch @@ -168,7 +178,7 @@ if EXIST "%BUILDDIR%\html\index.html" ( goto end :check -cmd /S /C "%PYTHON% tools\rstlint.py -i tools" +cmd /S /C "%SPHINXLINT% -i tools" goto end :serve diff --git a/Doc/requirements.txt b/Doc/requirements.txt index 0331a8dbebc46..3b28495d4b4d0 100644 --- a/Doc/requirements.txt +++ b/Doc/requirements.txt @@ -7,6 +7,8 @@ sphinx==4.2.0 blurb +sphinx-lint<1 + # The theme used by the documentation is stored separately, so we need # to install that as well. python-docs-theme>=2022.1 diff --git a/Doc/tools/rstlint.py b/Doc/tools/rstlint.py old mode 100755 new mode 100644 index 33cbaadfce944..d1c53dcb1a698 --- a/Doc/tools/rstlint.py +++ b/Doc/tools/rstlint.py @@ -345,6 +345,11 @@ def main(argv): count = defaultdict(int) + print("""? rstlint.py is no longer maintained here and will be removed +? in a future release. +? Please use https://pypi.org/p/sphinx-lint instead. +""") + for root, dirs, files in os.walk(path): # ignore subdirs in ignore list if abspath(root) in ignore: diff --git a/Misc/NEWS.d/next/Documentation/2022-02-03-11-24-59.bpo-42238.yJcMa8.rst b/Misc/NEWS.d/next/Documentation/2022-02-03-11-24-59.bpo-42238.yJcMa8.rst new file mode 100644 index 0000000000000..a8dffff3fcf28 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2022-02-03-11-24-59.bpo-42238.yJcMa8.rst @@ -0,0 +1,2 @@ +``Doc/tools/rstlint.py`` has moved to its own repository and is now packaged +on PyPI as ``sphinx-lint``. diff --git a/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst b/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst index 4f0de9519a00e..183e064b8308e 100644 --- a/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst +++ b/Misc/NEWS.d/next/Library/2022-02-06-08-54-03.bpo-46655.DiLzYv.rst @@ -1 +1 @@ -In :func:`typing.get_type_hints`, support evaluating bare stringified ``TypeAlias`` annotations. Patch by Gregory Beauregard. \ No newline at end of file +In :func:`typing.get_type_hints`, support evaluating bare stringified ``TypeAlias`` annotations. Patch by Gregory Beauregard. From webhook-mailer at python.org Thu Feb 10 04:09:00 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 10 Feb 2022 09:09:00 -0000 Subject: [Python-checkins] Fix sphinx-lint after #31097 and b878b3a (GH-31248) Message-ID: https://github.com/python/cpython/commit/dee020a6f5bf29f95bec6294da9bcd577114f592 commit: dee020a6f5bf29f95bec6294da9bcd577114f592 branch: main author: Nikita Sobolev committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-10T01:08:47-08:00 summary: Fix sphinx-lint after #31097 and b878b3a (GH-31248) `main` branch is failing, see https://dev.azure.com/python/cpython/_build/results?buildId=96616&view=logs&j=4db1505a-29e5-5cc0-240b-53a8a2681f75&t=a975920c-8356-5388-147c-613d5fab0171 Logs: ``` PATH=./venv/bin:$PATH sphinx-lint -i tools -i ./venv -i README.rst No problems found. PATH=./venv/bin:$PATH sphinx-lint ../Misc/NEWS.d/next/ [1] ../Misc/NEWS.d/next/Library/2022-02-09-00-53-23.[bpo-45863]().zqQXVv.rst:0: No newline at end of file (no-newline-at-end-of-file). [1] ../Misc/NEWS.d/next/Build/2022-01-19-11-08-32.[bpo-46430]().k403m_.rst:0: No newline at end of file (no-newline-at-end-of-file). 2 problems with severity 1 found. ``` This PR fixes these two problems, so `main` is green again. Related PR: https://github.com/python/cpython/pull/31097 CC @JulienPalard Automerge-Triggered-By: GH:JulienPalard files: M Misc/NEWS.d/next/Build/2022-01-19-11-08-32.bpo-46430.k403m_.rst M Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst diff --git a/Misc/NEWS.d/next/Build/2022-01-19-11-08-32.bpo-46430.k403m_.rst b/Misc/NEWS.d/next/Build/2022-01-19-11-08-32.bpo-46430.k403m_.rst index 2929c5187e1d6..21be6fc7a4691 100644 --- a/Misc/NEWS.d/next/Build/2022-01-19-11-08-32.bpo-46430.k403m_.rst +++ b/Misc/NEWS.d/next/Build/2022-01-19-11-08-32.bpo-46430.k403m_.rst @@ -1 +1 @@ -Intern strings in deep-frozen modules. Patch by Kumar Aditya. \ No newline at end of file +Intern strings in deep-frozen modules. Patch by Kumar Aditya. diff --git a/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst b/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst index 3a1335cc77b9d..6a52dacb474b7 100644 --- a/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst +++ b/Misc/NEWS.d/next/Library/2022-02-09-00-53-23.bpo-45863.zqQXVv.rst @@ -1 +1 @@ -When the :mod:`tarfile` module creates a pax format archive, it will put an integer representation of timestamps in the ustar header (if possible) for the benefit of older unarchivers, in addition to the existing full-precision timestamps in the pax extended header. \ No newline at end of file +When the :mod:`tarfile` module creates a pax format archive, it will put an integer representation of timestamps in the ustar header (if possible) for the benefit of older unarchivers, in addition to the existing full-precision timestamps in the pax extended header. From webhook-mailer at python.org Thu Feb 10 05:10:12 2022 From: webhook-mailer at python.org (corona10) Date: Thu, 10 Feb 2022 10:10:12 -0000 Subject: [Python-checkins] bpo-46323: Allow alloca(0) for python callback function of ctypes (GH-31249) Message-ID: https://github.com/python/cpython/commit/db052851a70fd95d047c6263fc16a75e4d47b3ed commit: db052851a70fd95d047c6263fc16a75e4d47b3ed branch: main author: Dong-hee Na committer: corona10 date: 2022-02-10T19:10:07+09:00 summary: bpo-46323: Allow alloca(0) for python callback function of ctypes (GH-31249) files: M Modules/_ctypes/callbacks.c diff --git a/Modules/_ctypes/callbacks.c b/Modules/_ctypes/callbacks.c index 591b944d76fcb..6037f5fd84286 100644 --- a/Modules/_ctypes/callbacks.c +++ b/Modules/_ctypes/callbacks.c @@ -165,11 +165,7 @@ static void _CallPythonObject(void *mem, assert(PyTuple_Check(converters)); nargs = PyTuple_GET_SIZE(converters); assert(nargs <= CTYPES_MAX_ARGCOUNT); - PyObject **args = NULL; - if (nargs > 0) { - args = alloca(nargs * sizeof(PyObject *)); - } - + PyObject **args = alloca(nargs * sizeof(PyObject *)); PyObject **cnvs = PySequence_Fast_ITEMS(converters); for (i = 0; i < nargs; i++) { PyObject *cnv = cnvs[i]; // borrowed ref From webhook-mailer at python.org Thu Feb 10 06:01:26 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 10 Feb 2022 11:01:26 -0000 Subject: [Python-checkins] Right justify numeric columns in stats summary script. (GH-31234) Message-ID: https://github.com/python/cpython/commit/cfc1cecd7b6f0f27163b5bf519530284140cf5b2 commit: cfc1cecd7b6f0f27163b5bf519530284140cf5b2 branch: main author: Mark Shannon committer: markshannon date: 2022-02-10T11:01:18Z summary: Right justify numeric columns in stats summary script. (GH-31234) files: M Tools/scripts/summarize_stats.py diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index da0bab2390f3d..6e0286f52a0ea 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -55,7 +55,7 @@ def print_specialization_stats(name, family_stats, defines): label = label[0].upper() + label[1:] val = family_stats.get(key, 0) rows.append((label, val, f"{100*val/total_attempts:0.1f}%")) - emit_table(("", "Count", "Ratio"), rows) + emit_table(("", "Count:", "Ratio:"), rows) total_failures = family_stats.get("specialization.failure", 0) failure_kinds = [ 0 ] * 30 for key in family_stats: @@ -71,7 +71,7 @@ def print_specialization_stats(name, family_stats, defines): if not value: continue rows.append((kind_to_text(index, defines, name), value, f"{100*value/total_failures:0.1f}%")) - emit_table(("Failure kind", "Count", "Ratio"), rows) + emit_table(("Failure kind", "Count:", "Ratio:"), rows) def gather_stats(): stats = collections.Counter() @@ -174,8 +174,17 @@ def __exit__(*args): def emit_table(header, rows): width = len(header) - print("|", " | ".join(header), "|") - print("|", " | ".join(["---"]*width), "|") + header_line = "|" + under_line = "|" + for item in header: + under = "---" + if item.endswith(":"): + item = item[:-1] + under += ":" + header_line += item + " | " + under_line += under + "|" + print(header_line) + print(under_line) for row in rows: if width is not None and len(row) != width: raise ValueError("Wrong number of elements in row '" + str(rows) + "'") @@ -204,7 +213,7 @@ def emit_execution_counts(opcode_stats, total): rows.append((name, count, f"{100*count/total:0.1f}%", f"{100*cumulative/total:0.1f}%", miss)) emit_table( - ("Name", "Count", "Self", "Cumulative", "Miss ratio"), + ("Name", "Count:", "Self:", "Cumulative:", "Miss ratio:"), rows ) @@ -221,7 +230,7 @@ def emit_specialization_stats(opcode_stats): def emit_specialization_overview(opcode_stats, total): basic, not_specialized, specialized = categorized_counts(opcode_stats) with Section("Specialization effectiveness"): - emit_table(("Instructions", "Count", "Ratio"), ( + emit_table(("Instructions", "Count:", "Ratio:"), ( ("Basic", basic, f"{basic*100/total:0.1f}%"), ("Not specialized", not_specialized, f"{not_specialized*100/total:0.1f}%"), ("Specialized", specialized, f"{specialized*100/total:0.1f}%"), @@ -240,7 +249,7 @@ def emit_call_stats(stats): for key, value in stats.items(): if key.startswith("Frame"): rows.append((key, value, f"{100*value/total:0.1f}%")) - emit_table(("", "Count", "Ratio"), rows) + emit_table(("", "Count:", "Ratio:"), rows) def emit_object_stats(stats): with Section("Object stats", summary="allocations, frees and dict materializatons"): @@ -255,7 +264,7 @@ def emit_object_stats(stats): label = key[6:].strip() label = label[0].upper() + label[1:] rows.append((label, value, materialize)) - emit_table(("", "Count", "Ratio"), rows) + emit_table(("", "Count:", "Ratio:"), rows) def main(): stats = gather_stats() From webhook-mailer at python.org Thu Feb 10 06:48:17 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 10 Feb 2022 11:48:17 -0000 Subject: [Python-checkins] Add stats for PRECALL_FUNCTION. (GH-31250) Message-ID: https://github.com/python/cpython/commit/b0662ae5c83d8678506989cccbf7ba7bf61fea9d commit: b0662ae5c83d8678506989cccbf7ba7bf61fea9d branch: main author: Mark Shannon committer: markshannon date: 2022-02-10T11:47:52Z summary: Add stats for PRECALL_FUNCTION. (GH-31250) files: M Python/ceval.c M Python/specialize.c M Tools/scripts/summarize_stats.py diff --git a/Python/ceval.c b/Python/ceval.c index 02e4e7b9e4d54..958ca11409c32 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4447,6 +4447,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr call_shape.total_args = oparg; call_shape.kwnames = NULL; +#ifdef Py_STATS + extern int _PySpecialization_ClassifyCallable(PyObject *); + _py_stats.opcode_stats[PRECALL_FUNCTION].specialization.failure++; + _py_stats.opcode_stats[PRECALL_FUNCTION].specialization.failure_kinds[_PySpecialization_ClassifyCallable(call_shape.callable)]++; +#endif DISPATCH(); } diff --git a/Python/specialize.c b/Python/specialize.c index b5e4de5d6d217..940ab172d55fe 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -175,6 +175,7 @@ print_spec_stats(FILE *out, OpcodeStats *stats) /* Mark some opcodes as specializable for stats, * even though we don't specialize them yet. */ fprintf(out, " opcode[%d].specializable : 1\n", FOR_ITER); + fprintf(out, " opcode[%d].specializable : 1\n", PRECALL_FUNCTION); fprintf(out, " opcode[%d].specializable : 1\n", UNPACK_SEQUENCE); for (int i = 0; i < 256; i++) { if (adaptive_opcodes[i]) { @@ -556,7 +557,7 @@ initial_counter_value(void) { #define SPEC_FAIL_CALL_BAD_CALL_FLAGS 17 #define SPEC_FAIL_CALL_CLASS 18 #define SPEC_FAIL_CALL_PYTHON_CLASS 19 -#define SPEC_FAIL_CALL_C_METHOD_CALL 20 +#define SPEC_FAIL_CALL_METHOD_DESCRIPTOR 20 #define SPEC_FAIL_CALL_BOUND_METHOD 21 #define SPEC_FAIL_CALL_STR 22 #define SPEC_FAIL_CALL_CLASS_NO_VECTORCALL 23 @@ -564,6 +565,7 @@ initial_counter_value(void) { #define SPEC_FAIL_CALL_KWNAMES 25 #define SPEC_FAIL_CALL_METHOD_WRAPPER 26 #define SPEC_FAIL_CALL_OPERATOR_WRAPPER 27 +#define SPEC_FAIL_CALL_PYFUNCTION 28 /* COMPARE_OP */ #define SPEC_FAIL_COMPARE_OP_DIFFERENT_TYPES 12 @@ -1651,7 +1653,13 @@ specialize_c_call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, static int call_fail_kind(PyObject *callable) { - if (PyInstanceMethod_Check(callable)) { + if (PyCFunction_CheckExact(callable)) { + return SPEC_FAIL_CALL_PYCFUNCTION; + } + else if (PyFunction_Check(callable)) { + return SPEC_FAIL_CALL_PYFUNCTION; + } + else if (PyInstanceMethod_Check(callable)) { return SPEC_FAIL_CALL_INSTANCE_METHOD; } else if (PyMethod_Check(callable)) { @@ -1662,7 +1670,15 @@ call_fail_kind(PyObject *callable) return SPEC_FAIL_CALL_CMETHOD; } else if (PyType_Check(callable)) { - return SPEC_FAIL_CALL_CLASS; + if (((PyTypeObject *)callable)->tp_new == PyBaseObject_Type.tp_new) { + return SPEC_FAIL_CALL_PYTHON_CLASS; + } + else { + return SPEC_FAIL_CALL_CLASS; + } + } + else if (Py_IS_TYPE(callable, &PyMethodDescr_Type)) { + return SPEC_FAIL_CALL_METHOD_DESCRIPTOR; } else if (Py_TYPE(callable) == &PyWrapperDescr_Type) { return SPEC_FAIL_CALL_OPERATOR_WRAPPER; @@ -1905,6 +1921,8 @@ _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, adaptive->counter = initial_counter_value(); } +#ifdef Py_STATS + int _PySpecialization_ClassifyIterator(PyObject *iter) { @@ -1966,3 +1984,11 @@ _PySpecialization_ClassifySequence(PyObject *seq) } return SPEC_FAIL_OTHER; } + +int +_PySpecialization_ClassifyCallable(PyObject *callable) +{ + return call_fail_kind(callable); +} + +#endif diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index 6e0286f52a0ea..f6e5b333b09fc 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -114,6 +114,8 @@ def kind_to_text(kind, defines, opname): opname = "ATTR" if opname.endswith("SUBSCR"): opname = "SUBSCR" + if opname.startswith("PRECALL"): + opname = "CALL" for name in defines[kind]: if name.startswith(opname): return pretty(name[len(opname)+1:]) From webhook-mailer at python.org Thu Feb 10 07:15:14 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 10 Feb 2022 12:15:14 -0000 Subject: [Python-checkins] Fix sys.getdxp() when configured with --enable-pystats. (GH-31251) Message-ID: https://github.com/python/cpython/commit/4f21d528f0fc29c861c5461c9fa60bc532b0d300 commit: 4f21d528f0fc29c861c5461c9fa60bc532b0d300 branch: main author: Mark Shannon committer: markshannon date: 2022-02-10T12:14:57Z summary: Fix sys.getdxp() when configured with --enable-pystats. (GH-31251) files: M Python/ceval.c diff --git a/Python/ceval.c b/Python/ceval.c index 958ca11409c32..7e19043c7aa64 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -7459,7 +7459,7 @@ _Py_GetDXProfile(PyObject *self, PyObject *args) int i; PyObject *l = PyList_New(257); if (l == NULL) return NULL; - for (i = 0; i < 257; i++) { + for (i = 0; i < 256; i++) { PyObject *x = getarray(_py_stats.opcode_stats[i].pair_count); if (x == NULL) { Py_DECREF(l); @@ -7467,6 +7467,22 @@ _Py_GetDXProfile(PyObject *self, PyObject *args) } PyList_SET_ITEM(l, i, x); } + PyObject *counts = PyList_New(256); + if (counts == NULL) { + Py_DECREF(l); + return NULL; + } + for (i = 0; i < 256; i++) { + PyObject *x = PyLong_FromUnsignedLongLong( + _py_stats.opcode_stats[i].execution_count); + if (x == NULL) { + Py_DECREF(counts); + Py_DECREF(l); + return NULL; + } + PyList_SET_ITEM(counts, i, x); + } + PyList_SET_ITEM(l, 256, counts); return l; } From webhook-mailer at python.org Thu Feb 10 07:57:26 2022 From: webhook-mailer at python.org (asvetlov) Date: Thu, 10 Feb 2022 12:57:26 -0000 Subject: [Python-checkins] Fix warning: asyncio.events._event_loop_policy was modified by test_asyncio (GH-31253) Message-ID: https://github.com/python/cpython/commit/012e77eb5c3ba3d411f5967a7f368ebdb42ab88c commit: 012e77eb5c3ba3d411f5967a7f368ebdb42ab88c branch: main author: Andrew Svetlov committer: asvetlov date: 2022-02-10T14:57:20+02:00 summary: Fix warning: asyncio.events._event_loop_policy was modified by test_asyncio (GH-31253) files: A Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst M Lib/test/test_asyncio/test_futures2.py M Lib/test/test_asyncio/test_protocols.py M Lib/test/test_asyncio/test_runners.py M Lib/test/test_asyncio/test_sock_lowlevel.py M Lib/test/test_asyncio/test_transports.py M Lib/test/test_asyncio/test_unix_events.py diff --git a/Lib/test/test_asyncio/test_futures2.py b/Lib/test/test_asyncio/test_futures2.py index 57d24190bc0bd..60b58850369f0 100644 --- a/Lib/test/test_asyncio/test_futures2.py +++ b/Lib/test/test_asyncio/test_futures2.py @@ -3,6 +3,10 @@ import unittest +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + class FutureTests(unittest.IsolatedAsyncioTestCase): async def test_recursive_repr_for_pending_tasks(self): # The call crashes if the guard for recursive call diff --git a/Lib/test/test_asyncio/test_protocols.py b/Lib/test/test_asyncio/test_protocols.py index d8cde6d89aadc..0f232631867db 100644 --- a/Lib/test/test_asyncio/test_protocols.py +++ b/Lib/test/test_asyncio/test_protocols.py @@ -4,6 +4,12 @@ import asyncio +def tearDownModule(): + # not needed for the test file but added for uniformness with all other + # asyncio test files for the sake of unified cleanup + asyncio.set_event_loop_policy(None) + + class ProtocolsAbsTests(unittest.TestCase): def test_base_protocol(self): diff --git a/Lib/test/test_asyncio/test_runners.py b/Lib/test/test_asyncio/test_runners.py index 5c06a1aaa830f..112273662b20b 100644 --- a/Lib/test/test_asyncio/test_runners.py +++ b/Lib/test/test_asyncio/test_runners.py @@ -5,6 +5,10 @@ from test.test_asyncio import utils as test_utils +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + class TestPolicy(asyncio.AbstractEventLoopPolicy): def __init__(self, loop_factory): diff --git a/Lib/test/test_asyncio/test_sock_lowlevel.py b/Lib/test/test_asyncio/test_sock_lowlevel.py index 448d835b04d57..14001a4a5001f 100644 --- a/Lib/test/test_asyncio/test_sock_lowlevel.py +++ b/Lib/test/test_asyncio/test_sock_lowlevel.py @@ -10,6 +10,10 @@ from test.support import socket_helper +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + class MyProto(asyncio.Protocol): connected = None done = None diff --git a/Lib/test/test_asyncio/test_transports.py b/Lib/test/test_asyncio/test_transports.py index df448557a7b7f..bbdb218efaa3b 100644 --- a/Lib/test/test_asyncio/test_transports.py +++ b/Lib/test/test_asyncio/test_transports.py @@ -7,6 +7,12 @@ from asyncio import transports +def tearDownModule(): + # not needed for the test file but added for uniformness with all other + # asyncio test files for the sake of unified cleanup + asyncio.set_event_loop_policy(None) + + class TransportTests(unittest.TestCase): def test_ctor_extra_is_none(self): diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py index c3422850ce1b7..2f68459d30cd4 100644 --- a/Lib/test/test_asyncio/test_unix_events.py +++ b/Lib/test/test_asyncio/test_unix_events.py @@ -26,6 +26,10 @@ from test.test_asyncio import utils as test_utils +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + MOCK_ANY = mock.ANY @@ -39,10 +43,6 @@ def SIGNAL(signum): return 32768 - signum -def tearDownModule(): - asyncio.set_event_loop_policy(None) - - def close_pipe_transport(transport): # Don't call transport.close() because the event loop and the selector # are mocked diff --git a/Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst b/Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst new file mode 100644 index 0000000000000..119107a8fb96c --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst @@ -0,0 +1,2 @@ +Prevent default asyncio event loop policy modification warning after +``test_asyncio`` execution. From webhook-mailer at python.org Thu Feb 10 07:58:00 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 10 Feb 2022 12:58:00 -0000 Subject: [Python-checkins] bpo-46532: Reduce number of memory writes to update call_shape.kwnames. (GH-31231) Message-ID: https://github.com/python/cpython/commit/2cea8c29cf975a8ad7d8c3ff19d1e836c2d54707 commit: 2cea8c29cf975a8ad7d8c3ff19d1e836c2d54707 branch: main author: Mark Shannon committer: markshannon date: 2022-02-10T12:57:55Z summary: bpo-46532: Reduce number of memory writes to update call_shape.kwnames. (GH-31231) files: M Python/ceval.c diff --git a/Python/ceval.c b/Python/ceval.c index 7e19043c7aa64..c3703a75ce0ca 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1615,7 +1615,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr CFrame cframe; CallShape call_shape; - call_shape.kwnames = NULL; // Borrowed reference + call_shape.kwnames = NULL; // Borrowed reference. Reset by CALL instructions. + /* The following three values are always set by the PRECALL instructions. + They are set here to keep the compiler happy. */ call_shape.postcall_shrink = 0; call_shape.total_args = 0; call_shape.callable = NULL; // Strong reference @@ -4446,7 +4448,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr call_shape.postcall_shrink = 1; call_shape.total_args = oparg; - call_shape.kwnames = NULL; + assert(call_shape.kwnames == NULL); #ifdef Py_STATS extern int _PySpecialization_ClassifyCallable(PyObject *); _py_stats.opcode_stats[PRECALL_FUNCTION].specialization.failure++; @@ -4490,12 +4492,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr call_shape.postcall_shrink = 2-is_method; call_shape.total_args = nargs; - call_shape.kwnames = NULL; + assert(call_shape.kwnames == NULL); DISPATCH(); } TARGET(KW_NAMES) { - assert(call_shape.kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(consts)); call_shape.kwnames = GETITEM(consts, oparg); DISPATCH(); @@ -4531,6 +4532,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr tstate, (PyFunctionObject *)function, locals, stack_pointer, positional_args, call_shape.kwnames ); + call_shape.kwnames = NULL; STACK_SHRINK(call_shape.postcall_shrink); // The frame has stolen all the arguments from the stack, // so there is no need to clean them up. @@ -4556,6 +4558,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr positional_args | PY_VECTORCALL_ARGUMENTS_OFFSET, call_shape.kwnames); } + call_shape.kwnames = NULL; assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); Py_DECREF(function); /* Clear the stack */ @@ -4597,6 +4600,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(CALL_PY_EXACT_ARGS) { + assert(call_shape.kwnames == NULL); SpecializedCacheEntry *caches = GET_CACHE(); int argcount = call_shape.total_args; DEOPT_IF(!PyFunction_Check(call_shape.callable), CALL); @@ -4625,6 +4629,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(CALL_PY_WITH_DEFAULTS) { + assert(call_shape.kwnames == NULL); SpecializedCacheEntry *caches = GET_CACHE(); int argcount = call_shape.total_args; DEOPT_IF(!PyFunction_Check(call_shape.callable), CALL); @@ -4661,9 +4666,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(CALL_NO_KW_TYPE_1) { + assert(call_shape.kwnames == NULL); assert(cframe.use_tracing == 0); DEOPT_IF(call_shape.total_args != 1, CALL); - assert(call_shape.kwnames == NULL); PyObject *obj = TOP(); PyObject *callable = SECOND(); DEOPT_IF(callable != (PyObject *)&PyType_Type, CALL); @@ -4676,13 +4681,13 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(CALL_NO_KW_STR_1) { + assert(call_shape.kwnames == NULL); assert(cframe.use_tracing == 0); DEOPT_IF(!PyType_Check(call_shape.callable), CALL); PyTypeObject *tp = (PyTypeObject *)call_shape.callable; DEOPT_IF(call_shape.total_args != 1, CALL); DEOPT_IF(tp != &PyUnicode_Type, CALL); STAT_INC(CALL, hit); - assert(call_shape.kwnames == NULL); PyObject *arg = TOP(); PyObject *res = PyObject_Str(arg); Py_DECREF(arg); @@ -4696,12 +4701,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(CALL_NO_KW_TUPLE_1) { + assert(call_shape.kwnames == NULL); DEOPT_IF(!PyType_Check(call_shape.callable), CALL); PyTypeObject *tp = (PyTypeObject *)call_shape.callable; DEOPT_IF(call_shape.total_args != 1, CALL); DEOPT_IF(tp != &PyTuple_Type, CALL); STAT_INC(CALL, hit); - assert(call_shape.kwnames == NULL); PyObject *arg = TOP(); PyObject *res = PySequence_Tuple(arg); Py_DECREF(arg); @@ -4724,6 +4729,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int nargs = call_shape.total_args - kwnames_len; STACK_SHRINK(call_shape.total_args); PyObject *res = tp->tp_vectorcall((PyObject *)tp, stack_pointer, nargs, call_shape.kwnames); + call_shape.kwnames = NULL; /* Free the arguments. */ for (int i = 0; i < call_shape.total_args; i++) { Py_DECREF(stack_pointer[i]); @@ -4833,6 +4839,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr call_shape.kwnames ); assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); + call_shape.kwnames = NULL; /* Free the arguments. */ for (int i = 0; i < call_shape.total_args; i++) { @@ -5398,6 +5405,7 @@ MISS_WITH_OPARG_COUNTER(STORE_SUBSCR) } error: + call_shape.kwnames = NULL; /* Double-check exception status. */ #ifdef NDEBUG if (!_PyErr_Occurred(tstate)) { From webhook-mailer at python.org Thu Feb 10 08:12:24 2022 From: webhook-mailer at python.org (pablogsal) Date: Thu, 10 Feb 2022 13:12:24 -0000 Subject: [Python-checkins] Allow the parser to avoid nested processing of invalid rules (GH-31252) Message-ID: https://github.com/python/cpython/commit/390459de6db1e68b79c0897cc88c0d562693ec5c commit: 390459de6db1e68b79c0897cc88c0d562693ec5c branch: main author: Pablo Galindo Salgado committer: pablogsal date: 2022-02-10T13:12:14Z summary: Allow the parser to avoid nested processing of invalid rules (GH-31252) files: M Grammar/python.gram M Lib/test/test_exceptions.py M Parser/parser.c M Parser/pegen.c M Parser/pegen.h M Tools/peg_generator/pegen/c_generator.py diff --git a/Grammar/python.gram b/Grammar/python.gram index 230d6ca6208e0..c0a64696e85dc 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -1078,6 +1078,7 @@ invalid_kwarg: RAISE_SYNTAX_ERROR_KNOWN_RANGE( a, b, "expression cannot contain assignment, perhaps you meant \"==\"?") } +# IMPORTANT: Note that the "_without_invalid" suffix causes the rule to not call invalid rules under it expression_without_invalid[expr_ty]: | a=disjunction 'if' b=disjunction 'else' c=expression { _PyAST_IfExp(b, a, c, EXTRA) } | disjunction @@ -1095,16 +1096,14 @@ invalid_expression: RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Perhaps you forgot a comma?") } | a=disjunction 'if' b=disjunction !('else'|':') { RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "expected 'else' after 'if' expression") } -invalid_left_assignment_prefixes(memo): list|tuple|genexp|'True'|'None'|'False' - -invalid_named_expression: +invalid_named_expression(memo): | a=expression ':=' expression { RAISE_SYNTAX_ERROR_KNOWN_LOCATION( a, "cannot use assignment expressions with %s", _PyPegen_get_expr_name(a)) } | a=NAME '=' b=bitwise_or !('='|':=') { - p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?") } - | !invalid_left_assignment_prefixes a=bitwise_or b='=' bitwise_or !('='|':=') { - p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "cannot assign to %s here. Maybe you meant '==' instead of '='?", + RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?") } + | !(list|tuple|genexp|'True'|'None'|'False') a=bitwise_or b='=' bitwise_or !('='|':=') { + RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "cannot assign to %s here. Maybe you meant '==' instead of '='?", _PyPegen_get_expr_name(a)) } invalid_assignment: diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py index 898b5cc067d67..e3897a0d89c5e 100644 --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -231,7 +231,7 @@ def testSyntaxErrorOffset(self): check('a = ? hello ? ? world ?', 1, 5) check('[\nfile\nfor str(file)\nin\n[]\n]', 3, 5) check('[file for\n str(file) in []]', 2, 2) - check("ages = {'Alice'=22, 'Bob'=23}", 1, 16) + check("ages = {'Alice'=22, 'Bob'=23}", 1, 9) check('match ...:\n case {**rest, "key": value}:\n ...', 2, 19) check("[a b c d e f]", 1, 2) check("for x yfff:", 1, 7) diff --git a/Parser/parser.c b/Parser/parser.c index cf1a19dc23d89..7f2d125996d2f 100644 --- a/Parser/parser.c +++ b/Parser/parser.c @@ -257,267 +257,267 @@ static char *soft_keywords[] = { #define expression_without_invalid_type 1181 #define invalid_legacy_expression_type 1182 #define invalid_expression_type 1183 -#define invalid_left_assignment_prefixes_type 1184 -#define invalid_named_expression_type 1185 -#define invalid_assignment_type 1186 -#define invalid_ann_assign_target_type 1187 -#define invalid_del_stmt_type 1188 -#define invalid_block_type 1189 -#define invalid_comprehension_type 1190 -#define invalid_dict_comprehension_type 1191 -#define invalid_parameters_type 1192 -#define invalid_parameters_helper_type 1193 -#define invalid_lambda_parameters_type 1194 -#define invalid_lambda_parameters_helper_type 1195 -#define invalid_star_etc_type 1196 -#define invalid_lambda_star_etc_type 1197 -#define invalid_double_type_comments_type 1198 -#define invalid_with_item_type 1199 -#define invalid_for_target_type 1200 -#define invalid_group_type 1201 -#define invalid_import_from_targets_type 1202 -#define invalid_with_stmt_type 1203 -#define invalid_with_stmt_indent_type 1204 -#define invalid_try_stmt_type 1205 -#define invalid_except_stmt_type 1206 -#define invalid_finally_stmt_type 1207 -#define invalid_except_stmt_indent_type 1208 -#define invalid_except_star_stmt_indent_type 1209 -#define invalid_match_stmt_type 1210 -#define invalid_case_block_type 1211 -#define invalid_as_pattern_type 1212 -#define invalid_class_pattern_type 1213 -#define invalid_class_argument_pattern_type 1214 -#define invalid_if_stmt_type 1215 -#define invalid_elif_stmt_type 1216 -#define invalid_else_stmt_type 1217 -#define invalid_while_stmt_type 1218 -#define invalid_for_stmt_type 1219 -#define invalid_def_raw_type 1220 -#define invalid_class_def_raw_type 1221 -#define invalid_double_starred_kvpairs_type 1222 -#define invalid_kvpair_type 1223 -#define _loop0_1_type 1224 -#define _loop0_2_type 1225 -#define _loop1_3_type 1226 -#define _loop0_5_type 1227 -#define _gather_4_type 1228 -#define _tmp_6_type 1229 -#define _tmp_7_type 1230 -#define _tmp_8_type 1231 -#define _tmp_9_type 1232 -#define _tmp_10_type 1233 -#define _tmp_11_type 1234 -#define _tmp_12_type 1235 -#define _tmp_13_type 1236 -#define _loop1_14_type 1237 -#define _tmp_15_type 1238 -#define _tmp_16_type 1239 -#define _tmp_17_type 1240 -#define _loop0_19_type 1241 -#define _gather_18_type 1242 -#define _loop0_21_type 1243 -#define _gather_20_type 1244 -#define _tmp_22_type 1245 -#define _tmp_23_type 1246 -#define _loop0_24_type 1247 -#define _loop1_25_type 1248 -#define _loop0_27_type 1249 -#define _gather_26_type 1250 -#define _tmp_28_type 1251 -#define _loop0_30_type 1252 -#define _gather_29_type 1253 -#define _tmp_31_type 1254 -#define _loop1_32_type 1255 -#define _tmp_33_type 1256 -#define _tmp_34_type 1257 -#define _tmp_35_type 1258 -#define _loop0_36_type 1259 -#define _loop0_37_type 1260 -#define _loop0_38_type 1261 -#define _loop1_39_type 1262 -#define _loop0_40_type 1263 -#define _loop1_41_type 1264 -#define _loop1_42_type 1265 -#define _loop1_43_type 1266 -#define _loop0_44_type 1267 -#define _loop1_45_type 1268 -#define _loop0_46_type 1269 -#define _loop1_47_type 1270 -#define _loop0_48_type 1271 -#define _loop1_49_type 1272 -#define _loop0_51_type 1273 -#define _gather_50_type 1274 -#define _loop0_53_type 1275 -#define _gather_52_type 1276 -#define _loop0_55_type 1277 -#define _gather_54_type 1278 -#define _loop0_57_type 1279 -#define _gather_56_type 1280 -#define _tmp_58_type 1281 -#define _loop1_59_type 1282 -#define _loop1_60_type 1283 -#define _tmp_61_type 1284 -#define _tmp_62_type 1285 -#define _loop1_63_type 1286 -#define _loop0_65_type 1287 -#define _gather_64_type 1288 -#define _tmp_66_type 1289 -#define _tmp_67_type 1290 -#define _tmp_68_type 1291 -#define _tmp_69_type 1292 -#define _loop0_71_type 1293 -#define _gather_70_type 1294 -#define _loop0_73_type 1295 -#define _gather_72_type 1296 -#define _tmp_74_type 1297 -#define _loop0_76_type 1298 -#define _gather_75_type 1299 -#define _loop0_78_type 1300 -#define _gather_77_type 1301 -#define _loop1_79_type 1302 -#define _loop1_80_type 1303 -#define _loop0_82_type 1304 -#define _gather_81_type 1305 -#define _loop1_83_type 1306 -#define _loop1_84_type 1307 -#define _loop1_85_type 1308 -#define _tmp_86_type 1309 -#define _loop0_88_type 1310 -#define _gather_87_type 1311 -#define _tmp_89_type 1312 -#define _tmp_90_type 1313 -#define _tmp_91_type 1314 -#define _tmp_92_type 1315 -#define _tmp_93_type 1316 -#define _loop0_94_type 1317 -#define _loop0_95_type 1318 -#define _loop0_96_type 1319 -#define _loop1_97_type 1320 -#define _loop0_98_type 1321 -#define _loop1_99_type 1322 -#define _loop1_100_type 1323 -#define _loop1_101_type 1324 -#define _loop0_102_type 1325 -#define _loop1_103_type 1326 -#define _loop0_104_type 1327 -#define _loop1_105_type 1328 -#define _loop0_106_type 1329 -#define _loop1_107_type 1330 -#define _loop1_108_type 1331 -#define _tmp_109_type 1332 -#define _loop0_111_type 1333 -#define _gather_110_type 1334 -#define _loop1_112_type 1335 -#define _loop0_113_type 1336 -#define _loop0_114_type 1337 -#define _tmp_115_type 1338 -#define _loop0_117_type 1339 -#define _gather_116_type 1340 -#define _tmp_118_type 1341 -#define _loop0_120_type 1342 -#define _gather_119_type 1343 -#define _loop0_122_type 1344 -#define _gather_121_type 1345 -#define _loop0_124_type 1346 -#define _gather_123_type 1347 -#define _loop0_126_type 1348 -#define _gather_125_type 1349 -#define _loop0_127_type 1350 -#define _loop0_129_type 1351 -#define _gather_128_type 1352 -#define _loop1_130_type 1353 -#define _tmp_131_type 1354 -#define _loop0_133_type 1355 -#define _gather_132_type 1356 -#define _loop0_135_type 1357 -#define _gather_134_type 1358 -#define _loop0_137_type 1359 -#define _gather_136_type 1360 -#define _loop0_139_type 1361 -#define _gather_138_type 1362 -#define _loop0_141_type 1363 -#define _gather_140_type 1364 -#define _tmp_142_type 1365 -#define _tmp_143_type 1366 -#define _tmp_144_type 1367 -#define _tmp_145_type 1368 -#define _tmp_146_type 1369 -#define _tmp_147_type 1370 -#define _tmp_148_type 1371 -#define _tmp_149_type 1372 -#define _loop0_150_type 1373 -#define _loop0_151_type 1374 -#define _loop0_152_type 1375 -#define _tmp_153_type 1376 -#define _tmp_154_type 1377 -#define _tmp_155_type 1378 -#define _tmp_156_type 1379 -#define _loop0_157_type 1380 -#define _loop0_158_type 1381 -#define _loop1_159_type 1382 -#define _loop1_160_type 1383 -#define _loop0_161_type 1384 -#define _loop0_162_type 1385 -#define _loop0_164_type 1386 -#define _gather_163_type 1387 -#define _loop1_165_type 1388 -#define _tmp_166_type 1389 -#define _tmp_167_type 1390 -#define _tmp_168_type 1391 -#define _loop0_170_type 1392 -#define _gather_169_type 1393 -#define _loop0_172_type 1394 -#define _gather_171_type 1395 -#define _loop0_174_type 1396 -#define _gather_173_type 1397 -#define _loop0_176_type 1398 -#define _gather_175_type 1399 -#define _tmp_177_type 1400 -#define _loop0_178_type 1401 -#define _tmp_179_type 1402 -#define _loop0_180_type 1403 -#define _tmp_181_type 1404 -#define _tmp_182_type 1405 -#define _tmp_183_type 1406 -#define _tmp_184_type 1407 -#define _tmp_185_type 1408 -#define _tmp_186_type 1409 -#define _tmp_187_type 1410 -#define _tmp_188_type 1411 -#define _loop0_190_type 1412 -#define _gather_189_type 1413 -#define _tmp_191_type 1414 -#define _tmp_192_type 1415 -#define _tmp_193_type 1416 -#define _tmp_194_type 1417 -#define _tmp_195_type 1418 -#define _tmp_196_type 1419 -#define _tmp_197_type 1420 -#define _tmp_198_type 1421 -#define _tmp_199_type 1422 -#define _tmp_200_type 1423 -#define _tmp_201_type 1424 -#define _tmp_202_type 1425 -#define _tmp_203_type 1426 -#define _tmp_204_type 1427 -#define _tmp_205_type 1428 -#define _tmp_206_type 1429 -#define _tmp_207_type 1430 -#define _tmp_208_type 1431 -#define _tmp_209_type 1432 -#define _tmp_210_type 1433 -#define _tmp_211_type 1434 -#define _tmp_212_type 1435 -#define _tmp_213_type 1436 -#define _tmp_214_type 1437 -#define _tmp_215_type 1438 -#define _tmp_216_type 1439 -#define _tmp_217_type 1440 -#define _tmp_218_type 1441 -#define _tmp_219_type 1442 -#define _loop1_220_type 1443 -#define _loop1_221_type 1444 +#define invalid_named_expression_type 1184 +#define invalid_assignment_type 1185 +#define invalid_ann_assign_target_type 1186 +#define invalid_del_stmt_type 1187 +#define invalid_block_type 1188 +#define invalid_comprehension_type 1189 +#define invalid_dict_comprehension_type 1190 +#define invalid_parameters_type 1191 +#define invalid_parameters_helper_type 1192 +#define invalid_lambda_parameters_type 1193 +#define invalid_lambda_parameters_helper_type 1194 +#define invalid_star_etc_type 1195 +#define invalid_lambda_star_etc_type 1196 +#define invalid_double_type_comments_type 1197 +#define invalid_with_item_type 1198 +#define invalid_for_target_type 1199 +#define invalid_group_type 1200 +#define invalid_import_from_targets_type 1201 +#define invalid_with_stmt_type 1202 +#define invalid_with_stmt_indent_type 1203 +#define invalid_try_stmt_type 1204 +#define invalid_except_stmt_type 1205 +#define invalid_finally_stmt_type 1206 +#define invalid_except_stmt_indent_type 1207 +#define invalid_except_star_stmt_indent_type 1208 +#define invalid_match_stmt_type 1209 +#define invalid_case_block_type 1210 +#define invalid_as_pattern_type 1211 +#define invalid_class_pattern_type 1212 +#define invalid_class_argument_pattern_type 1213 +#define invalid_if_stmt_type 1214 +#define invalid_elif_stmt_type 1215 +#define invalid_else_stmt_type 1216 +#define invalid_while_stmt_type 1217 +#define invalid_for_stmt_type 1218 +#define invalid_def_raw_type 1219 +#define invalid_class_def_raw_type 1220 +#define invalid_double_starred_kvpairs_type 1221 +#define invalid_kvpair_type 1222 +#define _loop0_1_type 1223 +#define _loop0_2_type 1224 +#define _loop1_3_type 1225 +#define _loop0_5_type 1226 +#define _gather_4_type 1227 +#define _tmp_6_type 1228 +#define _tmp_7_type 1229 +#define _tmp_8_type 1230 +#define _tmp_9_type 1231 +#define _tmp_10_type 1232 +#define _tmp_11_type 1233 +#define _tmp_12_type 1234 +#define _tmp_13_type 1235 +#define _loop1_14_type 1236 +#define _tmp_15_type 1237 +#define _tmp_16_type 1238 +#define _tmp_17_type 1239 +#define _loop0_19_type 1240 +#define _gather_18_type 1241 +#define _loop0_21_type 1242 +#define _gather_20_type 1243 +#define _tmp_22_type 1244 +#define _tmp_23_type 1245 +#define _loop0_24_type 1246 +#define _loop1_25_type 1247 +#define _loop0_27_type 1248 +#define _gather_26_type 1249 +#define _tmp_28_type 1250 +#define _loop0_30_type 1251 +#define _gather_29_type 1252 +#define _tmp_31_type 1253 +#define _loop1_32_type 1254 +#define _tmp_33_type 1255 +#define _tmp_34_type 1256 +#define _tmp_35_type 1257 +#define _loop0_36_type 1258 +#define _loop0_37_type 1259 +#define _loop0_38_type 1260 +#define _loop1_39_type 1261 +#define _loop0_40_type 1262 +#define _loop1_41_type 1263 +#define _loop1_42_type 1264 +#define _loop1_43_type 1265 +#define _loop0_44_type 1266 +#define _loop1_45_type 1267 +#define _loop0_46_type 1268 +#define _loop1_47_type 1269 +#define _loop0_48_type 1270 +#define _loop1_49_type 1271 +#define _loop0_51_type 1272 +#define _gather_50_type 1273 +#define _loop0_53_type 1274 +#define _gather_52_type 1275 +#define _loop0_55_type 1276 +#define _gather_54_type 1277 +#define _loop0_57_type 1278 +#define _gather_56_type 1279 +#define _tmp_58_type 1280 +#define _loop1_59_type 1281 +#define _loop1_60_type 1282 +#define _tmp_61_type 1283 +#define _tmp_62_type 1284 +#define _loop1_63_type 1285 +#define _loop0_65_type 1286 +#define _gather_64_type 1287 +#define _tmp_66_type 1288 +#define _tmp_67_type 1289 +#define _tmp_68_type 1290 +#define _tmp_69_type 1291 +#define _loop0_71_type 1292 +#define _gather_70_type 1293 +#define _loop0_73_type 1294 +#define _gather_72_type 1295 +#define _tmp_74_type 1296 +#define _loop0_76_type 1297 +#define _gather_75_type 1298 +#define _loop0_78_type 1299 +#define _gather_77_type 1300 +#define _loop1_79_type 1301 +#define _loop1_80_type 1302 +#define _loop0_82_type 1303 +#define _gather_81_type 1304 +#define _loop1_83_type 1305 +#define _loop1_84_type 1306 +#define _loop1_85_type 1307 +#define _tmp_86_type 1308 +#define _loop0_88_type 1309 +#define _gather_87_type 1310 +#define _tmp_89_type 1311 +#define _tmp_90_type 1312 +#define _tmp_91_type 1313 +#define _tmp_92_type 1314 +#define _tmp_93_type 1315 +#define _loop0_94_type 1316 +#define _loop0_95_type 1317 +#define _loop0_96_type 1318 +#define _loop1_97_type 1319 +#define _loop0_98_type 1320 +#define _loop1_99_type 1321 +#define _loop1_100_type 1322 +#define _loop1_101_type 1323 +#define _loop0_102_type 1324 +#define _loop1_103_type 1325 +#define _loop0_104_type 1326 +#define _loop1_105_type 1327 +#define _loop0_106_type 1328 +#define _loop1_107_type 1329 +#define _loop1_108_type 1330 +#define _tmp_109_type 1331 +#define _loop0_111_type 1332 +#define _gather_110_type 1333 +#define _loop1_112_type 1334 +#define _loop0_113_type 1335 +#define _loop0_114_type 1336 +#define _tmp_115_type 1337 +#define _loop0_117_type 1338 +#define _gather_116_type 1339 +#define _tmp_118_type 1340 +#define _loop0_120_type 1341 +#define _gather_119_type 1342 +#define _loop0_122_type 1343 +#define _gather_121_type 1344 +#define _loop0_124_type 1345 +#define _gather_123_type 1346 +#define _loop0_126_type 1347 +#define _gather_125_type 1348 +#define _loop0_127_type 1349 +#define _loop0_129_type 1350 +#define _gather_128_type 1351 +#define _loop1_130_type 1352 +#define _tmp_131_type 1353 +#define _loop0_133_type 1354 +#define _gather_132_type 1355 +#define _loop0_135_type 1356 +#define _gather_134_type 1357 +#define _loop0_137_type 1358 +#define _gather_136_type 1359 +#define _loop0_139_type 1360 +#define _gather_138_type 1361 +#define _loop0_141_type 1362 +#define _gather_140_type 1363 +#define _tmp_142_type 1364 +#define _tmp_143_type 1365 +#define _tmp_144_type 1366 +#define _tmp_145_type 1367 +#define _tmp_146_type 1368 +#define _tmp_147_type 1369 +#define _tmp_148_type 1370 +#define _tmp_149_type 1371 +#define _tmp_150_type 1372 +#define _loop0_151_type 1373 +#define _loop0_152_type 1374 +#define _loop0_153_type 1375 +#define _tmp_154_type 1376 +#define _tmp_155_type 1377 +#define _tmp_156_type 1378 +#define _tmp_157_type 1379 +#define _loop0_158_type 1380 +#define _loop0_159_type 1381 +#define _loop1_160_type 1382 +#define _loop1_161_type 1383 +#define _loop0_162_type 1384 +#define _loop0_163_type 1385 +#define _loop0_165_type 1386 +#define _gather_164_type 1387 +#define _loop1_166_type 1388 +#define _tmp_167_type 1389 +#define _tmp_168_type 1390 +#define _tmp_169_type 1391 +#define _loop0_171_type 1392 +#define _gather_170_type 1393 +#define _loop0_173_type 1394 +#define _gather_172_type 1395 +#define _loop0_175_type 1396 +#define _gather_174_type 1397 +#define _loop0_177_type 1398 +#define _gather_176_type 1399 +#define _tmp_178_type 1400 +#define _loop0_179_type 1401 +#define _tmp_180_type 1402 +#define _loop0_181_type 1403 +#define _tmp_182_type 1404 +#define _tmp_183_type 1405 +#define _tmp_184_type 1406 +#define _tmp_185_type 1407 +#define _tmp_186_type 1408 +#define _tmp_187_type 1409 +#define _tmp_188_type 1410 +#define _tmp_189_type 1411 +#define _loop0_191_type 1412 +#define _gather_190_type 1413 +#define _tmp_192_type 1414 +#define _tmp_193_type 1415 +#define _tmp_194_type 1416 +#define _tmp_195_type 1417 +#define _tmp_196_type 1418 +#define _tmp_197_type 1419 +#define _tmp_198_type 1420 +#define _tmp_199_type 1421 +#define _tmp_200_type 1422 +#define _tmp_201_type 1423 +#define _tmp_202_type 1424 +#define _tmp_203_type 1425 +#define _tmp_204_type 1426 +#define _tmp_205_type 1427 +#define _tmp_206_type 1428 +#define _tmp_207_type 1429 +#define _tmp_208_type 1430 +#define _tmp_209_type 1431 +#define _tmp_210_type 1432 +#define _tmp_211_type 1433 +#define _tmp_212_type 1434 +#define _tmp_213_type 1435 +#define _tmp_214_type 1436 +#define _tmp_215_type 1437 +#define _tmp_216_type 1438 +#define _tmp_217_type 1439 +#define _tmp_218_type 1440 +#define _tmp_219_type 1441 +#define _tmp_220_type 1442 +#define _loop1_221_type 1443 +#define _loop1_222_type 1444 static mod_ty file_rule(Parser *p); static mod_ty interactive_rule(Parser *p); @@ -703,7 +703,6 @@ static void *invalid_kwarg_rule(Parser *p); static expr_ty expression_without_invalid_rule(Parser *p); static void *invalid_legacy_expression_rule(Parser *p); static void *invalid_expression_rule(Parser *p); -static void *invalid_left_assignment_prefixes_rule(Parser *p); static void *invalid_named_expression_rule(Parser *p); static void *invalid_assignment_rule(Parser *p); static expr_ty invalid_ann_assign_target_rule(Parser *p); @@ -892,38 +891,38 @@ static void *_tmp_146_rule(Parser *p); static void *_tmp_147_rule(Parser *p); static void *_tmp_148_rule(Parser *p); static void *_tmp_149_rule(Parser *p); -static asdl_seq *_loop0_150_rule(Parser *p); +static void *_tmp_150_rule(Parser *p); static asdl_seq *_loop0_151_rule(Parser *p); static asdl_seq *_loop0_152_rule(Parser *p); -static void *_tmp_153_rule(Parser *p); +static asdl_seq *_loop0_153_rule(Parser *p); static void *_tmp_154_rule(Parser *p); static void *_tmp_155_rule(Parser *p); static void *_tmp_156_rule(Parser *p); -static asdl_seq *_loop0_157_rule(Parser *p); +static void *_tmp_157_rule(Parser *p); static asdl_seq *_loop0_158_rule(Parser *p); -static asdl_seq *_loop1_159_rule(Parser *p); +static asdl_seq *_loop0_159_rule(Parser *p); static asdl_seq *_loop1_160_rule(Parser *p); -static asdl_seq *_loop0_161_rule(Parser *p); +static asdl_seq *_loop1_161_rule(Parser *p); static asdl_seq *_loop0_162_rule(Parser *p); -static asdl_seq *_loop0_164_rule(Parser *p); -static asdl_seq *_gather_163_rule(Parser *p); -static asdl_seq *_loop1_165_rule(Parser *p); -static void *_tmp_166_rule(Parser *p); +static asdl_seq *_loop0_163_rule(Parser *p); +static asdl_seq *_loop0_165_rule(Parser *p); +static asdl_seq *_gather_164_rule(Parser *p); +static asdl_seq *_loop1_166_rule(Parser *p); static void *_tmp_167_rule(Parser *p); static void *_tmp_168_rule(Parser *p); -static asdl_seq *_loop0_170_rule(Parser *p); -static asdl_seq *_gather_169_rule(Parser *p); -static asdl_seq *_loop0_172_rule(Parser *p); -static asdl_seq *_gather_171_rule(Parser *p); -static asdl_seq *_loop0_174_rule(Parser *p); -static asdl_seq *_gather_173_rule(Parser *p); -static asdl_seq *_loop0_176_rule(Parser *p); -static asdl_seq *_gather_175_rule(Parser *p); -static void *_tmp_177_rule(Parser *p); -static asdl_seq *_loop0_178_rule(Parser *p); -static void *_tmp_179_rule(Parser *p); -static asdl_seq *_loop0_180_rule(Parser *p); -static void *_tmp_181_rule(Parser *p); +static void *_tmp_169_rule(Parser *p); +static asdl_seq *_loop0_171_rule(Parser *p); +static asdl_seq *_gather_170_rule(Parser *p); +static asdl_seq *_loop0_173_rule(Parser *p); +static asdl_seq *_gather_172_rule(Parser *p); +static asdl_seq *_loop0_175_rule(Parser *p); +static asdl_seq *_gather_174_rule(Parser *p); +static asdl_seq *_loop0_177_rule(Parser *p); +static asdl_seq *_gather_176_rule(Parser *p); +static void *_tmp_178_rule(Parser *p); +static asdl_seq *_loop0_179_rule(Parser *p); +static void *_tmp_180_rule(Parser *p); +static asdl_seq *_loop0_181_rule(Parser *p); static void *_tmp_182_rule(Parser *p); static void *_tmp_183_rule(Parser *p); static void *_tmp_184_rule(Parser *p); @@ -931,9 +930,9 @@ static void *_tmp_185_rule(Parser *p); static void *_tmp_186_rule(Parser *p); static void *_tmp_187_rule(Parser *p); static void *_tmp_188_rule(Parser *p); -static asdl_seq *_loop0_190_rule(Parser *p); -static asdl_seq *_gather_189_rule(Parser *p); -static void *_tmp_191_rule(Parser *p); +static void *_tmp_189_rule(Parser *p); +static asdl_seq *_loop0_191_rule(Parser *p); +static asdl_seq *_gather_190_rule(Parser *p); static void *_tmp_192_rule(Parser *p); static void *_tmp_193_rule(Parser *p); static void *_tmp_194_rule(Parser *p); @@ -962,8 +961,9 @@ static void *_tmp_216_rule(Parser *p); static void *_tmp_217_rule(Parser *p); static void *_tmp_218_rule(Parser *p); static void *_tmp_219_rule(Parser *p); -static asdl_seq *_loop1_220_rule(Parser *p); +static void *_tmp_220_rule(Parser *p); static asdl_seq *_loop1_221_rule(Parser *p); +static asdl_seq *_loop1_222_rule(Parser *p); // file: statements? $ @@ -3854,9 +3854,7 @@ dotted_name_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = dotted_name_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -8828,9 +8826,7 @@ attr_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = attr_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -12054,9 +12050,7 @@ bitwise_or_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = bitwise_or_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -12180,9 +12174,7 @@ bitwise_xor_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = bitwise_xor_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -12306,9 +12298,7 @@ bitwise_and_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = bitwise_and_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -12432,9 +12422,7 @@ shift_expr_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = shift_expr_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -12597,9 +12585,7 @@ sum_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = sum_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -12768,9 +12754,7 @@ term_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = term_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -13392,9 +13376,7 @@ primary_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = primary_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -17567,9 +17549,7 @@ t_primary_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = t_primary_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -18902,11 +18882,14 @@ invalid_kwarg_rule(Parser *p) static expr_ty expression_without_invalid_rule(Parser *p) { + int _prev_call_invalid = p->call_invalid_rules; + p->call_invalid_rules = 0; if (p->level++ == MAXSTACK) { p->error_indicator = 1; PyErr_NoMemory(); } if (p->error_indicator) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18914,6 +18897,7 @@ expression_without_invalid_rule(Parser *p) int _mark = p->mark; if (p->mark == p->fill && _PyPegen_fill_token(p) < 0) { p->error_indicator = 1; + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18923,6 +18907,7 @@ expression_without_invalid_rule(Parser *p) UNUSED(_start_col_offset); // Only used by EXTRA macro { // disjunction 'if' disjunction 'else' expression if (p->error_indicator) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18947,6 +18932,7 @@ expression_without_invalid_rule(Parser *p) D(fprintf(stderr, "%*c+ expression_without_invalid[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "disjunction 'if' disjunction 'else' expression")); Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); if (_token == NULL) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18957,6 +18943,7 @@ expression_without_invalid_rule(Parser *p) _res = _PyAST_IfExp ( b , a , c , EXTRA ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18968,6 +18955,7 @@ expression_without_invalid_rule(Parser *p) } { // disjunction if (p->error_indicator) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18987,6 +18975,7 @@ expression_without_invalid_rule(Parser *p) } { // lambdef if (p->error_indicator) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -19006,6 +18995,7 @@ expression_without_invalid_rule(Parser *p) } _res = NULL; done: + p->call_invalid_rules = _prev_call_invalid; p->level--; return _res; } @@ -19142,149 +19132,10 @@ invalid_expression_rule(Parser *p) return _res; } -// invalid_left_assignment_prefixes: list | tuple | genexp | 'True' | 'None' | 'False' -static void * -invalid_left_assignment_prefixes_rule(Parser *p) -{ - if (p->level++ == MAXSTACK) { - p->error_indicator = 1; - PyErr_NoMemory(); - } - if (p->error_indicator) { - p->level--; - return NULL; - } - void * _res = NULL; - if (_PyPegen_is_memoized(p, invalid_left_assignment_prefixes_type, &_res)) { - p->level--; - return _res; - } - int _mark = p->mark; - { // list - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "list")); - expr_ty list_var; - if ( - (list_var = list_rule(p)) // list - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "list")); - _res = list_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "list")); - } - { // tuple - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "tuple")); - expr_ty tuple_var; - if ( - (tuple_var = tuple_rule(p)) // tuple - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "tuple")); - _res = tuple_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "tuple")); - } - { // genexp - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "genexp")); - expr_ty genexp_var; - if ( - (genexp_var = genexp_rule(p)) // genexp - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "genexp")); - _res = genexp_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "genexp")); - } - { // 'True' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 600)) // token='True' - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'True'")); - } - { // 'None' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 601)) // token='None' - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'None'")); - } - { // 'False' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 602)) // token='False' - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'False'")); - } - _res = NULL; - done: - _PyPegen_insert_memo(p, _mark, invalid_left_assignment_prefixes_type, _res); - p->level--; - return _res; -} - // invalid_named_expression: // | expression ':=' expression // | NAME '=' bitwise_or !('=' | ':=') -// | !invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=') +// | !(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=') static void * invalid_named_expression_rule(Parser *p) { @@ -19297,6 +19148,10 @@ invalid_named_expression_rule(Parser *p) return NULL; } void * _res = NULL; + if (_PyPegen_is_memoized(p, invalid_named_expression_type, &_res)) { + p->level--; + return _res; + } int _mark = p->mark; { // expression ':=' expression if (p->error_indicator) { @@ -19348,7 +19203,7 @@ invalid_named_expression_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "NAME '=' bitwise_or !('=' | ':=')")); - _res = p -> in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_RANGE ( a , b , "invalid syntax. Maybe you meant '==' or ':=' instead of '='?" ); + _res = RAISE_SYNTAX_ERROR_KNOWN_RANGE ( a , b , "invalid syntax. Maybe you meant '==' or ':=' instead of '='?" ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; p->level--; @@ -19360,17 +19215,17 @@ invalid_named_expression_rule(Parser *p) D(fprintf(stderr, "%*c%s invalid_named_expression[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "NAME '=' bitwise_or !('=' | ':=')")); } - { // !invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=') + { // !(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=') if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_named_expression[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); + D(fprintf(stderr, "%*c> invalid_named_expression[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); expr_ty a; Token * b; expr_ty bitwise_or_var; if ( - _PyPegen_lookahead(0, invalid_left_assignment_prefixes_rule, p) + _PyPegen_lookahead(0, _tmp_149_rule, p) && (a = bitwise_or_rule(p)) // bitwise_or && @@ -19378,11 +19233,11 @@ invalid_named_expression_rule(Parser *p) && (bitwise_or_var = bitwise_or_rule(p)) // bitwise_or && - _PyPegen_lookahead(0, _tmp_149_rule, p) + _PyPegen_lookahead(0, _tmp_150_rule, p) ) { - D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); - _res = p -> in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_LOCATION ( a , "cannot assign to %s here. Maybe you meant '==' instead of '='?" , _PyPegen_get_expr_name ( a ) ); + D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); + _res = RAISE_SYNTAX_ERROR_KNOWN_LOCATION ( a , "cannot assign to %s here. Maybe you meant '==' instead of '='?" , _PyPegen_get_expr_name ( a ) ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; p->level--; @@ -19392,10 +19247,11 @@ invalid_named_expression_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_named_expression[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); } _res = NULL; done: + _PyPegen_insert_memo(p, _mark, invalid_named_expression_type, _res); p->level--; return _res; } @@ -19458,7 +19314,7 @@ invalid_assignment_rule(Parser *p) D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expression ',' star_named_expressions* ':' expression")); Token * _literal; Token * _literal_1; - asdl_seq * _loop0_150_var; + asdl_seq * _loop0_151_var; expr_ty a; expr_ty expression_var; if ( @@ -19466,7 +19322,7 @@ invalid_assignment_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_loop0_150_var = _loop0_150_rule(p)) // star_named_expressions* + (_loop0_151_var = _loop0_151_rule(p)) // star_named_expressions* && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -19523,10 +19379,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* star_expressions '='")); Token * _literal; - asdl_seq * _loop0_151_var; + asdl_seq * _loop0_152_var; expr_ty a; if ( - (_loop0_151_var = _loop0_151_rule(p)) // ((star_targets '='))* + (_loop0_152_var = _loop0_152_rule(p)) // ((star_targets '='))* && (a = star_expressions_rule(p)) // star_expressions && @@ -19553,10 +19409,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* yield_expr '='")); Token * _literal; - asdl_seq * _loop0_152_var; + asdl_seq * _loop0_153_var; expr_ty a; if ( - (_loop0_152_var = _loop0_152_rule(p)) // ((star_targets '='))* + (_loop0_153_var = _loop0_153_rule(p)) // ((star_targets '='))* && (a = yield_expr_rule(p)) // yield_expr && @@ -19582,7 +19438,7 @@ invalid_assignment_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); - void *_tmp_153_var; + void *_tmp_154_var; expr_ty a; AugOperator* augassign_var; if ( @@ -19590,7 +19446,7 @@ invalid_assignment_rule(Parser *p) && (augassign_var = augassign_rule(p)) // augassign && - (_tmp_153_var = _tmp_153_rule(p)) // yield_expr | star_expressions + (_tmp_154_var = _tmp_154_rule(p)) // yield_expr | star_expressions ) { D(fprintf(stderr, "%*c+ invalid_assignment[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); @@ -19816,11 +19672,11 @@ invalid_comprehension_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '(' | '{') starred_expression for_if_clauses")); - void *_tmp_154_var; + void *_tmp_155_var; expr_ty a; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_154_var = _tmp_154_rule(p)) // '[' | '(' | '{' + (_tmp_155_var = _tmp_155_rule(p)) // '[' | '(' | '{' && (a = starred_expression_rule(p)) // starred_expression && @@ -19847,12 +19703,12 @@ invalid_comprehension_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '{') star_named_expression ',' star_named_expressions for_if_clauses")); Token * _literal; - void *_tmp_155_var; + void *_tmp_156_var; expr_ty a; asdl_expr_seq* b; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_155_var = _tmp_155_rule(p)) // '[' | '{' + (_tmp_156_var = _tmp_156_rule(p)) // '[' | '{' && (a = star_named_expression_rule(p)) // star_named_expression && @@ -19882,12 +19738,12 @@ invalid_comprehension_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '{') star_named_expression ',' for_if_clauses")); - void *_tmp_156_var; + void *_tmp_157_var; expr_ty a; Token * b; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_156_var = _tmp_156_rule(p)) // '[' | '{' + (_tmp_157_var = _tmp_157_rule(p)) // '[' | '{' && (a = star_named_expression_rule(p)) // star_named_expression && @@ -19993,11 +19849,11 @@ invalid_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default* invalid_parameters_helper param_no_default")); - asdl_seq * _loop0_157_var; + asdl_seq * _loop0_158_var; arg_ty a; void *invalid_parameters_helper_var; if ( - (_loop0_157_var = _loop0_157_rule(p)) // param_no_default* + (_loop0_158_var = _loop0_158_rule(p)) // param_no_default* && (invalid_parameters_helper_var = invalid_parameters_helper_rule(p)) // invalid_parameters_helper && @@ -20023,18 +19879,18 @@ invalid_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default* '(' param_no_default+ ','? ')'")); - asdl_seq * _loop0_158_var; - asdl_seq * _loop1_159_var; + asdl_seq * _loop0_159_var; + asdl_seq * _loop1_160_var; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings Token * a; Token * b; if ( - (_loop0_158_var = _loop0_158_rule(p)) // param_no_default* + (_loop0_159_var = _loop0_159_rule(p)) // param_no_default* && (a = _PyPegen_expect_token(p, 7)) // token='(' && - (_loop1_159_var = _loop1_159_rule(p)) // param_no_default+ + (_loop1_160_var = _loop1_160_rule(p)) // param_no_default+ && (_opt_var = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20104,13 +19960,13 @@ invalid_parameters_helper_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters_helper[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - asdl_seq * _loop1_160_var; + asdl_seq * _loop1_161_var; if ( - (_loop1_160_var = _loop1_160_rule(p)) // param_with_default+ + (_loop1_161_var = _loop1_161_rule(p)) // param_with_default+ ) { D(fprintf(stderr, "%*c+ invalid_parameters_helper[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - _res = _loop1_160_var; + _res = _loop1_161_var; goto done; } p->mark = _mark; @@ -20145,11 +20001,11 @@ invalid_lambda_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default* invalid_lambda_parameters_helper lambda_param_no_default")); - asdl_seq * _loop0_161_var; + asdl_seq * _loop0_162_var; arg_ty a; void *invalid_lambda_parameters_helper_var; if ( - (_loop0_161_var = _loop0_161_rule(p)) // lambda_param_no_default* + (_loop0_162_var = _loop0_162_rule(p)) // lambda_param_no_default* && (invalid_lambda_parameters_helper_var = invalid_lambda_parameters_helper_rule(p)) // invalid_lambda_parameters_helper && @@ -20175,18 +20031,18 @@ invalid_lambda_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default* '(' ','.lambda_param+ ','? ')'")); - asdl_seq * _gather_163_var; - asdl_seq * _loop0_162_var; + asdl_seq * _gather_164_var; + asdl_seq * _loop0_163_var; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings Token * a; Token * b; if ( - (_loop0_162_var = _loop0_162_rule(p)) // lambda_param_no_default* + (_loop0_163_var = _loop0_163_rule(p)) // lambda_param_no_default* && (a = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_163_var = _gather_163_rule(p)) // ','.lambda_param+ + (_gather_164_var = _gather_164_rule(p)) // ','.lambda_param+ && (_opt_var = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20258,13 +20114,13 @@ invalid_lambda_parameters_helper_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters_helper[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - asdl_seq * _loop1_165_var; + asdl_seq * _loop1_166_var; if ( - (_loop1_165_var = _loop1_165_rule(p)) // lambda_param_with_default+ + (_loop1_166_var = _loop1_166_rule(p)) // lambda_param_with_default+ ) { D(fprintf(stderr, "%*c+ invalid_lambda_parameters_helper[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - _res = _loop1_165_var; + _res = _loop1_166_var; goto done; } p->mark = _mark; @@ -20297,12 +20153,12 @@ invalid_star_etc_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); - void *_tmp_166_var; + void *_tmp_167_var; Token * a; if ( (a = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_166_var = _tmp_166_rule(p)) // ')' | ',' (')' | '**') + (_tmp_167_var = _tmp_167_rule(p)) // ')' | ',' (')' | '**') ) { D(fprintf(stderr, "%*c+ invalid_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); @@ -20375,11 +20231,11 @@ invalid_lambda_star_etc_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_lambda_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); Token * _literal; - void *_tmp_167_var; + void *_tmp_168_var; if ( (_literal = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_167_var = _tmp_167_rule(p)) // ':' | ',' (':' | '**') + (_tmp_168_var = _tmp_168_rule(p)) // ':' | ',' (':' | '**') ) { D(fprintf(stderr, "%*c+ invalid_lambda_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); @@ -20487,7 +20343,7 @@ invalid_with_item_rule(Parser *p) && (a = expression_rule(p)) // expression && - _PyPegen_lookahead(1, _tmp_168_rule, p) + _PyPegen_lookahead(1, _tmp_169_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_with_item[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression 'as' expression &(',' | ')' | ':')")); @@ -20715,7 +20571,7 @@ invalid_with_stmt_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ &&':'")); - asdl_seq * _gather_169_var; + asdl_seq * _gather_170_var; Token * _keyword; Token * _literal; void *_opt_var; @@ -20725,13 +20581,13 @@ invalid_with_stmt_rule(Parser *p) && (_keyword = _PyPegen_expect_token(p, 612)) // token='with' && - (_gather_169_var = _gather_169_rule(p)) // ','.(expression ['as' star_target])+ + (_gather_170_var = _gather_170_rule(p)) // ','.(expression ['as' star_target])+ && (_literal = _PyPegen_expect_forced_token(p, 11, ":")) // forced_token=':' ) { D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ &&':'")); - _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _gather_169_var, _literal); + _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _gather_170_var, _literal); goto done; } p->mark = _mark; @@ -20744,7 +20600,7 @@ invalid_with_stmt_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' &&':'")); - asdl_seq * _gather_171_var; + asdl_seq * _gather_172_var; Token * _keyword; Token * _literal; Token * _literal_1; @@ -20760,7 +20616,7 @@ invalid_with_stmt_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_171_var = _gather_171_rule(p)) // ','.(expressions ['as' star_target])+ + (_gather_172_var = _gather_172_rule(p)) // ','.(expressions ['as' star_target])+ && (_opt_var_1 = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20770,7 +20626,7 @@ invalid_with_stmt_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' &&':'")); - _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _literal, _gather_171_var, _opt_var_1, _literal_1, _literal_2); + _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _literal, _gather_172_var, _opt_var_1, _literal_1, _literal_2); goto done; } p->mark = _mark; @@ -20805,7 +20661,7 @@ invalid_with_stmt_indent_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); - asdl_seq * _gather_173_var; + asdl_seq * _gather_174_var; Token * _literal; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings @@ -20816,7 +20672,7 @@ invalid_with_stmt_indent_rule(Parser *p) && (a = _PyPegen_expect_token(p, 612)) // token='with' && - (_gather_173_var = _gather_173_rule(p)) // ','.(expression ['as' star_target])+ + (_gather_174_var = _gather_174_rule(p)) // ','.(expression ['as' star_target])+ && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -20844,7 +20700,7 @@ invalid_with_stmt_indent_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); - asdl_seq * _gather_175_var; + asdl_seq * _gather_176_var; Token * _literal; Token * _literal_1; Token * _literal_2; @@ -20861,7 +20717,7 @@ invalid_with_stmt_indent_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_175_var = _gather_175_rule(p)) // ','.(expressions ['as' star_target])+ + (_gather_176_var = _gather_176_rule(p)) // ','.(expressions ['as' star_target])+ && (_opt_var_1 = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20958,7 +20814,7 @@ invalid_try_stmt_rule(Parser *p) && (block_var = block_rule(p)) // block && - _PyPegen_lookahead(0, _tmp_177_rule, p) + _PyPegen_lookahead(0, _tmp_178_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_try_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'try' ':' block !('except' | 'finally')")); @@ -20982,19 +20838,19 @@ invalid_try_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> invalid_try_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'try' ':' block* ((except_block+ except_star_block) | (except_star_block+ except_block)) block*")); Token * _keyword; Token * _literal; - asdl_seq * _loop0_178_var; - asdl_seq * _loop0_180_var; - void *_tmp_179_var; + asdl_seq * _loop0_179_var; + asdl_seq * _loop0_181_var; + void *_tmp_180_var; if ( (_keyword = _PyPegen_expect_token(p, 618)) // token='try' && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && - (_loop0_178_var = _loop0_178_rule(p)) // block* + (_loop0_179_var = _loop0_179_rule(p)) // block* && - (_tmp_179_var = _tmp_179_rule(p)) // (except_block+ except_star_block) | (except_star_block+ except_block) + (_tmp_180_var = _tmp_180_rule(p)) // (except_block+ except_star_block) | (except_star_block+ except_block) && - (_loop0_180_var = _loop0_180_rule(p)) // block* + (_loop0_181_var = _loop0_181_rule(p)) // block* ) { D(fprintf(stderr, "%*c+ invalid_try_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'try' ':' block* ((except_block+ except_star_block) | (except_star_block+ except_block)) block*")); @@ -21060,7 +20916,7 @@ invalid_except_stmt_rule(Parser *p) && (expressions_var = expressions_rule(p)) // expressions && - (_opt_var_1 = _tmp_181_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var_1 = _tmp_182_rule(p), !p->error_indicator) // ['as' NAME] && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' ) @@ -21098,7 +20954,7 @@ invalid_except_stmt_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var_1 = _tmp_182_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var_1 = _tmp_183_rule(p), !p->error_indicator) // ['as' NAME] && (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) @@ -21150,14 +21006,14 @@ invalid_except_stmt_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_except_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except' '*' (NEWLINE | ':')")); Token * _literal; - void *_tmp_183_var; + void *_tmp_184_var; Token * a; if ( (a = _PyPegen_expect_token(p, 629)) // token='except' && (_literal = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_183_var = _tmp_183_rule(p)) // NEWLINE | ':' + (_tmp_184_var = _tmp_184_rule(p)) // NEWLINE | ':' ) { D(fprintf(stderr, "%*c+ invalid_except_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except' '*' (NEWLINE | ':')")); @@ -21264,7 +21120,7 @@ invalid_except_stmt_indent_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_184_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_185_rule(p), !p->error_indicator) // ['as' NAME] && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21359,7 +21215,7 @@ invalid_except_star_stmt_indent_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_185_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_186_rule(p), !p->error_indicator) // ['as' NAME] && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21726,7 +21582,7 @@ invalid_class_argument_pattern_rule(Parser *p) asdl_pattern_seq* a; asdl_seq* keyword_patterns_var; if ( - (_opt_var = _tmp_186_rule(p), !p->error_indicator) // [positional_patterns ','] + (_opt_var = _tmp_187_rule(p), !p->error_indicator) // [positional_patterns ','] && (keyword_patterns_var = keyword_patterns_rule(p)) // keyword_patterns && @@ -22178,7 +22034,7 @@ invalid_def_raw_rule(Parser *p) && (_literal_1 = _PyPegen_expect_token(p, 8)) // token=')' && - (_opt_var_2 = _tmp_187_rule(p), !p->error_indicator) // ['->' expression] + (_opt_var_2 = _tmp_188_rule(p), !p->error_indicator) // ['->' expression] && (_literal_2 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -22237,7 +22093,7 @@ invalid_class_def_raw_rule(Parser *p) && (name_var = _PyPegen_name_token(p)) // NAME && - (_opt_var = _tmp_188_rule(p), !p->error_indicator) // ['(' arguments? ')'] + (_opt_var = _tmp_189_rule(p), !p->error_indicator) // ['(' arguments? ')'] && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -22288,11 +22144,11 @@ invalid_double_starred_kvpairs_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_double_starred_kvpairs[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.double_starred_kvpair+ ',' invalid_kvpair")); - asdl_seq * _gather_189_var; + asdl_seq * _gather_190_var; Token * _literal; void *invalid_kvpair_var; if ( - (_gather_189_var = _gather_189_rule(p)) // ','.double_starred_kvpair+ + (_gather_190_var = _gather_190_rule(p)) // ','.double_starred_kvpair+ && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && @@ -22300,7 +22156,7 @@ invalid_double_starred_kvpairs_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_double_starred_kvpairs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.double_starred_kvpair+ ',' invalid_kvpair")); - _res = _PyPegen_dummy_name(p, _gather_189_var, _literal, invalid_kvpair_var); + _res = _PyPegen_dummy_name(p, _gather_190_var, _literal, invalid_kvpair_var); goto done; } p->mark = _mark; @@ -22353,7 +22209,7 @@ invalid_double_starred_kvpairs_rule(Parser *p) && (a = _PyPegen_expect_token(p, 11)) // token=':' && - _PyPegen_lookahead(1, _tmp_191_rule, p) + _PyPegen_lookahead(1, _tmp_192_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_double_starred_kvpairs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ':' &('}' | ',')")); @@ -23315,12 +23171,12 @@ _loop1_14_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_14[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_192_var; + void *_tmp_193_var; while ( - (_tmp_192_var = _tmp_192_rule(p)) // star_targets '=' + (_tmp_193_var = _tmp_193_rule(p)) // star_targets '=' ) { - _res = _tmp_192_var; + _res = _tmp_193_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23897,12 +23753,12 @@ _loop0_24_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_24[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_193_var; + void *_tmp_194_var; while ( - (_tmp_193_var = _tmp_193_rule(p)) // '.' | '...' + (_tmp_194_var = _tmp_194_rule(p)) // '.' | '...' ) { - _res = _tmp_193_var; + _res = _tmp_194_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23966,12 +23822,12 @@ _loop1_25_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_25[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_194_var; + void *_tmp_195_var; while ( - (_tmp_194_var = _tmp_194_rule(p)) // '.' | '...' + (_tmp_195_var = _tmp_195_rule(p)) // '.' | '...' ) { - _res = _tmp_194_var; + _res = _tmp_195_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -24374,12 +24230,12 @@ _loop1_32_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_32[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('@' named_expression NEWLINE)")); - void *_tmp_195_var; + void *_tmp_196_var; while ( - (_tmp_195_var = _tmp_195_rule(p)) // '@' named_expression NEWLINE + (_tmp_196_var = _tmp_196_rule(p)) // '@' named_expression NEWLINE ) { - _res = _tmp_195_var; + _res = _tmp_196_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27394,12 +27250,12 @@ _loop1_79_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_79[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' expression)")); - void *_tmp_196_var; + void *_tmp_197_var; while ( - (_tmp_196_var = _tmp_196_rule(p)) // ',' expression + (_tmp_197_var = _tmp_197_rule(p)) // ',' expression ) { - _res = _tmp_196_var; + _res = _tmp_197_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27468,12 +27324,12 @@ _loop1_80_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_80[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_expression)")); - void *_tmp_197_var; + void *_tmp_198_var; while ( - (_tmp_197_var = _tmp_197_rule(p)) // ',' star_expression + (_tmp_198_var = _tmp_198_rule(p)) // ',' star_expression ) { - _res = _tmp_197_var; + _res = _tmp_198_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27662,12 +27518,12 @@ _loop1_83_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_83[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('or' conjunction)")); - void *_tmp_198_var; + void *_tmp_199_var; while ( - (_tmp_198_var = _tmp_198_rule(p)) // 'or' conjunction + (_tmp_199_var = _tmp_199_rule(p)) // 'or' conjunction ) { - _res = _tmp_198_var; + _res = _tmp_199_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27736,12 +27592,12 @@ _loop1_84_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_84[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('and' inversion)")); - void *_tmp_199_var; + void *_tmp_200_var; while ( - (_tmp_199_var = _tmp_199_rule(p)) // 'and' inversion + (_tmp_200_var = _tmp_200_rule(p)) // 'and' inversion ) { - _res = _tmp_199_var; + _res = _tmp_200_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29703,12 +29559,12 @@ _loop0_113_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_113[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_200_var; + void *_tmp_201_var; while ( - (_tmp_200_var = _tmp_200_rule(p)) // 'if' disjunction + (_tmp_201_var = _tmp_201_rule(p)) // 'if' disjunction ) { - _res = _tmp_200_var; + _res = _tmp_201_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29772,12 +29628,12 @@ _loop0_114_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_114[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_201_var; + void *_tmp_202_var; while ( - (_tmp_201_var = _tmp_201_rule(p)) // 'if' disjunction + (_tmp_202_var = _tmp_202_rule(p)) // 'if' disjunction ) { - _res = _tmp_201_var; + _res = _tmp_202_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29906,7 +29762,7 @@ _loop0_117_rule(Parser *p) while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_202_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' + (elem = _tmp_203_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' ) { _res = elem; @@ -29973,7 +29829,7 @@ _gather_116_rule(Parser *p) void *elem; asdl_seq * seq; if ( - (elem = _tmp_202_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' + (elem = _tmp_203_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' && (seq = _loop0_117_rule(p)) // _loop0_117 ) @@ -30549,12 +30405,12 @@ _loop0_127_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_127[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); - void *_tmp_203_var; + void *_tmp_204_var; while ( - (_tmp_203_var = _tmp_203_rule(p)) // ',' star_target + (_tmp_204_var = _tmp_204_rule(p)) // ',' star_target ) { - _res = _tmp_203_var; + _res = _tmp_204_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -30738,12 +30594,12 @@ _loop1_130_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_130[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); - void *_tmp_204_var; + void *_tmp_205_var; while ( - (_tmp_204_var = _tmp_204_rule(p)) // ',' star_target + (_tmp_205_var = _tmp_205_rule(p)) // ',' star_target ) { - _res = _tmp_204_var; + _res = _tmp_205_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -31822,9 +31678,143 @@ _tmp_148_rule(Parser *p) return _res; } -// _tmp_149: '=' | ':=' +// _tmp_149: list | tuple | genexp | 'True' | 'None' | 'False' static void * _tmp_149_rule(Parser *p) +{ + if (p->level++ == MAXSTACK) { + p->error_indicator = 1; + PyErr_NoMemory(); + } + if (p->error_indicator) { + p->level--; + return NULL; + } + void * _res = NULL; + int _mark = p->mark; + { // list + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "list")); + expr_ty list_var; + if ( + (list_var = list_rule(p)) // list + ) + { + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "list")); + _res = list_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "list")); + } + { // tuple + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "tuple")); + expr_ty tuple_var; + if ( + (tuple_var = tuple_rule(p)) // tuple + ) + { + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "tuple")); + _res = tuple_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "tuple")); + } + { // genexp + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "genexp")); + expr_ty genexp_var; + if ( + (genexp_var = genexp_rule(p)) // genexp + ) + { + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "genexp")); + _res = genexp_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "genexp")); + } + { // 'True' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 600)) // token='True' + ) + { + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'True'")); + } + { // 'None' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 601)) // token='None' + ) + { + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'None'")); + } + { // 'False' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 602)) // token='False' + ) + { + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'False'")); + } + _res = NULL; + done: + p->level--; + return _res; +} + +// _tmp_150: '=' | ':=' +static void * +_tmp_150_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31841,18 +31831,18 @@ _tmp_149_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'='")); + D(fprintf(stderr, "%*c> _tmp_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'='")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'='")); + D(fprintf(stderr, "%*c+ _tmp_150[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'='")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_150[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'='")); } { // ':=' @@ -31860,18 +31850,18 @@ _tmp_149_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':='")); + D(fprintf(stderr, "%*c> _tmp_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':='")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 53)) // token=':=' ) { - D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':='")); + D(fprintf(stderr, "%*c+ _tmp_150[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':='")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_150[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':='")); } _res = NULL; @@ -31880,9 +31870,9 @@ _tmp_149_rule(Parser *p) return _res; } -// _loop0_150: star_named_expressions +// _loop0_151: star_named_expressions static asdl_seq * -_loop0_150_rule(Parser *p) +_loop0_151_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31909,7 +31899,7 @@ _loop0_150_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); + D(fprintf(stderr, "%*c> _loop0_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); asdl_expr_seq* star_named_expressions_var; while ( (star_named_expressions_var = star_named_expressions_rule(p)) // star_named_expressions @@ -31931,7 +31921,7 @@ _loop0_150_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_150[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_151[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_named_expressions")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31944,14 +31934,14 @@ _loop0_150_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_150_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_151_type, _seq); p->level--; return _seq; } -// _loop0_151: (star_targets '=') +// _loop0_152: (star_targets '=') static asdl_seq * -_loop0_151_rule(Parser *p) +_loop0_152_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31978,13 +31968,13 @@ _loop0_151_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_205_var; + D(fprintf(stderr, "%*c> _loop0_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_206_var; while ( - (_tmp_205_var = _tmp_205_rule(p)) // star_targets '=' + (_tmp_206_var = _tmp_206_rule(p)) // star_targets '=' ) { - _res = _tmp_205_var; + _res = _tmp_206_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -32000,7 +31990,7 @@ _loop0_151_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_151[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_152[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32013,14 +32003,14 @@ _loop0_151_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_151_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_152_type, _seq); p->level--; return _seq; } -// _loop0_152: (star_targets '=') +// _loop0_153: (star_targets '=') static asdl_seq * -_loop0_152_rule(Parser *p) +_loop0_153_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32047,13 +32037,13 @@ _loop0_152_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_206_var; + D(fprintf(stderr, "%*c> _loop0_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_207_var; while ( - (_tmp_206_var = _tmp_206_rule(p)) // star_targets '=' + (_tmp_207_var = _tmp_207_rule(p)) // star_targets '=' ) { - _res = _tmp_206_var; + _res = _tmp_207_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -32069,7 +32059,7 @@ _loop0_152_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_152[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_153[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32082,14 +32072,14 @@ _loop0_152_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_152_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_153_type, _seq); p->level--; return _seq; } -// _tmp_153: yield_expr | star_expressions +// _tmp_154: yield_expr | star_expressions static void * -_tmp_153_rule(Parser *p) +_tmp_154_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32106,18 +32096,18 @@ _tmp_153_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); expr_ty yield_expr_var; if ( (yield_expr_var = yield_expr_rule(p)) // yield_expr ) { - D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); _res = yield_expr_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "yield_expr")); } { // star_expressions @@ -32125,18 +32115,18 @@ _tmp_153_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); expr_ty star_expressions_var; if ( (star_expressions_var = star_expressions_rule(p)) // star_expressions ) { - D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); _res = star_expressions_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_expressions")); } _res = NULL; @@ -32145,9 +32135,9 @@ _tmp_153_rule(Parser *p) return _res; } -// _tmp_154: '[' | '(' | '{' +// _tmp_155: '[' | '(' | '{' static void * -_tmp_154_rule(Parser *p) +_tmp_155_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32164,18 +32154,18 @@ _tmp_154_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '(' @@ -32183,18 +32173,18 @@ _tmp_154_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 7)) // token='(' ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'('")); } { // '{' @@ -32202,18 +32192,18 @@ _tmp_154_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -32222,9 +32212,9 @@ _tmp_154_rule(Parser *p) return _res; } -// _tmp_155: '[' | '{' +// _tmp_156: '[' | '{' static void * -_tmp_155_rule(Parser *p) +_tmp_156_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32241,18 +32231,18 @@ _tmp_155_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '{' @@ -32260,18 +32250,18 @@ _tmp_155_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -32280,9 +32270,9 @@ _tmp_155_rule(Parser *p) return _res; } -// _tmp_156: '[' | '{' +// _tmp_157: '[' | '{' static void * -_tmp_156_rule(Parser *p) +_tmp_157_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32299,18 +32289,18 @@ _tmp_156_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_157[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_157[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '{' @@ -32318,18 +32308,18 @@ _tmp_156_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_157[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_157[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -32338,9 +32328,9 @@ _tmp_156_rule(Parser *p) return _res; } -// _loop0_157: param_no_default +// _loop0_158: param_no_default static asdl_seq * -_loop0_157_rule(Parser *p) +_loop0_158_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32367,7 +32357,7 @@ _loop0_157_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); + D(fprintf(stderr, "%*c> _loop0_158[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); arg_ty param_no_default_var; while ( (param_no_default_var = param_no_default_rule(p)) // param_no_default @@ -32389,7 +32379,7 @@ _loop0_157_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_157[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_158[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32402,14 +32392,14 @@ _loop0_157_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_157_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_158_type, _seq); p->level--; return _seq; } -// _loop0_158: param_no_default +// _loop0_159: param_no_default static asdl_seq * -_loop0_158_rule(Parser *p) +_loop0_159_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32436,7 +32426,7 @@ _loop0_158_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_158[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); + D(fprintf(stderr, "%*c> _loop0_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); arg_ty param_no_default_var; while ( (param_no_default_var = param_no_default_rule(p)) // param_no_default @@ -32458,7 +32448,7 @@ _loop0_158_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_158[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_159[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32471,14 +32461,14 @@ _loop0_158_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_158_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_159_type, _seq); p->level--; return _seq; } -// _loop1_159: param_no_default +// _loop1_160: param_no_default static asdl_seq * -_loop1_159_rule(Parser *p) +_loop1_160_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32505,7 +32495,7 @@ _loop1_159_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); + D(fprintf(stderr, "%*c> _loop1_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); arg_ty param_no_default_var; while ( (param_no_default_var = param_no_default_rule(p)) // param_no_default @@ -32527,7 +32517,7 @@ _loop1_159_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_159[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_160[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } if (_n == 0 || p->error_indicator) { @@ -32545,14 +32535,14 @@ _loop1_159_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_159_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_160_type, _seq); p->level--; return _seq; } -// _loop1_160: param_with_default +// _loop1_161: param_with_default static asdl_seq * -_loop1_160_rule(Parser *p) +_loop1_161_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32579,7 +32569,7 @@ _loop1_160_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); + D(fprintf(stderr, "%*c> _loop1_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); NameDefaultPair* param_with_default_var; while ( (param_with_default_var = param_with_default_rule(p)) // param_with_default @@ -32601,7 +32591,7 @@ _loop1_160_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_160[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_161[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -32619,14 +32609,14 @@ _loop1_160_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_160_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_161_type, _seq); p->level--; return _seq; } -// _loop0_161: lambda_param_no_default +// _loop0_162: lambda_param_no_default static asdl_seq * -_loop0_161_rule(Parser *p) +_loop0_162_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32653,7 +32643,7 @@ _loop0_161_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); + D(fprintf(stderr, "%*c> _loop0_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); arg_ty lambda_param_no_default_var; while ( (lambda_param_no_default_var = lambda_param_no_default_rule(p)) // lambda_param_no_default @@ -32675,7 +32665,7 @@ _loop0_161_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_161[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_162[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32688,14 +32678,14 @@ _loop0_161_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_161_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_162_type, _seq); p->level--; return _seq; } -// _loop0_162: lambda_param_no_default +// _loop0_163: lambda_param_no_default static asdl_seq * -_loop0_162_rule(Parser *p) +_loop0_163_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32722,7 +32712,7 @@ _loop0_162_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); + D(fprintf(stderr, "%*c> _loop0_163[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); arg_ty lambda_param_no_default_var; while ( (lambda_param_no_default_var = lambda_param_no_default_rule(p)) // lambda_param_no_default @@ -32744,7 +32734,7 @@ _loop0_162_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_162[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_163[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32757,14 +32747,14 @@ _loop0_162_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_162_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_163_type, _seq); p->level--; return _seq; } -// _loop0_164: ',' lambda_param +// _loop0_165: ',' lambda_param static asdl_seq * -_loop0_164_rule(Parser *p) +_loop0_165_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32791,7 +32781,7 @@ _loop0_164_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_164[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' lambda_param")); + D(fprintf(stderr, "%*c> _loop0_165[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' lambda_param")); Token * _literal; arg_ty elem; while ( @@ -32822,7 +32812,7 @@ _loop0_164_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_164[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_165[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' lambda_param")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32835,14 +32825,14 @@ _loop0_164_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_164_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_165_type, _seq); p->level--; return _seq; } -// _gather_163: lambda_param _loop0_164 +// _gather_164: lambda_param _loop0_165 static asdl_seq * -_gather_163_rule(Parser *p) +_gather_164_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32854,27 +32844,27 @@ _gather_163_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // lambda_param _loop0_164 + { // lambda_param _loop0_165 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_163[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param _loop0_164")); + D(fprintf(stderr, "%*c> _gather_164[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param _loop0_165")); arg_ty elem; asdl_seq * seq; if ( (elem = lambda_param_rule(p)) // lambda_param && - (seq = _loop0_164_rule(p)) // _loop0_164 + (seq = _loop0_165_rule(p)) // _loop0_165 ) { - D(fprintf(stderr, "%*c+ _gather_163[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param _loop0_164")); + D(fprintf(stderr, "%*c+ _gather_164[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param _loop0_165")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_163[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param _loop0_164")); + D(fprintf(stderr, "%*c%s _gather_164[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param _loop0_165")); } _res = NULL; done: @@ -32882,9 +32872,9 @@ _gather_163_rule(Parser *p) return _res; } -// _loop1_165: lambda_param_with_default +// _loop1_166: lambda_param_with_default static asdl_seq * -_loop1_165_rule(Parser *p) +_loop1_166_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32911,7 +32901,7 @@ _loop1_165_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_165[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); + D(fprintf(stderr, "%*c> _loop1_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); NameDefaultPair* lambda_param_with_default_var; while ( (lambda_param_with_default_var = lambda_param_with_default_rule(p)) // lambda_param_with_default @@ -32933,7 +32923,7 @@ _loop1_165_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_165[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_166[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -32951,14 +32941,14 @@ _loop1_165_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_165_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_166_type, _seq); p->level--; return _seq; } -// _tmp_166: ')' | ',' (')' | '**') +// _tmp_167: ')' | ',' (')' | '**') static void * -_tmp_166_rule(Parser *p) +_tmp_167_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32975,18 +32965,18 @@ _tmp_166_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_166[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_166[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_167[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // ',' (')' | '**') @@ -32994,21 +32984,21 @@ _tmp_166_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + D(fprintf(stderr, "%*c> _tmp_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); Token * _literal; - void *_tmp_207_var; + void *_tmp_208_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_207_var = _tmp_207_rule(p)) // ')' | '**' + (_tmp_208_var = _tmp_208_rule(p)) // ')' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_166[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_207_var); + D(fprintf(stderr, "%*c+ _tmp_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_208_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_166[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_167[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (')' | '**')")); } _res = NULL; @@ -33017,9 +33007,9 @@ _tmp_166_rule(Parser *p) return _res; } -// _tmp_167: ':' | ',' (':' | '**') +// _tmp_168: ':' | ',' (':' | '**') static void * -_tmp_167_rule(Parser *p) +_tmp_168_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33036,18 +33026,18 @@ _tmp_167_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_167[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // ',' (':' | '**') @@ -33055,21 +33045,21 @@ _tmp_167_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); Token * _literal; - void *_tmp_208_var; + void *_tmp_209_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_208_var = _tmp_208_rule(p)) // ':' | '**' + (_tmp_209_var = _tmp_209_rule(p)) // ':' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_208_var); + D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_209_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_167[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (':' | '**')")); } _res = NULL; @@ -33078,9 +33068,9 @@ _tmp_167_rule(Parser *p) return _res; } -// _tmp_168: ',' | ')' | ':' +// _tmp_169: ',' | ')' | ':' static void * -_tmp_168_rule(Parser *p) +_tmp_169_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33097,18 +33087,18 @@ _tmp_168_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c> _tmp_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c+ _tmp_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_169[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "','")); } { // ')' @@ -33116,18 +33106,18 @@ _tmp_168_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_169[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // ':' @@ -33135,18 +33125,18 @@ _tmp_168_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_168[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_169[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } _res = NULL; @@ -33155,9 +33145,9 @@ _tmp_168_rule(Parser *p) return _res; } -// _loop0_170: ',' (expression ['as' star_target]) +// _loop0_171: ',' (expression ['as' star_target]) static asdl_seq * -_loop0_170_rule(Parser *p) +_loop0_171_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33184,13 +33174,13 @@ _loop0_170_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_209_rule(p)) // expression ['as' star_target] + (elem = _tmp_210_rule(p)) // expression ['as' star_target] ) { _res = elem; @@ -33215,7 +33205,7 @@ _loop0_170_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_170[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_171[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expression ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33228,14 +33218,14 @@ _loop0_170_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_170_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_171_type, _seq); p->level--; return _seq; } -// _gather_169: (expression ['as' star_target]) _loop0_170 +// _gather_170: (expression ['as' star_target]) _loop0_171 static asdl_seq * -_gather_169_rule(Parser *p) +_gather_170_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33247,27 +33237,27 @@ _gather_169_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expression ['as' star_target]) _loop0_170 + { // (expression ['as' star_target]) _loop0_171 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_170")); + D(fprintf(stderr, "%*c> _gather_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_171")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_209_rule(p)) // expression ['as' star_target] + (elem = _tmp_210_rule(p)) // expression ['as' star_target] && - (seq = _loop0_170_rule(p)) // _loop0_170 + (seq = _loop0_171_rule(p)) // _loop0_171 ) { - D(fprintf(stderr, "%*c+ _gather_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_170")); + D(fprintf(stderr, "%*c+ _gather_170[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_171")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_169[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_170")); + D(fprintf(stderr, "%*c%s _gather_170[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_171")); } _res = NULL; done: @@ -33275,9 +33265,9 @@ _gather_169_rule(Parser *p) return _res; } -// _loop0_172: ',' (expressions ['as' star_target]) +// _loop0_173: ',' (expressions ['as' star_target]) static asdl_seq * -_loop0_172_rule(Parser *p) +_loop0_173_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33304,13 +33294,13 @@ _loop0_172_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_172[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_173[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_210_rule(p)) // expressions ['as' star_target] + (elem = _tmp_211_rule(p)) // expressions ['as' star_target] ) { _res = elem; @@ -33335,7 +33325,7 @@ _loop0_172_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_172[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_173[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expressions ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33348,14 +33338,14 @@ _loop0_172_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_172_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_173_type, _seq); p->level--; return _seq; } -// _gather_171: (expressions ['as' star_target]) _loop0_172 +// _gather_172: (expressions ['as' star_target]) _loop0_173 static asdl_seq * -_gather_171_rule(Parser *p) +_gather_172_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33367,27 +33357,27 @@ _gather_171_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expressions ['as' star_target]) _loop0_172 + { // (expressions ['as' star_target]) _loop0_173 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_172")); + D(fprintf(stderr, "%*c> _gather_172[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_173")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_210_rule(p)) // expressions ['as' star_target] + (elem = _tmp_211_rule(p)) // expressions ['as' star_target] && - (seq = _loop0_172_rule(p)) // _loop0_172 + (seq = _loop0_173_rule(p)) // _loop0_173 ) { - D(fprintf(stderr, "%*c+ _gather_171[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_172")); + D(fprintf(stderr, "%*c+ _gather_172[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_173")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_171[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_172")); + D(fprintf(stderr, "%*c%s _gather_172[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_173")); } _res = NULL; done: @@ -33395,9 +33385,9 @@ _gather_171_rule(Parser *p) return _res; } -// _loop0_174: ',' (expression ['as' star_target]) +// _loop0_175: ',' (expression ['as' star_target]) static asdl_seq * -_loop0_174_rule(Parser *p) +_loop0_175_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33424,13 +33414,13 @@ _loop0_174_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_174[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_175[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_211_rule(p)) // expression ['as' star_target] + (elem = _tmp_212_rule(p)) // expression ['as' star_target] ) { _res = elem; @@ -33455,7 +33445,7 @@ _loop0_174_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_174[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_175[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expression ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33468,14 +33458,14 @@ _loop0_174_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_174_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_175_type, _seq); p->level--; return _seq; } -// _gather_173: (expression ['as' star_target]) _loop0_174 +// _gather_174: (expression ['as' star_target]) _loop0_175 static asdl_seq * -_gather_173_rule(Parser *p) +_gather_174_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33487,27 +33477,27 @@ _gather_173_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expression ['as' star_target]) _loop0_174 + { // (expression ['as' star_target]) _loop0_175 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_173[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_174")); + D(fprintf(stderr, "%*c> _gather_174[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_175")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_211_rule(p)) // expression ['as' star_target] + (elem = _tmp_212_rule(p)) // expression ['as' star_target] && - (seq = _loop0_174_rule(p)) // _loop0_174 + (seq = _loop0_175_rule(p)) // _loop0_175 ) { - D(fprintf(stderr, "%*c+ _gather_173[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_174")); + D(fprintf(stderr, "%*c+ _gather_174[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_175")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_173[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_174")); + D(fprintf(stderr, "%*c%s _gather_174[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_175")); } _res = NULL; done: @@ -33515,9 +33505,9 @@ _gather_173_rule(Parser *p) return _res; } -// _loop0_176: ',' (expressions ['as' star_target]) +// _loop0_177: ',' (expressions ['as' star_target]) static asdl_seq * -_loop0_176_rule(Parser *p) +_loop0_177_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33544,13 +33534,13 @@ _loop0_176_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_176[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_212_rule(p)) // expressions ['as' star_target] + (elem = _tmp_213_rule(p)) // expressions ['as' star_target] ) { _res = elem; @@ -33575,7 +33565,7 @@ _loop0_176_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_176[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_177[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expressions ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33588,14 +33578,14 @@ _loop0_176_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_176_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_177_type, _seq); p->level--; return _seq; } -// _gather_175: (expressions ['as' star_target]) _loop0_176 +// _gather_176: (expressions ['as' star_target]) _loop0_177 static asdl_seq * -_gather_175_rule(Parser *p) +_gather_176_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33607,27 +33597,27 @@ _gather_175_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expressions ['as' star_target]) _loop0_176 + { // (expressions ['as' star_target]) _loop0_177 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_175[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_176")); + D(fprintf(stderr, "%*c> _gather_176[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_177")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_212_rule(p)) // expressions ['as' star_target] + (elem = _tmp_213_rule(p)) // expressions ['as' star_target] && - (seq = _loop0_176_rule(p)) // _loop0_176 + (seq = _loop0_177_rule(p)) // _loop0_177 ) { - D(fprintf(stderr, "%*c+ _gather_175[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_176")); + D(fprintf(stderr, "%*c+ _gather_176[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_177")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_175[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_176")); + D(fprintf(stderr, "%*c%s _gather_176[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_177")); } _res = NULL; done: @@ -33635,9 +33625,9 @@ _gather_175_rule(Parser *p) return _res; } -// _tmp_177: 'except' | 'finally' +// _tmp_178: 'except' | 'finally' static void * -_tmp_177_rule(Parser *p) +_tmp_178_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33654,18 +33644,18 @@ _tmp_177_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except'")); + D(fprintf(stderr, "%*c> _tmp_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except'")); Token * _keyword; if ( (_keyword = _PyPegen_expect_token(p, 629)) // token='except' ) { - D(fprintf(stderr, "%*c+ _tmp_177[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except'")); + D(fprintf(stderr, "%*c+ _tmp_178[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except'")); _res = _keyword; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_177[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_178[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'except'")); } { // 'finally' @@ -33673,18 +33663,18 @@ _tmp_177_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally'")); + D(fprintf(stderr, "%*c> _tmp_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally'")); Token * _keyword; if ( (_keyword = _PyPegen_expect_token(p, 625)) // token='finally' ) { - D(fprintf(stderr, "%*c+ _tmp_177[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'finally'")); + D(fprintf(stderr, "%*c+ _tmp_178[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'finally'")); _res = _keyword; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_177[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_178[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'finally'")); } _res = NULL; @@ -33693,9 +33683,9 @@ _tmp_177_rule(Parser *p) return _res; } -// _loop0_178: block +// _loop0_179: block static asdl_seq * -_loop0_178_rule(Parser *p) +_loop0_179_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33722,7 +33712,7 @@ _loop0_178_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "block")); + D(fprintf(stderr, "%*c> _loop0_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "block")); asdl_stmt_seq* block_var; while ( (block_var = block_rule(p)) // block @@ -33744,7 +33734,7 @@ _loop0_178_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_178[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_179[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "block")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33757,14 +33747,14 @@ _loop0_178_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_178_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_179_type, _seq); p->level--; return _seq; } -// _tmp_179: (except_block+ except_star_block) | (except_star_block+ except_block) +// _tmp_180: (except_block+ except_star_block) | (except_star_block+ except_block) static void * -_tmp_179_rule(Parser *p) +_tmp_180_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33781,18 +33771,18 @@ _tmp_179_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(except_block+ except_star_block)")); - void *_tmp_213_var; + D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(except_block+ except_star_block)")); + void *_tmp_214_var; if ( - (_tmp_213_var = _tmp_213_rule(p)) // except_block+ except_star_block + (_tmp_214_var = _tmp_214_rule(p)) // except_block+ except_star_block ) { - D(fprintf(stderr, "%*c+ _tmp_179[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(except_block+ except_star_block)")); - _res = _tmp_213_var; + D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(except_block+ except_star_block)")); + _res = _tmp_214_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_179[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(except_block+ except_star_block)")); } { // (except_star_block+ except_block) @@ -33800,18 +33790,18 @@ _tmp_179_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(except_star_block+ except_block)")); - void *_tmp_214_var; + D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(except_star_block+ except_block)")); + void *_tmp_215_var; if ( - (_tmp_214_var = _tmp_214_rule(p)) // except_star_block+ except_block + (_tmp_215_var = _tmp_215_rule(p)) // except_star_block+ except_block ) { - D(fprintf(stderr, "%*c+ _tmp_179[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(except_star_block+ except_block)")); - _res = _tmp_214_var; + D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(except_star_block+ except_block)")); + _res = _tmp_215_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_179[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(except_star_block+ except_block)")); } _res = NULL; @@ -33820,9 +33810,9 @@ _tmp_179_rule(Parser *p) return _res; } -// _loop0_180: block +// _loop0_181: block static asdl_seq * -_loop0_180_rule(Parser *p) +_loop0_181_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33849,7 +33839,7 @@ _loop0_180_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "block")); + D(fprintf(stderr, "%*c> _loop0_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "block")); asdl_stmt_seq* block_var; while ( (block_var = block_rule(p)) // block @@ -33871,7 +33861,7 @@ _loop0_180_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_180[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_181[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "block")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33884,53 +33874,11 @@ _loop0_180_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_180_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_181_type, _seq); p->level--; return _seq; } -// _tmp_181: 'as' NAME -static void * -_tmp_181_rule(Parser *p) -{ - if (p->level++ == MAXSTACK) { - p->error_indicator = 1; - PyErr_NoMemory(); - } - if (p->error_indicator) { - p->level--; - return NULL; - } - void * _res = NULL; - int _mark = p->mark; - { // 'as' NAME - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> _tmp_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); - Token * _keyword; - expr_ty name_var; - if ( - (_keyword = _PyPegen_expect_token(p, 632)) // token='as' - && - (name_var = _PyPegen_name_token(p)) // NAME - ) - { - D(fprintf(stderr, "%*c+ _tmp_181[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); - _res = _PyPegen_dummy_name(p, _keyword, name_var); - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_181[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); - } - _res = NULL; - done: - p->level--; - return _res; -} - // _tmp_182: 'as' NAME static void * _tmp_182_rule(Parser *p) @@ -33973,9 +33921,51 @@ _tmp_182_rule(Parser *p) return _res; } -// _tmp_183: NEWLINE | ':' +// _tmp_183: 'as' NAME static void * _tmp_183_rule(Parser *p) +{ + if (p->level++ == MAXSTACK) { + p->error_indicator = 1; + PyErr_NoMemory(); + } + if (p->error_indicator) { + p->level--; + return NULL; + } + void * _res = NULL; + int _mark = p->mark; + { // 'as' NAME + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + Token * _keyword; + expr_ty name_var; + if ( + (_keyword = _PyPegen_expect_token(p, 632)) // token='as' + && + (name_var = _PyPegen_name_token(p)) // NAME + ) + { + D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + _res = _PyPegen_dummy_name(p, _keyword, name_var); + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); + } + _res = NULL; + done: + p->level--; + return _res; +} + +// _tmp_184: NEWLINE | ':' +static void * +_tmp_184_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33992,18 +33982,18 @@ _tmp_183_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "NEWLINE")); + D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "NEWLINE")); Token * newline_var; if ( (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "NEWLINE")); + D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "NEWLINE")); _res = newline_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "NEWLINE")); } { // ':' @@ -34011,18 +34001,18 @@ _tmp_183_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } _res = NULL; @@ -34031,9 +34021,9 @@ _tmp_183_rule(Parser *p) return _res; } -// _tmp_184: 'as' NAME +// _tmp_185: 'as' NAME static void * -_tmp_184_rule(Parser *p) +_tmp_185_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34050,7 +34040,7 @@ _tmp_184_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_185[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -34059,12 +34049,12 @@ _tmp_184_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_185[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_185[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -34073,9 +34063,9 @@ _tmp_184_rule(Parser *p) return _res; } -// _tmp_185: 'as' NAME +// _tmp_186: 'as' NAME static void * -_tmp_185_rule(Parser *p) +_tmp_186_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34092,7 +34082,7 @@ _tmp_185_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_185[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_186[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -34101,12 +34091,12 @@ _tmp_185_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_185[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_186[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_185[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_186[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -34115,9 +34105,9 @@ _tmp_185_rule(Parser *p) return _res; } -// _tmp_186: positional_patterns ',' +// _tmp_187: positional_patterns ',' static void * -_tmp_186_rule(Parser *p) +_tmp_187_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34134,7 +34124,7 @@ _tmp_186_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_186[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); + D(fprintf(stderr, "%*c> _tmp_187[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); Token * _literal; asdl_pattern_seq* positional_patterns_var; if ( @@ -34143,12 +34133,12 @@ _tmp_186_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_186[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); + D(fprintf(stderr, "%*c+ _tmp_187[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); _res = _PyPegen_dummy_name(p, positional_patterns_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_186[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_187[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "positional_patterns ','")); } _res = NULL; @@ -34157,9 +34147,9 @@ _tmp_186_rule(Parser *p) return _res; } -// _tmp_187: '->' expression +// _tmp_188: '->' expression static void * -_tmp_187_rule(Parser *p) +_tmp_188_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34176,7 +34166,7 @@ _tmp_187_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_187[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'->' expression")); + D(fprintf(stderr, "%*c> _tmp_188[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'->' expression")); Token * _literal; expr_ty expression_var; if ( @@ -34185,12 +34175,12 @@ _tmp_187_rule(Parser *p) (expression_var = expression_rule(p)) // expression ) { - D(fprintf(stderr, "%*c+ _tmp_187[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'->' expression")); + D(fprintf(stderr, "%*c+ _tmp_188[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'->' expression")); _res = _PyPegen_dummy_name(p, _literal, expression_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_187[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_188[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'->' expression")); } _res = NULL; @@ -34199,9 +34189,9 @@ _tmp_187_rule(Parser *p) return _res; } -// _tmp_188: '(' arguments? ')' +// _tmp_189: '(' arguments? ')' static void * -_tmp_188_rule(Parser *p) +_tmp_189_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34218,7 +34208,7 @@ _tmp_188_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_188[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); + D(fprintf(stderr, "%*c> _tmp_189[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); Token * _literal; Token * _literal_1; void *_opt_var; @@ -34231,12 +34221,12 @@ _tmp_188_rule(Parser *p) (_literal_1 = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_188[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); + D(fprintf(stderr, "%*c+ _tmp_189[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); _res = _PyPegen_dummy_name(p, _literal, _opt_var, _literal_1); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_188[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_189[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'(' arguments? ')'")); } _res = NULL; @@ -34245,9 +34235,9 @@ _tmp_188_rule(Parser *p) return _res; } -// _loop0_190: ',' double_starred_kvpair +// _loop0_191: ',' double_starred_kvpair static asdl_seq * -_loop0_190_rule(Parser *p) +_loop0_191_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34274,7 +34264,7 @@ _loop0_190_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' double_starred_kvpair")); + D(fprintf(stderr, "%*c> _loop0_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' double_starred_kvpair")); Token * _literal; KeyValuePair* elem; while ( @@ -34305,7 +34295,7 @@ _loop0_190_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_190[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_191[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' double_starred_kvpair")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -34318,14 +34308,14 @@ _loop0_190_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_190_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_191_type, _seq); p->level--; return _seq; } -// _gather_189: double_starred_kvpair _loop0_190 +// _gather_190: double_starred_kvpair _loop0_191 static asdl_seq * -_gather_189_rule(Parser *p) +_gather_190_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34337,27 +34327,27 @@ _gather_189_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // double_starred_kvpair _loop0_190 + { // double_starred_kvpair _loop0_191 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_189[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_190")); + D(fprintf(stderr, "%*c> _gather_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_191")); KeyValuePair* elem; asdl_seq * seq; if ( (elem = double_starred_kvpair_rule(p)) // double_starred_kvpair && - (seq = _loop0_190_rule(p)) // _loop0_190 + (seq = _loop0_191_rule(p)) // _loop0_191 ) { - D(fprintf(stderr, "%*c+ _gather_189[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_190")); + D(fprintf(stderr, "%*c+ _gather_190[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_191")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_189[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "double_starred_kvpair _loop0_190")); + D(fprintf(stderr, "%*c%s _gather_190[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "double_starred_kvpair _loop0_191")); } _res = NULL; done: @@ -34365,9 +34355,9 @@ _gather_189_rule(Parser *p) return _res; } -// _tmp_191: '}' | ',' +// _tmp_192: '}' | ',' static void * -_tmp_191_rule(Parser *p) +_tmp_192_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34384,18 +34374,18 @@ _tmp_191_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'}'")); + D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'}'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 26)) // token='}' ) { - D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'}'")); + D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'}'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'}'")); } { // ',' @@ -34403,18 +34393,18 @@ _tmp_191_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "','")); } _res = NULL; @@ -34423,9 +34413,9 @@ _tmp_191_rule(Parser *p) return _res; } -// _tmp_192: star_targets '=' +// _tmp_193: star_targets '=' static void * -_tmp_192_rule(Parser *p) +_tmp_193_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34442,7 +34432,7 @@ _tmp_192_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty z; if ( @@ -34451,7 +34441,7 @@ _tmp_192_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34461,7 +34451,7 @@ _tmp_192_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -34470,9 +34460,9 @@ _tmp_192_rule(Parser *p) return _res; } -// _tmp_193: '.' | '...' +// _tmp_194: '.' | '...' static void * -_tmp_193_rule(Parser *p) +_tmp_194_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34489,18 +34479,18 @@ _tmp_193_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -34508,18 +34498,18 @@ _tmp_193_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -34528,9 +34518,9 @@ _tmp_193_rule(Parser *p) return _res; } -// _tmp_194: '.' | '...' +// _tmp_195: '.' | '...' static void * -_tmp_194_rule(Parser *p) +_tmp_195_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34547,18 +34537,18 @@ _tmp_194_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -34566,18 +34556,18 @@ _tmp_194_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -34586,9 +34576,9 @@ _tmp_194_rule(Parser *p) return _res; } -// _tmp_195: '@' named_expression NEWLINE +// _tmp_196: '@' named_expression NEWLINE static void * -_tmp_195_rule(Parser *p) +_tmp_196_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34605,7 +34595,7 @@ _tmp_195_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); Token * _literal; expr_ty f; Token * newline_var; @@ -34617,7 +34607,7 @@ _tmp_195_rule(Parser *p) (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); _res = f; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34627,7 +34617,7 @@ _tmp_195_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'@' named_expression NEWLINE")); } _res = NULL; @@ -34636,9 +34626,9 @@ _tmp_195_rule(Parser *p) return _res; } -// _tmp_196: ',' expression +// _tmp_197: ',' expression static void * -_tmp_196_rule(Parser *p) +_tmp_197_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34655,7 +34645,7 @@ _tmp_196_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); Token * _literal; expr_ty c; if ( @@ -34664,7 +34654,7 @@ _tmp_196_rule(Parser *p) (c = expression_rule(p)) // expression ) { - D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34674,7 +34664,7 @@ _tmp_196_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' expression")); } _res = NULL; @@ -34683,9 +34673,9 @@ _tmp_196_rule(Parser *p) return _res; } -// _tmp_197: ',' star_expression +// _tmp_198: ',' star_expression static void * -_tmp_197_rule(Parser *p) +_tmp_198_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34702,7 +34692,7 @@ _tmp_197_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c> _tmp_198[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); Token * _literal; expr_ty c; if ( @@ -34711,7 +34701,7 @@ _tmp_197_rule(Parser *p) (c = star_expression_rule(p)) // star_expression ) { - D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c+ _tmp_198[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34721,7 +34711,7 @@ _tmp_197_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_198[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_expression")); } _res = NULL; @@ -34730,9 +34720,9 @@ _tmp_197_rule(Parser *p) return _res; } -// _tmp_198: 'or' conjunction +// _tmp_199: 'or' conjunction static void * -_tmp_198_rule(Parser *p) +_tmp_199_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34749,7 +34739,7 @@ _tmp_198_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_198[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c> _tmp_199[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); Token * _keyword; expr_ty c; if ( @@ -34758,7 +34748,7 @@ _tmp_198_rule(Parser *p) (c = conjunction_rule(p)) // conjunction ) { - D(fprintf(stderr, "%*c+ _tmp_198[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c+ _tmp_199[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34768,7 +34758,7 @@ _tmp_198_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_198[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_199[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'or' conjunction")); } _res = NULL; @@ -34777,9 +34767,9 @@ _tmp_198_rule(Parser *p) return _res; } -// _tmp_199: 'and' inversion +// _tmp_200: 'and' inversion static void * -_tmp_199_rule(Parser *p) +_tmp_200_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34796,7 +34786,7 @@ _tmp_199_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_199[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c> _tmp_200[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); Token * _keyword; expr_ty c; if ( @@ -34805,7 +34795,7 @@ _tmp_199_rule(Parser *p) (c = inversion_rule(p)) // inversion ) { - D(fprintf(stderr, "%*c+ _tmp_199[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c+ _tmp_200[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34815,7 +34805,7 @@ _tmp_199_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_199[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_200[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'and' inversion")); } _res = NULL; @@ -34824,9 +34814,9 @@ _tmp_199_rule(Parser *p) return _res; } -// _tmp_200: 'if' disjunction +// _tmp_201: 'if' disjunction static void * -_tmp_200_rule(Parser *p) +_tmp_201_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34843,7 +34833,7 @@ _tmp_200_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_200[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -34852,7 +34842,7 @@ _tmp_200_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_200[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34862,7 +34852,7 @@ _tmp_200_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_200[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -34871,9 +34861,9 @@ _tmp_200_rule(Parser *p) return _res; } -// _tmp_201: 'if' disjunction +// _tmp_202: 'if' disjunction static void * -_tmp_201_rule(Parser *p) +_tmp_202_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34890,7 +34880,7 @@ _tmp_201_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -34899,7 +34889,7 @@ _tmp_201_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -34909,7 +34899,7 @@ _tmp_201_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -34918,9 +34908,9 @@ _tmp_201_rule(Parser *p) return _res; } -// _tmp_202: starred_expression | (assignment_expression | expression !':=') !'=' +// _tmp_203: starred_expression | (assignment_expression | expression !':=') !'=' static void * -_tmp_202_rule(Parser *p) +_tmp_203_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34937,18 +34927,18 @@ _tmp_202_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression")); + D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression")); expr_ty starred_expression_var; if ( (starred_expression_var = starred_expression_rule(p)) // starred_expression ) { - D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression")); + D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression")); _res = starred_expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "starred_expression")); } { // (assignment_expression | expression !':=') !'=' @@ -34956,20 +34946,20 @@ _tmp_202_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); - void *_tmp_215_var; + D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); + void *_tmp_216_var; if ( - (_tmp_215_var = _tmp_215_rule(p)) // assignment_expression | expression !':=' + (_tmp_216_var = _tmp_216_rule(p)) // assignment_expression | expression !':=' && _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 22) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); - _res = _tmp_215_var; + D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); + _res = _tmp_216_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(assignment_expression | expression !':=') !'='")); } _res = NULL; @@ -34978,9 +34968,9 @@ _tmp_202_rule(Parser *p) return _res; } -// _tmp_203: ',' star_target +// _tmp_204: ',' star_target static void * -_tmp_203_rule(Parser *p) +_tmp_204_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34997,7 +34987,7 @@ _tmp_203_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _tmp_204[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty c; if ( @@ -35006,7 +34996,7 @@ _tmp_203_rule(Parser *p) (c = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c+ _tmp_204[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -35016,7 +35006,7 @@ _tmp_203_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_204[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } _res = NULL; @@ -35025,9 +35015,9 @@ _tmp_203_rule(Parser *p) return _res; } -// _tmp_204: ',' star_target +// _tmp_205: ',' star_target static void * -_tmp_204_rule(Parser *p) +_tmp_205_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35044,7 +35034,7 @@ _tmp_204_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_204[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _tmp_205[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty c; if ( @@ -35053,7 +35043,7 @@ _tmp_204_rule(Parser *p) (c = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_204[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c+ _tmp_205[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -35063,7 +35053,7 @@ _tmp_204_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_204[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_205[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } _res = NULL; @@ -35072,9 +35062,9 @@ _tmp_204_rule(Parser *p) return _res; } -// _tmp_205: star_targets '=' +// _tmp_206: star_targets '=' static void * -_tmp_205_rule(Parser *p) +_tmp_206_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35091,7 +35081,7 @@ _tmp_205_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_205[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_206[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -35100,12 +35090,12 @@ _tmp_205_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_205[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_206[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_205[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_206[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -35114,9 +35104,9 @@ _tmp_205_rule(Parser *p) return _res; } -// _tmp_206: star_targets '=' +// _tmp_207: star_targets '=' static void * -_tmp_206_rule(Parser *p) +_tmp_207_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35133,7 +35123,7 @@ _tmp_206_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_206[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_207[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -35142,12 +35132,12 @@ _tmp_206_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_206[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_207[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_206[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_207[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -35156,9 +35146,9 @@ _tmp_206_rule(Parser *p) return _res; } -// _tmp_207: ')' | '**' +// _tmp_208: ')' | '**' static void * -_tmp_207_rule(Parser *p) +_tmp_208_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35175,18 +35165,18 @@ _tmp_207_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_207[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_208[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_207[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_208[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_207[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_208[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // '**' @@ -35194,18 +35184,18 @@ _tmp_207_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_207[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_208[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_207[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_208[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_207[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_208[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; @@ -35214,9 +35204,9 @@ _tmp_207_rule(Parser *p) return _res; } -// _tmp_208: ':' | '**' +// _tmp_209: ':' | '**' static void * -_tmp_208_rule(Parser *p) +_tmp_209_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35233,18 +35223,18 @@ _tmp_208_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_208[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_209[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_208[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_209[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_208[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_209[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // '**' @@ -35252,18 +35242,18 @@ _tmp_208_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_208[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_209[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_208[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_209[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_208[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_209[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; @@ -35272,9 +35262,9 @@ _tmp_208_rule(Parser *p) return _res; } -// _tmp_209: expression ['as' star_target] +// _tmp_210: expression ['as' star_target] static void * -_tmp_209_rule(Parser *p) +_tmp_210_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35291,22 +35281,22 @@ _tmp_209_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_209[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_210[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_216_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_217_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_209[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_210[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); _res = _PyPegen_dummy_name(p, expression_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_209[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_210[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression ['as' star_target]")); } _res = NULL; @@ -35315,9 +35305,9 @@ _tmp_209_rule(Parser *p) return _res; } -// _tmp_210: expressions ['as' star_target] +// _tmp_211: expressions ['as' star_target] static void * -_tmp_210_rule(Parser *p) +_tmp_211_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35334,22 +35324,22 @@ _tmp_210_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_210[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_211[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expressions_var; if ( (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_217_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_218_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_210[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_211[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); _res = _PyPegen_dummy_name(p, expressions_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_210[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_211[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expressions ['as' star_target]")); } _res = NULL; @@ -35358,9 +35348,9 @@ _tmp_210_rule(Parser *p) return _res; } -// _tmp_211: expression ['as' star_target] +// _tmp_212: expression ['as' star_target] static void * -_tmp_211_rule(Parser *p) +_tmp_212_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35377,22 +35367,22 @@ _tmp_211_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_211[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_212[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_218_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_219_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_211[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_212[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); _res = _PyPegen_dummy_name(p, expression_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_211[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_212[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression ['as' star_target]")); } _res = NULL; @@ -35401,9 +35391,9 @@ _tmp_211_rule(Parser *p) return _res; } -// _tmp_212: expressions ['as' star_target] +// _tmp_213: expressions ['as' star_target] static void * -_tmp_212_rule(Parser *p) +_tmp_213_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35420,22 +35410,22 @@ _tmp_212_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_212[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_213[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expressions_var; if ( (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_219_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_220_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_212[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_213[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); _res = _PyPegen_dummy_name(p, expressions_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_212[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_213[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expressions ['as' star_target]")); } _res = NULL; @@ -35444,9 +35434,9 @@ _tmp_212_rule(Parser *p) return _res; } -// _tmp_213: except_block+ except_star_block +// _tmp_214: except_block+ except_star_block static void * -_tmp_213_rule(Parser *p) +_tmp_214_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35463,21 +35453,21 @@ _tmp_213_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_213[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_block+ except_star_block")); - asdl_seq * _loop1_220_var; + D(fprintf(stderr, "%*c> _tmp_214[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_block+ except_star_block")); + asdl_seq * _loop1_221_var; excepthandler_ty except_star_block_var; if ( - (_loop1_220_var = _loop1_220_rule(p)) // except_block+ + (_loop1_221_var = _loop1_221_rule(p)) // except_block+ && (except_star_block_var = except_star_block_rule(p)) // except_star_block ) { - D(fprintf(stderr, "%*c+ _tmp_213[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "except_block+ except_star_block")); - _res = _PyPegen_dummy_name(p, _loop1_220_var, except_star_block_var); + D(fprintf(stderr, "%*c+ _tmp_214[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "except_block+ except_star_block")); + _res = _PyPegen_dummy_name(p, _loop1_221_var, except_star_block_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_213[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_214[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "except_block+ except_star_block")); } _res = NULL; @@ -35486,9 +35476,9 @@ _tmp_213_rule(Parser *p) return _res; } -// _tmp_214: except_star_block+ except_block +// _tmp_215: except_star_block+ except_block static void * -_tmp_214_rule(Parser *p) +_tmp_215_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35505,21 +35495,21 @@ _tmp_214_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_214[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_star_block+ except_block")); - asdl_seq * _loop1_221_var; + D(fprintf(stderr, "%*c> _tmp_215[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_star_block+ except_block")); + asdl_seq * _loop1_222_var; excepthandler_ty except_block_var; if ( - (_loop1_221_var = _loop1_221_rule(p)) // except_star_block+ + (_loop1_222_var = _loop1_222_rule(p)) // except_star_block+ && (except_block_var = except_block_rule(p)) // except_block ) { - D(fprintf(stderr, "%*c+ _tmp_214[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "except_star_block+ except_block")); - _res = _PyPegen_dummy_name(p, _loop1_221_var, except_block_var); + D(fprintf(stderr, "%*c+ _tmp_215[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "except_star_block+ except_block")); + _res = _PyPegen_dummy_name(p, _loop1_222_var, except_block_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_214[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_215[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "except_star_block+ except_block")); } _res = NULL; @@ -35528,9 +35518,9 @@ _tmp_214_rule(Parser *p) return _res; } -// _tmp_215: assignment_expression | expression !':=' +// _tmp_216: assignment_expression | expression !':=' static void * -_tmp_215_rule(Parser *p) +_tmp_216_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35547,18 +35537,18 @@ _tmp_215_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_215[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "assignment_expression")); + D(fprintf(stderr, "%*c> _tmp_216[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "assignment_expression")); expr_ty assignment_expression_var; if ( (assignment_expression_var = assignment_expression_rule(p)) // assignment_expression ) { - D(fprintf(stderr, "%*c+ _tmp_215[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "assignment_expression")); + D(fprintf(stderr, "%*c+ _tmp_216[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "assignment_expression")); _res = assignment_expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_215[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_216[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "assignment_expression")); } { // expression !':=' @@ -35566,7 +35556,7 @@ _tmp_215_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_215[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression !':='")); + D(fprintf(stderr, "%*c> _tmp_216[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression !':='")); expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression @@ -35574,12 +35564,12 @@ _tmp_215_rule(Parser *p) _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 53) // token=':=' ) { - D(fprintf(stderr, "%*c+ _tmp_215[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression !':='")); + D(fprintf(stderr, "%*c+ _tmp_216[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression !':='")); _res = expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_215[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_216[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression !':='")); } _res = NULL; @@ -35588,9 +35578,9 @@ _tmp_215_rule(Parser *p) return _res; } -// _tmp_216: 'as' star_target +// _tmp_217: 'as' star_target static void * -_tmp_216_rule(Parser *p) +_tmp_217_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35607,7 +35597,7 @@ _tmp_216_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_216[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_217[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -35616,12 +35606,12 @@ _tmp_216_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_216[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_217[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_216[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_217[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -35630,9 +35620,9 @@ _tmp_216_rule(Parser *p) return _res; } -// _tmp_217: 'as' star_target +// _tmp_218: 'as' star_target static void * -_tmp_217_rule(Parser *p) +_tmp_218_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35649,7 +35639,7 @@ _tmp_217_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_217[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_218[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -35658,12 +35648,12 @@ _tmp_217_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_217[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_218[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_217[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_218[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -35672,9 +35662,9 @@ _tmp_217_rule(Parser *p) return _res; } -// _tmp_218: 'as' star_target +// _tmp_219: 'as' star_target static void * -_tmp_218_rule(Parser *p) +_tmp_219_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35691,7 +35681,7 @@ _tmp_218_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_218[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_219[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -35700,12 +35690,12 @@ _tmp_218_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_218[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_219[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_218[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_219[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -35714,9 +35704,9 @@ _tmp_218_rule(Parser *p) return _res; } -// _tmp_219: 'as' star_target +// _tmp_220: 'as' star_target static void * -_tmp_219_rule(Parser *p) +_tmp_220_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35733,7 +35723,7 @@ _tmp_219_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_219[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_220[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -35742,12 +35732,12 @@ _tmp_219_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_219[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_220[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_219[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_220[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -35756,9 +35746,9 @@ _tmp_219_rule(Parser *p) return _res; } -// _loop1_220: except_block +// _loop1_221: except_block static asdl_seq * -_loop1_220_rule(Parser *p) +_loop1_221_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35785,7 +35775,7 @@ _loop1_220_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_220[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_block")); + D(fprintf(stderr, "%*c> _loop1_221[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_block")); excepthandler_ty except_block_var; while ( (except_block_var = except_block_rule(p)) // except_block @@ -35807,7 +35797,7 @@ _loop1_220_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_220[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_221[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "except_block")); } if (_n == 0 || p->error_indicator) { @@ -35825,14 +35815,14 @@ _loop1_220_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_220_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_221_type, _seq); p->level--; return _seq; } -// _loop1_221: except_star_block +// _loop1_222: except_star_block static asdl_seq * -_loop1_221_rule(Parser *p) +_loop1_222_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -35859,7 +35849,7 @@ _loop1_221_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_221[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_star_block")); + D(fprintf(stderr, "%*c> _loop1_222[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "except_star_block")); excepthandler_ty except_star_block_var; while ( (except_star_block_var = except_star_block_rule(p)) // except_star_block @@ -35881,7 +35871,7 @@ _loop1_221_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_221[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_222[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "except_star_block")); } if (_n == 0 || p->error_indicator) { @@ -35899,7 +35889,7 @@ _loop1_221_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_221_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_222_type, _seq); p->level--; return _seq; } diff --git a/Parser/pegen.c b/Parser/pegen.c index 6adde8432307d..839ffeff76fb1 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -381,6 +381,7 @@ _PyPegen_expect_token(Parser *p, int type) } Token *t = p->tokens[p->mark]; if (t->type != type) { + if (Py_DebugFlag) fprintf(stderr, "Token = %s\n", PyBytes_AsString(t->bytes)); return NULL; } p->mark += 1; @@ -785,7 +786,6 @@ _PyPegen_Parser_New(struct tok_state *tok, int start_rule, int flags, p->known_err_token = NULL; p->level = 0; p->call_invalid_rules = 0; - p->in_raw_rule = 0; return p; } diff --git a/Parser/pegen.h b/Parser/pegen.h index 061ca3a2013cb..77d5ca8418a97 100644 --- a/Parser/pegen.h +++ b/Parser/pegen.h @@ -78,7 +78,6 @@ typedef struct { Token *known_err_token; int level; int call_invalid_rules; - int in_raw_rule; } Parser; typedef struct { diff --git a/Tools/peg_generator/pegen/c_generator.py b/Tools/peg_generator/pegen/c_generator.py index ee255c8016386..56a1e5a5a14fb 100644 --- a/Tools/peg_generator/pegen/c_generator.py +++ b/Tools/peg_generator/pegen/c_generator.py @@ -122,6 +122,7 @@ def __init__( self.exact_tokens = exact_tokens self.non_exact_tokens = non_exact_tokens self.cache: Dict[Any, FunctionCall] = {} + self.cleanup_statements: List[str] = [] def keyword_helper(self, keyword: str) -> FunctionCall: return FunctionCall( @@ -364,6 +365,7 @@ def __init__( self._varname_counter = 0 self.debug = debug self.skip_actions = skip_actions + self.cleanup_statements: List[str] = [] def add_level(self) -> None: self.print("if (p->level++ == MAXSTACK) {") @@ -376,6 +378,8 @@ def remove_level(self) -> None: self.print("p->level--;") def add_return(self, ret_val: str) -> None: + for stmt in self.cleanup_statements: + self.print(stmt) self.remove_level() self.print(f"return {ret_val};") @@ -547,9 +551,7 @@ def _set_up_rule_memoization(self, node: Rule, result_type: str) -> None: f"_PyPegen_update_memo(p, _mark, {node.name}_type, _res)", "_res" ) self.print("p->mark = _mark;") - self.print("p->in_raw_rule++;") self.print(f"void *_raw = {node.name}_raw(p);") - self.print("p->in_raw_rule--;") self.print("if (p->error_indicator) {") with self.indent(): self.add_return("NULL") @@ -663,10 +665,21 @@ def visit_Rule(self, node: Rule) -> None: self._set_up_rule_memoization(node, result_type) self.print("{") + + if node.name.endswith("without_invalid"): + with self.indent(): + self.print("int _prev_call_invalid = p->call_invalid_rules;") + self.print("p->call_invalid_rules = 0;") + self.cleanup_statements.append("p->call_invalid_rules = _prev_call_invalid;") + if is_loop: self._handle_loop_rule_body(node, rhs) else: self._handle_default_rule_body(node, rhs, result_type) + + if node.name.endswith("without_invalid"): + self.cleanup_statements.pop() + self.print("}") def visit_NamedItem(self, node: NamedItem) -> None: From webhook-mailer at python.org Thu Feb 10 08:27:19 2022 From: webhook-mailer at python.org (asvetlov) Date: Thu, 10 Feb 2022 13:27:19 -0000 Subject: [Python-checkins] [3.10] Fix warning: asyncio.events._event_loop_policy was modified by test_asyncio (GH-31253) (GH-31255) Message-ID: https://github.com/python/cpython/commit/9f5145403b40b19e5c00ba43eaa35d5ea2b7b545 commit: 9f5145403b40b19e5c00ba43eaa35d5ea2b7b545 branch: 3.10 author: Andrew Svetlov committer: asvetlov date: 2022-02-10T15:27:15+02:00 summary: [3.10] Fix warning: asyncio.events._event_loop_policy was modified by test_asyncio (GH-31253) (GH-31255) (cherry picked from commit 012e77eb5c3ba3d411f5967a7f368ebdb42ab88c) Co-authored-by: Andrew Svetlov files: A Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst M Lib/test/test_asyncio/test_futures2.py M Lib/test/test_asyncio/test_protocols.py M Lib/test/test_asyncio/test_runners.py M Lib/test/test_asyncio/test_sock_lowlevel.py M Lib/test/test_asyncio/test_transports.py M Lib/test/test_asyncio/test_unix_events.py diff --git a/Lib/test/test_asyncio/test_futures2.py b/Lib/test/test_asyncio/test_futures2.py index 57d24190bc0bd..60b58850369f0 100644 --- a/Lib/test/test_asyncio/test_futures2.py +++ b/Lib/test/test_asyncio/test_futures2.py @@ -3,6 +3,10 @@ import unittest +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + class FutureTests(unittest.IsolatedAsyncioTestCase): async def test_recursive_repr_for_pending_tasks(self): # The call crashes if the guard for recursive call diff --git a/Lib/test/test_asyncio/test_protocols.py b/Lib/test/test_asyncio/test_protocols.py index d8cde6d89aadc..0f232631867db 100644 --- a/Lib/test/test_asyncio/test_protocols.py +++ b/Lib/test/test_asyncio/test_protocols.py @@ -4,6 +4,12 @@ import asyncio +def tearDownModule(): + # not needed for the test file but added for uniformness with all other + # asyncio test files for the sake of unified cleanup + asyncio.set_event_loop_policy(None) + + class ProtocolsAbsTests(unittest.TestCase): def test_base_protocol(self): diff --git a/Lib/test/test_asyncio/test_runners.py b/Lib/test/test_asyncio/test_runners.py index 5c06a1aaa830f..112273662b20b 100644 --- a/Lib/test/test_asyncio/test_runners.py +++ b/Lib/test/test_asyncio/test_runners.py @@ -5,6 +5,10 @@ from test.test_asyncio import utils as test_utils +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + class TestPolicy(asyncio.AbstractEventLoopPolicy): def __init__(self, loop_factory): diff --git a/Lib/test/test_asyncio/test_sock_lowlevel.py b/Lib/test/test_asyncio/test_sock_lowlevel.py index 448d835b04d57..14001a4a5001f 100644 --- a/Lib/test/test_asyncio/test_sock_lowlevel.py +++ b/Lib/test/test_asyncio/test_sock_lowlevel.py @@ -10,6 +10,10 @@ from test.support import socket_helper +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + class MyProto(asyncio.Protocol): connected = None done = None diff --git a/Lib/test/test_asyncio/test_transports.py b/Lib/test/test_asyncio/test_transports.py index df448557a7b7f..bbdb218efaa3b 100644 --- a/Lib/test/test_asyncio/test_transports.py +++ b/Lib/test/test_asyncio/test_transports.py @@ -7,6 +7,12 @@ from asyncio import transports +def tearDownModule(): + # not needed for the test file but added for uniformness with all other + # asyncio test files for the sake of unified cleanup + asyncio.set_event_loop_policy(None) + + class TransportTests(unittest.TestCase): def test_ctor_extra_is_none(self): diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py index c3422850ce1b7..2f68459d30cd4 100644 --- a/Lib/test/test_asyncio/test_unix_events.py +++ b/Lib/test/test_asyncio/test_unix_events.py @@ -26,6 +26,10 @@ from test.test_asyncio import utils as test_utils +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + MOCK_ANY = mock.ANY @@ -39,10 +43,6 @@ def SIGNAL(signum): return 32768 - signum -def tearDownModule(): - asyncio.set_event_loop_policy(None) - - def close_pipe_transport(transport): # Don't call transport.close() because the event loop and the selector # are mocked diff --git a/Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst b/Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst new file mode 100644 index 0000000000000..119107a8fb96c --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst @@ -0,0 +1,2 @@ +Prevent default asyncio event loop policy modification warning after +``test_asyncio`` execution. From webhook-mailer at python.org Thu Feb 10 08:32:14 2022 From: webhook-mailer at python.org (asvetlov) Date: Thu, 10 Feb 2022 13:32:14 -0000 Subject: [Python-checkins] [3.9] Fix warning: asyncio.events._event_loop_policy was modified by test_asyncio (GH-31253). (GH-31256) Message-ID: https://github.com/python/cpython/commit/a65be6e077122017081618278d758113a4a2eefd commit: a65be6e077122017081618278d758113a4a2eefd branch: 3.9 author: Andrew Svetlov committer: asvetlov date: 2022-02-10T15:32:05+02:00 summary: [3.9] Fix warning: asyncio.events._event_loop_policy was modified by test_asyncio (GH-31253). (GH-31256) (cherry picked from commit 012e77eb5c3ba3d411f5967a7f368ebdb42ab88c) Co-authored-by: Andrew Svetlov files: A Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst M Lib/test/test_asyncio/test_futures2.py M Lib/test/test_asyncio/test_protocols.py M Lib/test/test_asyncio/test_runners.py M Lib/test/test_asyncio/test_sock_lowlevel.py M Lib/test/test_asyncio/test_transports.py M Lib/test/test_asyncio/test_unix_events.py diff --git a/Lib/test/test_asyncio/test_futures2.py b/Lib/test/test_asyncio/test_futures2.py index 57d24190bc0bd..60b58850369f0 100644 --- a/Lib/test/test_asyncio/test_futures2.py +++ b/Lib/test/test_asyncio/test_futures2.py @@ -3,6 +3,10 @@ import unittest +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + class FutureTests(unittest.IsolatedAsyncioTestCase): async def test_recursive_repr_for_pending_tasks(self): # The call crashes if the guard for recursive call diff --git a/Lib/test/test_asyncio/test_protocols.py b/Lib/test/test_asyncio/test_protocols.py index d8cde6d89aadc..0f232631867db 100644 --- a/Lib/test/test_asyncio/test_protocols.py +++ b/Lib/test/test_asyncio/test_protocols.py @@ -4,6 +4,12 @@ import asyncio +def tearDownModule(): + # not needed for the test file but added for uniformness with all other + # asyncio test files for the sake of unified cleanup + asyncio.set_event_loop_policy(None) + + class ProtocolsAbsTests(unittest.TestCase): def test_base_protocol(self): diff --git a/Lib/test/test_asyncio/test_runners.py b/Lib/test/test_asyncio/test_runners.py index 5c06a1aaa830f..112273662b20b 100644 --- a/Lib/test/test_asyncio/test_runners.py +++ b/Lib/test/test_asyncio/test_runners.py @@ -5,6 +5,10 @@ from test.test_asyncio import utils as test_utils +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + class TestPolicy(asyncio.AbstractEventLoopPolicy): def __init__(self, loop_factory): diff --git a/Lib/test/test_asyncio/test_sock_lowlevel.py b/Lib/test/test_asyncio/test_sock_lowlevel.py index 7de27b11154a4..025c6c0132a97 100644 --- a/Lib/test/test_asyncio/test_sock_lowlevel.py +++ b/Lib/test/test_asyncio/test_sock_lowlevel.py @@ -10,6 +10,10 @@ from test.support import socket_helper +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + class MyProto(asyncio.Protocol): connected = None done = None diff --git a/Lib/test/test_asyncio/test_transports.py b/Lib/test/test_asyncio/test_transports.py index df448557a7b7f..bbdb218efaa3b 100644 --- a/Lib/test/test_asyncio/test_transports.py +++ b/Lib/test/test_asyncio/test_transports.py @@ -7,6 +7,12 @@ from asyncio import transports +def tearDownModule(): + # not needed for the test file but added for uniformness with all other + # asyncio test files for the sake of unified cleanup + asyncio.set_event_loop_policy(None) + + class TransportTests(unittest.TestCase): def test_ctor_extra_is_none(self): diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py index 10bd46dea1991..70d306ffe8fbb 100644 --- a/Lib/test/test_asyncio/test_unix_events.py +++ b/Lib/test/test_asyncio/test_unix_events.py @@ -27,13 +27,13 @@ from test.test_asyncio import utils as test_utils -MOCK_ANY = mock.ANY - - def tearDownModule(): asyncio.set_event_loop_policy(None) +MOCK_ANY = mock.ANY + + def close_pipe_transport(transport): # Don't call transport.close() because the event loop and the selector # are mocked diff --git a/Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst b/Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst new file mode 100644 index 0000000000000..119107a8fb96c --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-10-14-33-47.bpo-46708.avLfCb.rst @@ -0,0 +1,2 @@ +Prevent default asyncio event loop policy modification warning after +``test_asyncio`` execution. From webhook-mailer at python.org Thu Feb 10 09:38:39 2022 From: webhook-mailer at python.org (pablogsal) Date: Thu, 10 Feb 2022 14:38:39 -0000 Subject: [Python-checkins] Don't print rejected tokens when using the debug flags in the parser (GH-31258) Message-ID: https://github.com/python/cpython/commit/e19059ecd80d39dca378cd19f72c5008b1957976 commit: e19059ecd80d39dca378cd19f72c5008b1957976 branch: main author: Pablo Galindo Salgado committer: pablogsal date: 2022-02-10T14:38:27Z summary: Don't print rejected tokens when using the debug flags in the parser (GH-31258) files: M Parser/pegen.c diff --git a/Parser/pegen.c b/Parser/pegen.c index 839ffeff76fb1..95dc54dd6040f 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -381,7 +381,6 @@ _PyPegen_expect_token(Parser *p, int type) } Token *t = p->tokens[p->mark]; if (t->type != type) { - if (Py_DebugFlag) fprintf(stderr, "Token = %s\n", PyBytes_AsString(t->bytes)); return NULL; } p->mark += 1; From webhook-mailer at python.org Thu Feb 10 09:38:40 2022 From: webhook-mailer at python.org (pablogsal) Date: Thu, 10 Feb 2022 14:38:40 -0000 Subject: [Python-checkins] [3.10] Allow the parser to avoid nested processing of invalid rules (GH-31252). (GH-31257) Message-ID: https://github.com/python/cpython/commit/14284b0e71c2110f94941b774478fb594d0ef192 commit: 14284b0e71c2110f94941b774478fb594d0ef192 branch: 3.10 author: Pablo Galindo Salgado committer: pablogsal date: 2022-02-10T14:38:31Z summary: [3.10] Allow the parser to avoid nested processing of invalid rules (GH-31252). (GH-31257) (cherry picked from commit 390459de6db1e68b79c0897cc88c0d562693ec5c) Co-authored-by: Pablo Galindo Salgado files: M Grammar/python.gram M Lib/test/test_exceptions.py M Parser/parser.c M Parser/pegen.c M Parser/pegen.h M Tools/peg_generator/pegen/c_generator.py diff --git a/Grammar/python.gram b/Grammar/python.gram index 69c20bddf46eb..99e01355a582c 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -846,6 +846,7 @@ invalid_kwarg: RAISE_SYNTAX_ERROR_KNOWN_RANGE( a, b, "expression cannot contain assignment, perhaps you meant \"==\"?") } +# IMPORTANT: Note that the "_without_invalid" suffix causes the rule to not call invalid rules under it expression_without_invalid[expr_ty]: | a=disjunction 'if' b=disjunction 'else' c=expression { _PyAST_IfExp(b, a, c, EXTRA) } | disjunction @@ -863,16 +864,14 @@ invalid_expression: RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Perhaps you forgot a comma?") } | a=disjunction 'if' b=disjunction !('else'|':') { RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "expected 'else' after 'if' expression") } -invalid_left_assignment_prefixes(memo): list|tuple|genexp|'True'|'None'|'False' - -invalid_named_expression: +invalid_named_expression(memo): | a=expression ':=' expression { RAISE_SYNTAX_ERROR_KNOWN_LOCATION( a, "cannot use assignment expressions with %s", _PyPegen_get_expr_name(a)) } | a=NAME '=' b=bitwise_or !('='|':=') { - p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?") } - | !invalid_left_assignment_prefixes a=bitwise_or b='=' bitwise_or !('='|':=') { - p->in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "cannot assign to %s here. Maybe you meant '==' instead of '='?", + RAISE_SYNTAX_ERROR_KNOWN_RANGE(a, b, "invalid syntax. Maybe you meant '==' or ':=' instead of '='?") } + | !(list|tuple|genexp|'True'|'None'|'False') a=bitwise_or b='=' bitwise_or !('='|':=') { + RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "cannot assign to %s here. Maybe you meant '==' instead of '='?", _PyPegen_get_expr_name(a)) } invalid_assignment: diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py index 606e6852627c4..49739723ee473 100644 --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -231,7 +231,7 @@ def testSyntaxErrorOffset(self): check('a = ? hello ? ? world ?', 1, 5) check('[\nfile\nfor str(file)\nin\n[]\n]', 3, 5) check('[file for\n str(file) in []]', 2, 2) - check("ages = {'Alice'=22, 'Bob'=23}", 1, 16) + check("ages = {'Alice'=22, 'Bob'=23}", 1, 9) check('match ...:\n case {**rest, "key": value}:\n ...', 2, 19) check("[a b c d e f]", 1, 2) check("for x yfff:", 1, 7) diff --git a/Parser/parser.c b/Parser/parser.c index 763716982ed83..3f73003b78973 100644 --- a/Parser/parser.c +++ b/Parser/parser.c @@ -256,250 +256,250 @@ static char *soft_keywords[] = { #define expression_without_invalid_type 1180 #define invalid_legacy_expression_type 1181 #define invalid_expression_type 1182 -#define invalid_left_assignment_prefixes_type 1183 -#define invalid_named_expression_type 1184 -#define invalid_assignment_type 1185 -#define invalid_ann_assign_target_type 1186 -#define invalid_del_stmt_type 1187 -#define invalid_block_type 1188 -#define invalid_comprehension_type 1189 -#define invalid_dict_comprehension_type 1190 -#define invalid_parameters_type 1191 -#define invalid_parameters_helper_type 1192 -#define invalid_lambda_parameters_type 1193 -#define invalid_lambda_parameters_helper_type 1194 -#define invalid_star_etc_type 1195 -#define invalid_lambda_star_etc_type 1196 -#define invalid_double_type_comments_type 1197 -#define invalid_with_item_type 1198 -#define invalid_for_target_type 1199 -#define invalid_group_type 1200 -#define invalid_import_from_targets_type 1201 -#define invalid_with_stmt_type 1202 -#define invalid_with_stmt_indent_type 1203 -#define invalid_try_stmt_type 1204 -#define invalid_except_stmt_type 1205 -#define invalid_finally_stmt_type 1206 -#define invalid_except_stmt_indent_type 1207 -#define invalid_match_stmt_type 1208 -#define invalid_case_block_type 1209 -#define invalid_as_pattern_type 1210 -#define invalid_class_pattern_type 1211 -#define invalid_class_argument_pattern_type 1212 -#define invalid_if_stmt_type 1213 -#define invalid_elif_stmt_type 1214 -#define invalid_else_stmt_type 1215 -#define invalid_while_stmt_type 1216 -#define invalid_for_stmt_type 1217 -#define invalid_def_raw_type 1218 -#define invalid_class_def_raw_type 1219 -#define invalid_double_starred_kvpairs_type 1220 -#define invalid_kvpair_type 1221 -#define _loop0_1_type 1222 -#define _loop0_2_type 1223 -#define _loop0_4_type 1224 -#define _gather_3_type 1225 -#define _loop0_6_type 1226 -#define _gather_5_type 1227 -#define _loop0_8_type 1228 -#define _gather_7_type 1229 -#define _loop0_10_type 1230 -#define _gather_9_type 1231 -#define _loop1_11_type 1232 -#define _loop0_13_type 1233 -#define _gather_12_type 1234 -#define _tmp_14_type 1235 -#define _tmp_15_type 1236 -#define _tmp_16_type 1237 -#define _tmp_17_type 1238 -#define _tmp_18_type 1239 -#define _tmp_19_type 1240 -#define _tmp_20_type 1241 -#define _tmp_21_type 1242 -#define _loop1_22_type 1243 -#define _tmp_23_type 1244 -#define _tmp_24_type 1245 -#define _loop0_26_type 1246 -#define _gather_25_type 1247 -#define _loop0_28_type 1248 -#define _gather_27_type 1249 -#define _tmp_29_type 1250 -#define _tmp_30_type 1251 -#define _loop0_31_type 1252 -#define _loop1_32_type 1253 -#define _loop0_34_type 1254 -#define _gather_33_type 1255 -#define _tmp_35_type 1256 -#define _loop0_37_type 1257 -#define _gather_36_type 1258 -#define _tmp_38_type 1259 -#define _loop0_40_type 1260 -#define _gather_39_type 1261 -#define _loop0_42_type 1262 -#define _gather_41_type 1263 -#define _loop0_44_type 1264 -#define _gather_43_type 1265 -#define _loop0_46_type 1266 -#define _gather_45_type 1267 -#define _tmp_47_type 1268 -#define _loop1_48_type 1269 -#define _tmp_49_type 1270 -#define _loop1_50_type 1271 -#define _loop0_52_type 1272 -#define _gather_51_type 1273 -#define _tmp_53_type 1274 -#define _tmp_54_type 1275 -#define _tmp_55_type 1276 -#define _tmp_56_type 1277 -#define _loop0_58_type 1278 -#define _gather_57_type 1279 -#define _loop0_60_type 1280 -#define _gather_59_type 1281 -#define _tmp_61_type 1282 -#define _loop0_63_type 1283 -#define _gather_62_type 1284 -#define _loop0_65_type 1285 -#define _gather_64_type 1286 -#define _tmp_66_type 1287 -#define _tmp_67_type 1288 -#define _tmp_68_type 1289 -#define _tmp_69_type 1290 -#define _loop0_70_type 1291 -#define _loop0_71_type 1292 -#define _loop0_72_type 1293 -#define _loop1_73_type 1294 -#define _loop0_74_type 1295 -#define _loop1_75_type 1296 -#define _loop1_76_type 1297 -#define _loop1_77_type 1298 -#define _loop0_78_type 1299 -#define _loop1_79_type 1300 -#define _loop0_80_type 1301 -#define _loop1_81_type 1302 -#define _loop0_82_type 1303 -#define _loop1_83_type 1304 -#define _loop1_84_type 1305 -#define _tmp_85_type 1306 -#define _loop1_86_type 1307 -#define _loop0_88_type 1308 -#define _gather_87_type 1309 -#define _loop1_89_type 1310 -#define _loop0_90_type 1311 -#define _loop0_91_type 1312 -#define _loop0_92_type 1313 -#define _loop1_93_type 1314 -#define _loop0_94_type 1315 -#define _loop1_95_type 1316 -#define _loop1_96_type 1317 -#define _loop1_97_type 1318 -#define _loop0_98_type 1319 -#define _loop1_99_type 1320 -#define _loop0_100_type 1321 -#define _loop1_101_type 1322 -#define _loop0_102_type 1323 -#define _loop1_103_type 1324 -#define _loop1_104_type 1325 -#define _loop1_105_type 1326 -#define _loop1_106_type 1327 -#define _tmp_107_type 1328 -#define _loop0_109_type 1329 -#define _gather_108_type 1330 -#define _tmp_110_type 1331 -#define _tmp_111_type 1332 -#define _tmp_112_type 1333 -#define _tmp_113_type 1334 -#define _loop1_114_type 1335 -#define _tmp_115_type 1336 -#define _tmp_116_type 1337 -#define _tmp_117_type 1338 -#define _loop0_119_type 1339 -#define _gather_118_type 1340 -#define _loop1_120_type 1341 -#define _loop0_121_type 1342 -#define _loop0_122_type 1343 -#define _loop0_124_type 1344 -#define _gather_123_type 1345 -#define _tmp_125_type 1346 -#define _loop0_127_type 1347 -#define _gather_126_type 1348 -#define _loop0_129_type 1349 -#define _gather_128_type 1350 -#define _loop0_131_type 1351 -#define _gather_130_type 1352 -#define _loop0_133_type 1353 -#define _gather_132_type 1354 -#define _loop0_134_type 1355 -#define _loop0_136_type 1356 -#define _gather_135_type 1357 -#define _loop1_137_type 1358 -#define _tmp_138_type 1359 -#define _loop0_140_type 1360 -#define _gather_139_type 1361 -#define _tmp_141_type 1362 -#define _tmp_142_type 1363 -#define _tmp_143_type 1364 -#define _tmp_144_type 1365 -#define _tmp_145_type 1366 -#define _tmp_146_type 1367 -#define _tmp_147_type 1368 -#define _loop0_148_type 1369 -#define _loop0_149_type 1370 -#define _loop0_150_type 1371 -#define _tmp_151_type 1372 -#define _tmp_152_type 1373 -#define _tmp_153_type 1374 -#define _tmp_154_type 1375 -#define _loop0_155_type 1376 -#define _loop1_156_type 1377 -#define _loop0_157_type 1378 -#define _loop1_158_type 1379 -#define _tmp_159_type 1380 -#define _tmp_160_type 1381 -#define _tmp_161_type 1382 -#define _loop0_163_type 1383 -#define _gather_162_type 1384 -#define _loop0_165_type 1385 -#define _gather_164_type 1386 -#define _loop0_167_type 1387 -#define _gather_166_type 1388 -#define _loop0_169_type 1389 -#define _gather_168_type 1390 -#define _tmp_170_type 1391 -#define _tmp_171_type 1392 -#define _tmp_172_type 1393 -#define _tmp_173_type 1394 -#define _tmp_174_type 1395 -#define _tmp_175_type 1396 -#define _tmp_176_type 1397 -#define _loop0_178_type 1398 -#define _gather_177_type 1399 -#define _tmp_179_type 1400 -#define _tmp_180_type 1401 -#define _tmp_181_type 1402 -#define _tmp_182_type 1403 -#define _tmp_183_type 1404 -#define _tmp_184_type 1405 -#define _tmp_185_type 1406 -#define _tmp_186_type 1407 -#define _tmp_187_type 1408 -#define _tmp_188_type 1409 -#define _tmp_189_type 1410 -#define _tmp_190_type 1411 -#define _tmp_191_type 1412 -#define _tmp_192_type 1413 -#define _tmp_193_type 1414 -#define _tmp_194_type 1415 -#define _tmp_195_type 1416 -#define _tmp_196_type 1417 -#define _tmp_197_type 1418 -#define _tmp_198_type 1419 -#define _tmp_199_type 1420 -#define _tmp_200_type 1421 -#define _tmp_201_type 1422 -#define _tmp_202_type 1423 -#define _tmp_203_type 1424 -#define _tmp_204_type 1425 -#define _tmp_205_type 1426 +#define invalid_named_expression_type 1183 +#define invalid_assignment_type 1184 +#define invalid_ann_assign_target_type 1185 +#define invalid_del_stmt_type 1186 +#define invalid_block_type 1187 +#define invalid_comprehension_type 1188 +#define invalid_dict_comprehension_type 1189 +#define invalid_parameters_type 1190 +#define invalid_parameters_helper_type 1191 +#define invalid_lambda_parameters_type 1192 +#define invalid_lambda_parameters_helper_type 1193 +#define invalid_star_etc_type 1194 +#define invalid_lambda_star_etc_type 1195 +#define invalid_double_type_comments_type 1196 +#define invalid_with_item_type 1197 +#define invalid_for_target_type 1198 +#define invalid_group_type 1199 +#define invalid_import_from_targets_type 1200 +#define invalid_with_stmt_type 1201 +#define invalid_with_stmt_indent_type 1202 +#define invalid_try_stmt_type 1203 +#define invalid_except_stmt_type 1204 +#define invalid_finally_stmt_type 1205 +#define invalid_except_stmt_indent_type 1206 +#define invalid_match_stmt_type 1207 +#define invalid_case_block_type 1208 +#define invalid_as_pattern_type 1209 +#define invalid_class_pattern_type 1210 +#define invalid_class_argument_pattern_type 1211 +#define invalid_if_stmt_type 1212 +#define invalid_elif_stmt_type 1213 +#define invalid_else_stmt_type 1214 +#define invalid_while_stmt_type 1215 +#define invalid_for_stmt_type 1216 +#define invalid_def_raw_type 1217 +#define invalid_class_def_raw_type 1218 +#define invalid_double_starred_kvpairs_type 1219 +#define invalid_kvpair_type 1220 +#define _loop0_1_type 1221 +#define _loop0_2_type 1222 +#define _loop0_4_type 1223 +#define _gather_3_type 1224 +#define _loop0_6_type 1225 +#define _gather_5_type 1226 +#define _loop0_8_type 1227 +#define _gather_7_type 1228 +#define _loop0_10_type 1229 +#define _gather_9_type 1230 +#define _loop1_11_type 1231 +#define _loop0_13_type 1232 +#define _gather_12_type 1233 +#define _tmp_14_type 1234 +#define _tmp_15_type 1235 +#define _tmp_16_type 1236 +#define _tmp_17_type 1237 +#define _tmp_18_type 1238 +#define _tmp_19_type 1239 +#define _tmp_20_type 1240 +#define _tmp_21_type 1241 +#define _loop1_22_type 1242 +#define _tmp_23_type 1243 +#define _tmp_24_type 1244 +#define _loop0_26_type 1245 +#define _gather_25_type 1246 +#define _loop0_28_type 1247 +#define _gather_27_type 1248 +#define _tmp_29_type 1249 +#define _tmp_30_type 1250 +#define _loop0_31_type 1251 +#define _loop1_32_type 1252 +#define _loop0_34_type 1253 +#define _gather_33_type 1254 +#define _tmp_35_type 1255 +#define _loop0_37_type 1256 +#define _gather_36_type 1257 +#define _tmp_38_type 1258 +#define _loop0_40_type 1259 +#define _gather_39_type 1260 +#define _loop0_42_type 1261 +#define _gather_41_type 1262 +#define _loop0_44_type 1263 +#define _gather_43_type 1264 +#define _loop0_46_type 1265 +#define _gather_45_type 1266 +#define _tmp_47_type 1267 +#define _loop1_48_type 1268 +#define _tmp_49_type 1269 +#define _loop1_50_type 1270 +#define _loop0_52_type 1271 +#define _gather_51_type 1272 +#define _tmp_53_type 1273 +#define _tmp_54_type 1274 +#define _tmp_55_type 1275 +#define _tmp_56_type 1276 +#define _loop0_58_type 1277 +#define _gather_57_type 1278 +#define _loop0_60_type 1279 +#define _gather_59_type 1280 +#define _tmp_61_type 1281 +#define _loop0_63_type 1282 +#define _gather_62_type 1283 +#define _loop0_65_type 1284 +#define _gather_64_type 1285 +#define _tmp_66_type 1286 +#define _tmp_67_type 1287 +#define _tmp_68_type 1288 +#define _tmp_69_type 1289 +#define _loop0_70_type 1290 +#define _loop0_71_type 1291 +#define _loop0_72_type 1292 +#define _loop1_73_type 1293 +#define _loop0_74_type 1294 +#define _loop1_75_type 1295 +#define _loop1_76_type 1296 +#define _loop1_77_type 1297 +#define _loop0_78_type 1298 +#define _loop1_79_type 1299 +#define _loop0_80_type 1300 +#define _loop1_81_type 1301 +#define _loop0_82_type 1302 +#define _loop1_83_type 1303 +#define _loop1_84_type 1304 +#define _tmp_85_type 1305 +#define _loop1_86_type 1306 +#define _loop0_88_type 1307 +#define _gather_87_type 1308 +#define _loop1_89_type 1309 +#define _loop0_90_type 1310 +#define _loop0_91_type 1311 +#define _loop0_92_type 1312 +#define _loop1_93_type 1313 +#define _loop0_94_type 1314 +#define _loop1_95_type 1315 +#define _loop1_96_type 1316 +#define _loop1_97_type 1317 +#define _loop0_98_type 1318 +#define _loop1_99_type 1319 +#define _loop0_100_type 1320 +#define _loop1_101_type 1321 +#define _loop0_102_type 1322 +#define _loop1_103_type 1323 +#define _loop1_104_type 1324 +#define _loop1_105_type 1325 +#define _loop1_106_type 1326 +#define _tmp_107_type 1327 +#define _loop0_109_type 1328 +#define _gather_108_type 1329 +#define _tmp_110_type 1330 +#define _tmp_111_type 1331 +#define _tmp_112_type 1332 +#define _tmp_113_type 1333 +#define _loop1_114_type 1334 +#define _tmp_115_type 1335 +#define _tmp_116_type 1336 +#define _tmp_117_type 1337 +#define _loop0_119_type 1338 +#define _gather_118_type 1339 +#define _loop1_120_type 1340 +#define _loop0_121_type 1341 +#define _loop0_122_type 1342 +#define _loop0_124_type 1343 +#define _gather_123_type 1344 +#define _tmp_125_type 1345 +#define _loop0_127_type 1346 +#define _gather_126_type 1347 +#define _loop0_129_type 1348 +#define _gather_128_type 1349 +#define _loop0_131_type 1350 +#define _gather_130_type 1351 +#define _loop0_133_type 1352 +#define _gather_132_type 1353 +#define _loop0_134_type 1354 +#define _loop0_136_type 1355 +#define _gather_135_type 1356 +#define _loop1_137_type 1357 +#define _tmp_138_type 1358 +#define _loop0_140_type 1359 +#define _gather_139_type 1360 +#define _tmp_141_type 1361 +#define _tmp_142_type 1362 +#define _tmp_143_type 1363 +#define _tmp_144_type 1364 +#define _tmp_145_type 1365 +#define _tmp_146_type 1366 +#define _tmp_147_type 1367 +#define _tmp_148_type 1368 +#define _loop0_149_type 1369 +#define _loop0_150_type 1370 +#define _loop0_151_type 1371 +#define _tmp_152_type 1372 +#define _tmp_153_type 1373 +#define _tmp_154_type 1374 +#define _tmp_155_type 1375 +#define _loop0_156_type 1376 +#define _loop1_157_type 1377 +#define _loop0_158_type 1378 +#define _loop1_159_type 1379 +#define _tmp_160_type 1380 +#define _tmp_161_type 1381 +#define _tmp_162_type 1382 +#define _loop0_164_type 1383 +#define _gather_163_type 1384 +#define _loop0_166_type 1385 +#define _gather_165_type 1386 +#define _loop0_168_type 1387 +#define _gather_167_type 1388 +#define _loop0_170_type 1389 +#define _gather_169_type 1390 +#define _tmp_171_type 1391 +#define _tmp_172_type 1392 +#define _tmp_173_type 1393 +#define _tmp_174_type 1394 +#define _tmp_175_type 1395 +#define _tmp_176_type 1396 +#define _tmp_177_type 1397 +#define _loop0_179_type 1398 +#define _gather_178_type 1399 +#define _tmp_180_type 1400 +#define _tmp_181_type 1401 +#define _tmp_182_type 1402 +#define _tmp_183_type 1403 +#define _tmp_184_type 1404 +#define _tmp_185_type 1405 +#define _tmp_186_type 1406 +#define _tmp_187_type 1407 +#define _tmp_188_type 1408 +#define _tmp_189_type 1409 +#define _tmp_190_type 1410 +#define _tmp_191_type 1411 +#define _tmp_192_type 1412 +#define _tmp_193_type 1413 +#define _tmp_194_type 1414 +#define _tmp_195_type 1415 +#define _tmp_196_type 1416 +#define _tmp_197_type 1417 +#define _tmp_198_type 1418 +#define _tmp_199_type 1419 +#define _tmp_200_type 1420 +#define _tmp_201_type 1421 +#define _tmp_202_type 1422 +#define _tmp_203_type 1423 +#define _tmp_204_type 1424 +#define _tmp_205_type 1425 +#define _tmp_206_type 1426 static mod_ty file_rule(Parser *p); static mod_ty interactive_rule(Parser *p); @@ -684,7 +684,6 @@ static void *invalid_kwarg_rule(Parser *p); static expr_ty expression_without_invalid_rule(Parser *p); static void *invalid_legacy_expression_rule(Parser *p); static void *invalid_expression_rule(Parser *p); -static void *invalid_left_assignment_prefixes_rule(Parser *p); static void *invalid_named_expression_rule(Parser *p); static void *invalid_assignment_rule(Parser *p); static expr_ty invalid_ann_assign_target_rule(Parser *p); @@ -870,38 +869,38 @@ static void *_tmp_144_rule(Parser *p); static void *_tmp_145_rule(Parser *p); static void *_tmp_146_rule(Parser *p); static void *_tmp_147_rule(Parser *p); -static asdl_seq *_loop0_148_rule(Parser *p); +static void *_tmp_148_rule(Parser *p); static asdl_seq *_loop0_149_rule(Parser *p); static asdl_seq *_loop0_150_rule(Parser *p); -static void *_tmp_151_rule(Parser *p); +static asdl_seq *_loop0_151_rule(Parser *p); static void *_tmp_152_rule(Parser *p); static void *_tmp_153_rule(Parser *p); static void *_tmp_154_rule(Parser *p); -static asdl_seq *_loop0_155_rule(Parser *p); -static asdl_seq *_loop1_156_rule(Parser *p); -static asdl_seq *_loop0_157_rule(Parser *p); -static asdl_seq *_loop1_158_rule(Parser *p); -static void *_tmp_159_rule(Parser *p); +static void *_tmp_155_rule(Parser *p); +static asdl_seq *_loop0_156_rule(Parser *p); +static asdl_seq *_loop1_157_rule(Parser *p); +static asdl_seq *_loop0_158_rule(Parser *p); +static asdl_seq *_loop1_159_rule(Parser *p); static void *_tmp_160_rule(Parser *p); static void *_tmp_161_rule(Parser *p); -static asdl_seq *_loop0_163_rule(Parser *p); -static asdl_seq *_gather_162_rule(Parser *p); -static asdl_seq *_loop0_165_rule(Parser *p); -static asdl_seq *_gather_164_rule(Parser *p); -static asdl_seq *_loop0_167_rule(Parser *p); -static asdl_seq *_gather_166_rule(Parser *p); -static asdl_seq *_loop0_169_rule(Parser *p); -static asdl_seq *_gather_168_rule(Parser *p); -static void *_tmp_170_rule(Parser *p); +static void *_tmp_162_rule(Parser *p); +static asdl_seq *_loop0_164_rule(Parser *p); +static asdl_seq *_gather_163_rule(Parser *p); +static asdl_seq *_loop0_166_rule(Parser *p); +static asdl_seq *_gather_165_rule(Parser *p); +static asdl_seq *_loop0_168_rule(Parser *p); +static asdl_seq *_gather_167_rule(Parser *p); +static asdl_seq *_loop0_170_rule(Parser *p); +static asdl_seq *_gather_169_rule(Parser *p); static void *_tmp_171_rule(Parser *p); static void *_tmp_172_rule(Parser *p); static void *_tmp_173_rule(Parser *p); static void *_tmp_174_rule(Parser *p); static void *_tmp_175_rule(Parser *p); static void *_tmp_176_rule(Parser *p); -static asdl_seq *_loop0_178_rule(Parser *p); -static asdl_seq *_gather_177_rule(Parser *p); -static void *_tmp_179_rule(Parser *p); +static void *_tmp_177_rule(Parser *p); +static asdl_seq *_loop0_179_rule(Parser *p); +static asdl_seq *_gather_178_rule(Parser *p); static void *_tmp_180_rule(Parser *p); static void *_tmp_181_rule(Parser *p); static void *_tmp_182_rule(Parser *p); @@ -928,6 +927,7 @@ static void *_tmp_202_rule(Parser *p); static void *_tmp_203_rule(Parser *p); static void *_tmp_204_rule(Parser *p); static void *_tmp_205_rule(Parser *p); +static void *_tmp_206_rule(Parser *p); // file: statements? $ @@ -3843,9 +3843,7 @@ dotted_name_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = dotted_name_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -7134,9 +7132,7 @@ attr_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = attr_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -13025,9 +13021,7 @@ bitwise_or_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = bitwise_or_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -13151,9 +13145,7 @@ bitwise_xor_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = bitwise_xor_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -13277,9 +13269,7 @@ bitwise_and_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = bitwise_and_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -13403,9 +13393,7 @@ shift_expr_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = shift_expr_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -13568,9 +13556,7 @@ sum_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = sum_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -13739,9 +13725,7 @@ term_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = term_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -14363,9 +14347,7 @@ primary_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = primary_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -18053,9 +18035,7 @@ t_primary_rule(Parser *p) return _res; } p->mark = _mark; - p->in_raw_rule++; void *_raw = t_primary_raw(p); - p->in_raw_rule--; if (p->error_indicator) { p->level--; return NULL; @@ -18699,11 +18679,14 @@ invalid_kwarg_rule(Parser *p) static expr_ty expression_without_invalid_rule(Parser *p) { + int _prev_call_invalid = p->call_invalid_rules; + p->call_invalid_rules = 0; if (p->level++ == MAXSTACK) { p->error_indicator = 1; PyErr_NoMemory(); } if (p->error_indicator) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18711,6 +18694,7 @@ expression_without_invalid_rule(Parser *p) int _mark = p->mark; if (p->mark == p->fill && _PyPegen_fill_token(p) < 0) { p->error_indicator = 1; + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18720,6 +18704,7 @@ expression_without_invalid_rule(Parser *p) UNUSED(_start_col_offset); // Only used by EXTRA macro { // disjunction 'if' disjunction 'else' expression if (p->error_indicator) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18744,6 +18729,7 @@ expression_without_invalid_rule(Parser *p) D(fprintf(stderr, "%*c+ expression_without_invalid[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "disjunction 'if' disjunction 'else' expression")); Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); if (_token == NULL) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18754,6 +18740,7 @@ expression_without_invalid_rule(Parser *p) _res = _PyAST_IfExp ( b , a , c , EXTRA ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18765,6 +18752,7 @@ expression_without_invalid_rule(Parser *p) } { // disjunction if (p->error_indicator) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18784,6 +18772,7 @@ expression_without_invalid_rule(Parser *p) } { // lambdef if (p->error_indicator) { + p->call_invalid_rules = _prev_call_invalid; p->level--; return NULL; } @@ -18803,6 +18792,7 @@ expression_without_invalid_rule(Parser *p) } _res = NULL; done: + p->call_invalid_rules = _prev_call_invalid; p->level--; return _res; } @@ -18939,149 +18929,10 @@ invalid_expression_rule(Parser *p) return _res; } -// invalid_left_assignment_prefixes: list | tuple | genexp | 'True' | 'None' | 'False' -static void * -invalid_left_assignment_prefixes_rule(Parser *p) -{ - if (p->level++ == MAXSTACK) { - p->error_indicator = 1; - PyErr_NoMemory(); - } - if (p->error_indicator) { - p->level--; - return NULL; - } - void * _res = NULL; - if (_PyPegen_is_memoized(p, invalid_left_assignment_prefixes_type, &_res)) { - p->level--; - return _res; - } - int _mark = p->mark; - { // list - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "list")); - expr_ty list_var; - if ( - (list_var = list_rule(p)) // list - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "list")); - _res = list_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "list")); - } - { // tuple - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "tuple")); - expr_ty tuple_var; - if ( - (tuple_var = tuple_rule(p)) // tuple - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "tuple")); - _res = tuple_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "tuple")); - } - { // genexp - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "genexp")); - expr_ty genexp_var; - if ( - (genexp_var = genexp_rule(p)) // genexp - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "genexp")); - _res = genexp_var; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "genexp")); - } - { // 'True' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 524)) // token='True' - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'True'")); - } - { // 'None' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 523)) // token='None' - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'None'")); - } - { // 'False' - if (p->error_indicator) { - p->level--; - return NULL; - } - D(fprintf(stderr, "%*c> invalid_left_assignment_prefixes[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); - Token * _keyword; - if ( - (_keyword = _PyPegen_expect_token(p, 525)) // token='False' - ) - { - D(fprintf(stderr, "%*c+ invalid_left_assignment_prefixes[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); - _res = _keyword; - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s invalid_left_assignment_prefixes[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'False'")); - } - _res = NULL; - done: - _PyPegen_insert_memo(p, _mark, invalid_left_assignment_prefixes_type, _res); - p->level--; - return _res; -} - // invalid_named_expression: // | expression ':=' expression // | NAME '=' bitwise_or !('=' | ':=') -// | !invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=') +// | !(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=') static void * invalid_named_expression_rule(Parser *p) { @@ -19094,6 +18945,10 @@ invalid_named_expression_rule(Parser *p) return NULL; } void * _res = NULL; + if (_PyPegen_is_memoized(p, invalid_named_expression_type, &_res)) { + p->level--; + return _res; + } int _mark = p->mark; { // expression ':=' expression if (p->error_indicator) { @@ -19145,7 +19000,7 @@ invalid_named_expression_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "NAME '=' bitwise_or !('=' | ':=')")); - _res = p -> in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_RANGE ( a , b , "invalid syntax. Maybe you meant '==' or ':=' instead of '='?" ); + _res = RAISE_SYNTAX_ERROR_KNOWN_RANGE ( a , b , "invalid syntax. Maybe you meant '==' or ':=' instead of '='?" ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; p->level--; @@ -19157,17 +19012,17 @@ invalid_named_expression_rule(Parser *p) D(fprintf(stderr, "%*c%s invalid_named_expression[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "NAME '=' bitwise_or !('=' | ':=')")); } - { // !invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=') + { // !(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=') if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> invalid_named_expression[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); + D(fprintf(stderr, "%*c> invalid_named_expression[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); expr_ty a; Token * b; expr_ty bitwise_or_var; if ( - _PyPegen_lookahead(0, invalid_left_assignment_prefixes_rule, p) + _PyPegen_lookahead(0, _tmp_147_rule, p) && (a = bitwise_or_rule(p)) // bitwise_or && @@ -19175,11 +19030,11 @@ invalid_named_expression_rule(Parser *p) && (bitwise_or_var = bitwise_or_rule(p)) // bitwise_or && - _PyPegen_lookahead(0, _tmp_147_rule, p) + _PyPegen_lookahead(0, _tmp_148_rule, p) ) { - D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); - _res = p -> in_raw_rule ? NULL : RAISE_SYNTAX_ERROR_KNOWN_LOCATION ( a , "cannot assign to %s here. Maybe you meant '==' instead of '='?" , _PyPegen_get_expr_name ( a ) ); + D(fprintf(stderr, "%*c+ invalid_named_expression[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); + _res = RAISE_SYNTAX_ERROR_KNOWN_LOCATION ( a , "cannot assign to %s here. Maybe you meant '==' instead of '='?" , _PyPegen_get_expr_name ( a ) ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; p->level--; @@ -19189,10 +19044,11 @@ invalid_named_expression_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s invalid_named_expression[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "!invalid_left_assignment_prefixes bitwise_or '=' bitwise_or !('=' | ':=')")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "!(list | tuple | genexp | 'True' | 'None' | 'False') bitwise_or '=' bitwise_or !('=' | ':=')")); } _res = NULL; done: + _PyPegen_insert_memo(p, _mark, invalid_named_expression_type, _res); p->level--; return _res; } @@ -19255,7 +19111,7 @@ invalid_assignment_rule(Parser *p) D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expression ',' star_named_expressions* ':' expression")); Token * _literal; Token * _literal_1; - asdl_seq * _loop0_148_var; + asdl_seq * _loop0_149_var; expr_ty a; expr_ty expression_var; if ( @@ -19263,7 +19119,7 @@ invalid_assignment_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_loop0_148_var = _loop0_148_rule(p)) // star_named_expressions* + (_loop0_149_var = _loop0_149_rule(p)) // star_named_expressions* && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -19320,10 +19176,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* star_expressions '='")); Token * _literal; - asdl_seq * _loop0_149_var; + asdl_seq * _loop0_150_var; expr_ty a; if ( - (_loop0_149_var = _loop0_149_rule(p)) // ((star_targets '='))* + (_loop0_150_var = _loop0_150_rule(p)) // ((star_targets '='))* && (a = star_expressions_rule(p)) // star_expressions && @@ -19350,10 +19206,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* yield_expr '='")); Token * _literal; - asdl_seq * _loop0_150_var; + asdl_seq * _loop0_151_var; expr_ty a; if ( - (_loop0_150_var = _loop0_150_rule(p)) // ((star_targets '='))* + (_loop0_151_var = _loop0_151_rule(p)) // ((star_targets '='))* && (a = yield_expr_rule(p)) // yield_expr && @@ -19379,7 +19235,7 @@ invalid_assignment_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); - void *_tmp_151_var; + void *_tmp_152_var; expr_ty a; AugOperator* augassign_var; if ( @@ -19387,7 +19243,7 @@ invalid_assignment_rule(Parser *p) && (augassign_var = augassign_rule(p)) // augassign && - (_tmp_151_var = _tmp_151_rule(p)) // yield_expr | star_expressions + (_tmp_152_var = _tmp_152_rule(p)) // yield_expr | star_expressions ) { D(fprintf(stderr, "%*c+ invalid_assignment[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); @@ -19613,11 +19469,11 @@ invalid_comprehension_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '(' | '{') starred_expression for_if_clauses")); - void *_tmp_152_var; + void *_tmp_153_var; expr_ty a; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_152_var = _tmp_152_rule(p)) // '[' | '(' | '{' + (_tmp_153_var = _tmp_153_rule(p)) // '[' | '(' | '{' && (a = starred_expression_rule(p)) // starred_expression && @@ -19644,12 +19500,12 @@ invalid_comprehension_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '{') star_named_expression ',' star_named_expressions for_if_clauses")); Token * _literal; - void *_tmp_153_var; + void *_tmp_154_var; expr_ty a; asdl_expr_seq* b; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_153_var = _tmp_153_rule(p)) // '[' | '{' + (_tmp_154_var = _tmp_154_rule(p)) // '[' | '{' && (a = star_named_expression_rule(p)) // star_named_expression && @@ -19679,12 +19535,12 @@ invalid_comprehension_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '{') star_named_expression ',' for_if_clauses")); - void *_tmp_154_var; + void *_tmp_155_var; expr_ty a; Token * b; asdl_comprehension_seq* for_if_clauses_var; if ( - (_tmp_154_var = _tmp_154_rule(p)) // '[' | '{' + (_tmp_155_var = _tmp_155_rule(p)) // '[' | '{' && (a = star_named_expression_rule(p)) // star_named_expression && @@ -19788,11 +19644,11 @@ invalid_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default* invalid_parameters_helper param_no_default")); - asdl_seq * _loop0_155_var; + asdl_seq * _loop0_156_var; arg_ty a; void *invalid_parameters_helper_var; if ( - (_loop0_155_var = _loop0_155_rule(p)) // param_no_default* + (_loop0_156_var = _loop0_156_rule(p)) // param_no_default* && (invalid_parameters_helper_var = invalid_parameters_helper_rule(p)) // invalid_parameters_helper && @@ -19862,13 +19718,13 @@ invalid_parameters_helper_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters_helper[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - asdl_seq * _loop1_156_var; + asdl_seq * _loop1_157_var; if ( - (_loop1_156_var = _loop1_156_rule(p)) // param_with_default+ + (_loop1_157_var = _loop1_157_rule(p)) // param_with_default+ ) { D(fprintf(stderr, "%*c+ invalid_parameters_helper[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - _res = _loop1_156_var; + _res = _loop1_157_var; goto done; } p->mark = _mark; @@ -19902,11 +19758,11 @@ invalid_lambda_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default* invalid_lambda_parameters_helper lambda_param_no_default")); - asdl_seq * _loop0_157_var; + asdl_seq * _loop0_158_var; arg_ty a; void *invalid_lambda_parameters_helper_var; if ( - (_loop0_157_var = _loop0_157_rule(p)) // lambda_param_no_default* + (_loop0_158_var = _loop0_158_rule(p)) // lambda_param_no_default* && (invalid_lambda_parameters_helper_var = invalid_lambda_parameters_helper_rule(p)) // invalid_lambda_parameters_helper && @@ -19978,13 +19834,13 @@ invalid_lambda_parameters_helper_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters_helper[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - asdl_seq * _loop1_158_var; + asdl_seq * _loop1_159_var; if ( - (_loop1_158_var = _loop1_158_rule(p)) // lambda_param_with_default+ + (_loop1_159_var = _loop1_159_rule(p)) // lambda_param_with_default+ ) { D(fprintf(stderr, "%*c+ invalid_lambda_parameters_helper[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - _res = _loop1_158_var; + _res = _loop1_159_var; goto done; } p->mark = _mark; @@ -20017,12 +19873,12 @@ invalid_star_etc_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); - void *_tmp_159_var; + void *_tmp_160_var; Token * a; if ( (a = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_159_var = _tmp_159_rule(p)) // ')' | ',' (')' | '**') + (_tmp_160_var = _tmp_160_rule(p)) // ')' | ',' (')' | '**') ) { D(fprintf(stderr, "%*c+ invalid_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); @@ -20095,11 +19951,11 @@ invalid_lambda_star_etc_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_lambda_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); Token * _literal; - void *_tmp_160_var; + void *_tmp_161_var; if ( (_literal = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_160_var = _tmp_160_rule(p)) // ':' | ',' (':' | '**') + (_tmp_161_var = _tmp_161_rule(p)) // ':' | ',' (':' | '**') ) { D(fprintf(stderr, "%*c+ invalid_lambda_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); @@ -20207,7 +20063,7 @@ invalid_with_item_rule(Parser *p) && (a = expression_rule(p)) // expression && - _PyPegen_lookahead(1, _tmp_161_rule, p) + _PyPegen_lookahead(1, _tmp_162_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_with_item[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression 'as' expression &(',' | ')' | ':')")); @@ -20435,7 +20291,7 @@ invalid_with_stmt_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ &&':'")); - asdl_seq * _gather_162_var; + asdl_seq * _gather_163_var; Token * _keyword; Token * _literal; void *_opt_var; @@ -20445,13 +20301,13 @@ invalid_with_stmt_rule(Parser *p) && (_keyword = _PyPegen_expect_token(p, 519)) // token='with' && - (_gather_162_var = _gather_162_rule(p)) // ','.(expression ['as' star_target])+ + (_gather_163_var = _gather_163_rule(p)) // ','.(expression ['as' star_target])+ && (_literal = _PyPegen_expect_forced_token(p, 11, ":")) // forced_token=':' ) { D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ &&':'")); - _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _gather_162_var, _literal); + _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _gather_163_var, _literal); goto done; } p->mark = _mark; @@ -20464,7 +20320,7 @@ invalid_with_stmt_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' &&':'")); - asdl_seq * _gather_164_var; + asdl_seq * _gather_165_var; Token * _keyword; Token * _literal; Token * _literal_1; @@ -20480,7 +20336,7 @@ invalid_with_stmt_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_164_var = _gather_164_rule(p)) // ','.(expressions ['as' star_target])+ + (_gather_165_var = _gather_165_rule(p)) // ','.(expressions ['as' star_target])+ && (_opt_var_1 = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20490,7 +20346,7 @@ invalid_with_stmt_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' &&':'")); - _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _literal, _gather_164_var, _opt_var_1, _literal_1, _literal_2); + _res = _PyPegen_dummy_name(p, _opt_var, _keyword, _literal, _gather_165_var, _opt_var_1, _literal_1, _literal_2); goto done; } p->mark = _mark; @@ -20525,7 +20381,7 @@ invalid_with_stmt_indent_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' ','.(expression ['as' star_target])+ ':' NEWLINE !INDENT")); - asdl_seq * _gather_166_var; + asdl_seq * _gather_167_var; Token * _literal; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings @@ -20536,7 +20392,7 @@ invalid_with_stmt_indent_rule(Parser *p) && (a = _PyPegen_expect_token(p, 519)) // token='with' && - (_gather_166_var = _gather_166_rule(p)) // ','.(expression ['as' star_target])+ + (_gather_167_var = _gather_167_rule(p)) // ','.(expression ['as' star_target])+ && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -20564,7 +20420,7 @@ invalid_with_stmt_indent_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_with_stmt_indent[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC? 'with' '(' ','.(expressions ['as' star_target])+ ','? ')' ':' NEWLINE !INDENT")); - asdl_seq * _gather_168_var; + asdl_seq * _gather_169_var; Token * _literal; Token * _literal_1; Token * _literal_2; @@ -20581,7 +20437,7 @@ invalid_with_stmt_indent_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (_gather_168_var = _gather_168_rule(p)) // ','.(expressions ['as' star_target])+ + (_gather_169_var = _gather_169_rule(p)) // ','.(expressions ['as' star_target])+ && (_opt_var_1 = _PyPegen_expect_token(p, 12), !p->error_indicator) // ','? && @@ -20675,7 +20531,7 @@ invalid_try_stmt_rule(Parser *p) && (block_var = block_rule(p)) // block && - _PyPegen_lookahead(0, _tmp_170_rule, p) + _PyPegen_lookahead(0, _tmp_171_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_try_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'try' ':' block !('except' | 'finally')")); @@ -20736,7 +20592,7 @@ invalid_except_stmt_rule(Parser *p) && (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_171_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_172_rule(p), !p->error_indicator) // ['as' NAME] && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' ) @@ -20770,7 +20626,7 @@ invalid_except_stmt_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_172_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_173_rule(p), !p->error_indicator) // ['as' NAME] && (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) @@ -20906,7 +20762,7 @@ invalid_except_stmt_indent_rule(Parser *p) && (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_173_rule(p), !p->error_indicator) // ['as' NAME] + (_opt_var = _tmp_174_rule(p), !p->error_indicator) // ['as' NAME] && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21305,7 +21161,7 @@ invalid_class_argument_pattern_rule(Parser *p) asdl_pattern_seq* a; asdl_seq* keyword_patterns_var; if ( - (_opt_var = _tmp_174_rule(p), !p->error_indicator) // [positional_patterns ','] + (_opt_var = _tmp_175_rule(p), !p->error_indicator) // [positional_patterns ','] && (keyword_patterns_var = keyword_patterns_rule(p)) // keyword_patterns && @@ -21757,7 +21613,7 @@ invalid_def_raw_rule(Parser *p) && (_literal_1 = _PyPegen_expect_token(p, 8)) // token=')' && - (_opt_var_2 = _tmp_175_rule(p), !p->error_indicator) // ['->' expression] + (_opt_var_2 = _tmp_176_rule(p), !p->error_indicator) // ['->' expression] && (_literal_2 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21816,7 +21672,7 @@ invalid_class_def_raw_rule(Parser *p) && (name_var = _PyPegen_name_token(p)) // NAME && - (_opt_var = _tmp_176_rule(p), !p->error_indicator) // ['(' arguments? ')'] + (_opt_var = _tmp_177_rule(p), !p->error_indicator) // ['(' arguments? ')'] && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -21867,11 +21723,11 @@ invalid_double_starred_kvpairs_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_double_starred_kvpairs[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.double_starred_kvpair+ ',' invalid_kvpair")); - asdl_seq * _gather_177_var; + asdl_seq * _gather_178_var; Token * _literal; void *invalid_kvpair_var; if ( - (_gather_177_var = _gather_177_rule(p)) // ','.double_starred_kvpair+ + (_gather_178_var = _gather_178_rule(p)) // ','.double_starred_kvpair+ && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && @@ -21879,7 +21735,7 @@ invalid_double_starred_kvpairs_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ invalid_double_starred_kvpairs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.double_starred_kvpair+ ',' invalid_kvpair")); - _res = _PyPegen_dummy_name(p, _gather_177_var, _literal, invalid_kvpair_var); + _res = _PyPegen_dummy_name(p, _gather_178_var, _literal, invalid_kvpair_var); goto done; } p->mark = _mark; @@ -21932,7 +21788,7 @@ invalid_double_starred_kvpairs_rule(Parser *p) && (a = _PyPegen_expect_token(p, 11)) // token=':' && - _PyPegen_lookahead(1, _tmp_179_rule, p) + _PyPegen_lookahead(1, _tmp_180_rule, p) ) { D(fprintf(stderr, "%*c+ invalid_double_starred_kvpairs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ':' &('}' | ',')")); @@ -23374,12 +23230,12 @@ _loop1_22_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_22[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_180_var; + void *_tmp_181_var; while ( - (_tmp_180_var = _tmp_180_rule(p)) // star_targets '=' + (_tmp_181_var = _tmp_181_rule(p)) // star_targets '=' ) { - _res = _tmp_180_var; + _res = _tmp_181_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23909,12 +23765,12 @@ _loop0_31_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_31[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_181_var; + void *_tmp_182_var; while ( - (_tmp_181_var = _tmp_181_rule(p)) // '.' | '...' + (_tmp_182_var = _tmp_182_rule(p)) // '.' | '...' ) { - _res = _tmp_181_var; + _res = _tmp_182_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23978,12 +23834,12 @@ _loop1_32_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_32[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_182_var; + void *_tmp_183_var; while ( - (_tmp_182_var = _tmp_182_rule(p)) // '.' | '...' + (_tmp_183_var = _tmp_183_rule(p)) // '.' | '...' ) { - _res = _tmp_182_var; + _res = _tmp_183_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27250,12 +27106,12 @@ _loop1_84_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_84[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('@' named_expression NEWLINE)")); - void *_tmp_183_var; + void *_tmp_184_var; while ( - (_tmp_183_var = _tmp_183_rule(p)) // '@' named_expression NEWLINE + (_tmp_184_var = _tmp_184_rule(p)) // '@' named_expression NEWLINE ) { - _res = _tmp_183_var; + _res = _tmp_184_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27374,12 +27230,12 @@ _loop1_86_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_86[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_expression)")); - void *_tmp_184_var; + void *_tmp_185_var; while ( - (_tmp_184_var = _tmp_184_rule(p)) // ',' star_expression + (_tmp_185_var = _tmp_185_rule(p)) // ',' star_expression ) { - _res = _tmp_184_var; + _res = _tmp_185_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -27568,12 +27424,12 @@ _loop1_89_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_89[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' expression)")); - void *_tmp_185_var; + void *_tmp_186_var; while ( - (_tmp_185_var = _tmp_185_rule(p)) // ',' expression + (_tmp_186_var = _tmp_186_rule(p)) // ',' expression ) { - _res = _tmp_185_var; + _res = _tmp_186_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -28643,12 +28499,12 @@ _loop1_104_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_104[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('or' conjunction)")); - void *_tmp_186_var; + void *_tmp_187_var; while ( - (_tmp_186_var = _tmp_186_rule(p)) // 'or' conjunction + (_tmp_187_var = _tmp_187_rule(p)) // 'or' conjunction ) { - _res = _tmp_186_var; + _res = _tmp_187_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -28717,12 +28573,12 @@ _loop1_105_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_105[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('and' inversion)")); - void *_tmp_187_var; + void *_tmp_188_var; while ( - (_tmp_187_var = _tmp_187_rule(p)) // 'and' inversion + (_tmp_188_var = _tmp_188_rule(p)) // 'and' inversion ) { - _res = _tmp_187_var; + _res = _tmp_188_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29743,12 +29599,12 @@ _loop0_121_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_121[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_188_var; + void *_tmp_189_var; while ( - (_tmp_188_var = _tmp_188_rule(p)) // 'if' disjunction + (_tmp_189_var = _tmp_189_rule(p)) // 'if' disjunction ) { - _res = _tmp_188_var; + _res = _tmp_189_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29812,12 +29668,12 @@ _loop0_122_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_122[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_189_var; + void *_tmp_190_var; while ( - (_tmp_189_var = _tmp_189_rule(p)) // 'if' disjunction + (_tmp_190_var = _tmp_190_rule(p)) // 'if' disjunction ) { - _res = _tmp_189_var; + _res = _tmp_190_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -29886,7 +29742,7 @@ _loop0_124_rule(Parser *p) while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_190_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' + (elem = _tmp_191_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' ) { _res = elem; @@ -29953,7 +29809,7 @@ _gather_123_rule(Parser *p) void *elem; asdl_seq * seq; if ( - (elem = _tmp_190_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' + (elem = _tmp_191_rule(p)) // starred_expression | (assignment_expression | expression !':=') !'=' && (seq = _loop0_124_rule(p)) // _loop0_124 ) @@ -30529,12 +30385,12 @@ _loop0_134_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_134[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); - void *_tmp_191_var; + void *_tmp_192_var; while ( - (_tmp_191_var = _tmp_191_rule(p)) // ',' star_target + (_tmp_192_var = _tmp_192_rule(p)) // ',' star_target ) { - _res = _tmp_191_var; + _res = _tmp_192_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -30718,12 +30574,12 @@ _loop1_137_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_137[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); - void *_tmp_192_var; + void *_tmp_193_var; while ( - (_tmp_192_var = _tmp_192_rule(p)) // ',' star_target + (_tmp_193_var = _tmp_193_rule(p)) // ',' star_target ) { - _res = _tmp_192_var; + _res = _tmp_193_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -31280,9 +31136,143 @@ _tmp_146_rule(Parser *p) return _res; } -// _tmp_147: '=' | ':=' +// _tmp_147: list | tuple | genexp | 'True' | 'None' | 'False' static void * _tmp_147_rule(Parser *p) +{ + if (p->level++ == MAXSTACK) { + p->error_indicator = 1; + PyErr_NoMemory(); + } + if (p->error_indicator) { + p->level--; + return NULL; + } + void * _res = NULL; + int _mark = p->mark; + { // list + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "list")); + expr_ty list_var; + if ( + (list_var = list_rule(p)) // list + ) + { + D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "list")); + _res = list_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "list")); + } + { // tuple + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "tuple")); + expr_ty tuple_var; + if ( + (tuple_var = tuple_rule(p)) // tuple + ) + { + D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "tuple")); + _res = tuple_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "tuple")); + } + { // genexp + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "genexp")); + expr_ty genexp_var; + if ( + (genexp_var = genexp_rule(p)) // genexp + ) + { + D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "genexp")); + _res = genexp_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "genexp")); + } + { // 'True' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'True'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 524)) // token='True' + ) + { + D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'True'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'True'")); + } + { // 'None' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'None'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 523)) // token='None' + ) + { + D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'None'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'None'")); + } + { // 'False' + if (p->error_indicator) { + p->level--; + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'False'")); + Token * _keyword; + if ( + (_keyword = _PyPegen_expect_token(p, 525)) // token='False' + ) + { + D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'False'")); + _res = _keyword; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'False'")); + } + _res = NULL; + done: + p->level--; + return _res; +} + +// _tmp_148: '=' | ':=' +static void * +_tmp_148_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31299,18 +31289,18 @@ _tmp_147_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'='")); + D(fprintf(stderr, "%*c> _tmp_148[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'='")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'='")); + D(fprintf(stderr, "%*c+ _tmp_148[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'='")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_148[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'='")); } { // ':=' @@ -31318,18 +31308,18 @@ _tmp_147_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':='")); + D(fprintf(stderr, "%*c> _tmp_148[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':='")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 53)) // token=':=' ) { - D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':='")); + D(fprintf(stderr, "%*c+ _tmp_148[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':='")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_148[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':='")); } _res = NULL; @@ -31338,9 +31328,9 @@ _tmp_147_rule(Parser *p) return _res; } -// _loop0_148: star_named_expressions +// _loop0_149: star_named_expressions static asdl_seq * -_loop0_148_rule(Parser *p) +_loop0_149_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31367,7 +31357,7 @@ _loop0_148_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_148[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); + D(fprintf(stderr, "%*c> _loop0_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); asdl_expr_seq* star_named_expressions_var; while ( (star_named_expressions_var = star_named_expressions_rule(p)) // star_named_expressions @@ -31389,7 +31379,7 @@ _loop0_148_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_148[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_149[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_named_expressions")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31402,14 +31392,14 @@ _loop0_148_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_148_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_149_type, _seq); p->level--; return _seq; } -// _loop0_149: (star_targets '=') +// _loop0_150: (star_targets '=') static asdl_seq * -_loop0_149_rule(Parser *p) +_loop0_150_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31436,13 +31426,13 @@ _loop0_149_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_193_var; + D(fprintf(stderr, "%*c> _loop0_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_194_var; while ( - (_tmp_193_var = _tmp_193_rule(p)) // star_targets '=' + (_tmp_194_var = _tmp_194_rule(p)) // star_targets '=' ) { - _res = _tmp_193_var; + _res = _tmp_194_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -31458,7 +31448,7 @@ _loop0_149_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_149[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_150[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31471,14 +31461,14 @@ _loop0_149_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_149_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_150_type, _seq); p->level--; return _seq; } -// _loop0_150: (star_targets '=') +// _loop0_151: (star_targets '=') static asdl_seq * -_loop0_150_rule(Parser *p) +_loop0_151_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31505,13 +31495,13 @@ _loop0_150_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_194_var; + D(fprintf(stderr, "%*c> _loop0_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_195_var; while ( - (_tmp_194_var = _tmp_194_rule(p)) // star_targets '=' + (_tmp_195_var = _tmp_195_rule(p)) // star_targets '=' ) { - _res = _tmp_194_var; + _res = _tmp_195_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -31527,7 +31517,7 @@ _loop0_150_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_150[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_151[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31540,14 +31530,14 @@ _loop0_150_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_150_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_151_type, _seq); p->level--; return _seq; } -// _tmp_151: yield_expr | star_expressions +// _tmp_152: yield_expr | star_expressions static void * -_tmp_151_rule(Parser *p) +_tmp_152_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31564,18 +31554,18 @@ _tmp_151_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); expr_ty yield_expr_var; if ( (yield_expr_var = yield_expr_rule(p)) // yield_expr ) { - D(fprintf(stderr, "%*c+ _tmp_151[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); _res = yield_expr_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_151[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "yield_expr")); } { // star_expressions @@ -31583,18 +31573,18 @@ _tmp_151_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); expr_ty star_expressions_var; if ( (star_expressions_var = star_expressions_rule(p)) // star_expressions ) { - D(fprintf(stderr, "%*c+ _tmp_151[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); _res = star_expressions_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_151[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_expressions")); } _res = NULL; @@ -31603,9 +31593,9 @@ _tmp_151_rule(Parser *p) return _res; } -// _tmp_152: '[' | '(' | '{' +// _tmp_153: '[' | '(' | '{' static void * -_tmp_152_rule(Parser *p) +_tmp_153_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31622,18 +31612,18 @@ _tmp_152_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '(' @@ -31641,18 +31631,18 @@ _tmp_152_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 7)) // token='(' ) { - D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'('")); } { // '{' @@ -31660,18 +31650,18 @@ _tmp_152_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -31680,9 +31670,9 @@ _tmp_152_rule(Parser *p) return _res; } -// _tmp_153: '[' | '{' +// _tmp_154: '[' | '{' static void * -_tmp_153_rule(Parser *p) +_tmp_154_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31699,18 +31689,18 @@ _tmp_153_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '{' @@ -31718,18 +31708,18 @@ _tmp_153_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -31738,9 +31728,9 @@ _tmp_153_rule(Parser *p) return _res; } -// _tmp_154: '[' | '{' +// _tmp_155: '[' | '{' static void * -_tmp_154_rule(Parser *p) +_tmp_155_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31757,18 +31747,18 @@ _tmp_154_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '{' @@ -31776,18 +31766,18 @@ _tmp_154_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -31796,9 +31786,9 @@ _tmp_154_rule(Parser *p) return _res; } -// _loop0_155: param_no_default +// _loop0_156: param_no_default static asdl_seq * -_loop0_155_rule(Parser *p) +_loop0_156_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31825,7 +31815,7 @@ _loop0_155_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); + D(fprintf(stderr, "%*c> _loop0_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); arg_ty param_no_default_var; while ( (param_no_default_var = param_no_default_rule(p)) // param_no_default @@ -31847,7 +31837,7 @@ _loop0_155_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_155[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_156[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -31860,14 +31850,14 @@ _loop0_155_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_155_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_156_type, _seq); p->level--; return _seq; } -// _loop1_156: param_with_default +// _loop1_157: param_with_default static asdl_seq * -_loop1_156_rule(Parser *p) +_loop1_157_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31894,7 +31884,7 @@ _loop1_156_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); + D(fprintf(stderr, "%*c> _loop1_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); NameDefaultPair* param_with_default_var; while ( (param_with_default_var = param_with_default_rule(p)) // param_with_default @@ -31916,7 +31906,7 @@ _loop1_156_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_156[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_157[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -31934,14 +31924,14 @@ _loop1_156_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_156_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_157_type, _seq); p->level--; return _seq; } -// _loop0_157: lambda_param_no_default +// _loop0_158: lambda_param_no_default static asdl_seq * -_loop0_157_rule(Parser *p) +_loop0_158_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -31968,7 +31958,7 @@ _loop0_157_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_157[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); + D(fprintf(stderr, "%*c> _loop0_158[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); arg_ty lambda_param_no_default_var; while ( (lambda_param_no_default_var = lambda_param_no_default_rule(p)) // lambda_param_no_default @@ -31990,7 +31980,7 @@ _loop0_157_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_157[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_158[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32003,14 +31993,14 @@ _loop0_157_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_157_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_158_type, _seq); p->level--; return _seq; } -// _loop1_158: lambda_param_with_default +// _loop1_159: lambda_param_with_default static asdl_seq * -_loop1_158_rule(Parser *p) +_loop1_159_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32037,7 +32027,7 @@ _loop1_158_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop1_158[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); + D(fprintf(stderr, "%*c> _loop1_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); NameDefaultPair* lambda_param_with_default_var; while ( (lambda_param_with_default_var = lambda_param_with_default_rule(p)) // lambda_param_with_default @@ -32059,7 +32049,7 @@ _loop1_158_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_158[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_159[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -32077,14 +32067,14 @@ _loop1_158_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_158_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_159_type, _seq); p->level--; return _seq; } -// _tmp_159: ')' | ',' (')' | '**') +// _tmp_160: ')' | ',' (')' | '**') static void * -_tmp_159_rule(Parser *p) +_tmp_160_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32101,18 +32091,18 @@ _tmp_159_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_159[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_160[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_159[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_160[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // ',' (')' | '**') @@ -32120,21 +32110,21 @@ _tmp_159_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_159[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + D(fprintf(stderr, "%*c> _tmp_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); Token * _literal; - void *_tmp_195_var; + void *_tmp_196_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_195_var = _tmp_195_rule(p)) // ')' | '**' + (_tmp_196_var = _tmp_196_rule(p)) // ')' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_159[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_195_var); + D(fprintf(stderr, "%*c+ _tmp_160[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_196_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_159[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_160[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (')' | '**')")); } _res = NULL; @@ -32143,9 +32133,9 @@ _tmp_159_rule(Parser *p) return _res; } -// _tmp_160: ':' | ',' (':' | '**') +// _tmp_161: ':' | ',' (':' | '**') static void * -_tmp_160_rule(Parser *p) +_tmp_161_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32162,18 +32152,18 @@ _tmp_160_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_160[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_160[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // ',' (':' | '**') @@ -32181,21 +32171,21 @@ _tmp_160_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_160[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); Token * _literal; - void *_tmp_196_var; + void *_tmp_197_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_196_var = _tmp_196_rule(p)) // ':' | '**' + (_tmp_197_var = _tmp_197_rule(p)) // ':' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_160[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_196_var); + D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_197_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_160[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (':' | '**')")); } _res = NULL; @@ -32204,9 +32194,9 @@ _tmp_160_rule(Parser *p) return _res; } -// _tmp_161: ',' | ')' | ':' +// _tmp_162: ',' | ')' | ':' static void * -_tmp_161_rule(Parser *p) +_tmp_162_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32223,18 +32213,18 @@ _tmp_161_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c> _tmp_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c+ _tmp_162[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_162[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "','")); } { // ')' @@ -32242,18 +32232,18 @@ _tmp_161_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_162[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_162[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // ':' @@ -32261,18 +32251,18 @@ _tmp_161_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_161[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_161[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_162[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_161[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_162[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } _res = NULL; @@ -32281,9 +32271,9 @@ _tmp_161_rule(Parser *p) return _res; } -// _loop0_163: ',' (expression ['as' star_target]) +// _loop0_164: ',' (expression ['as' star_target]) static asdl_seq * -_loop0_163_rule(Parser *p) +_loop0_164_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32310,13 +32300,13 @@ _loop0_163_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_163[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_164[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_197_rule(p)) // expression ['as' star_target] + (elem = _tmp_198_rule(p)) // expression ['as' star_target] ) { _res = elem; @@ -32341,7 +32331,7 @@ _loop0_163_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_163[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_164[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expression ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32354,14 +32344,14 @@ _loop0_163_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_163_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_164_type, _seq); p->level--; return _seq; } -// _gather_162: (expression ['as' star_target]) _loop0_163 +// _gather_163: (expression ['as' star_target]) _loop0_164 static asdl_seq * -_gather_162_rule(Parser *p) +_gather_163_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32373,27 +32363,27 @@ _gather_162_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expression ['as' star_target]) _loop0_163 + { // (expression ['as' star_target]) _loop0_164 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_162[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_163")); + D(fprintf(stderr, "%*c> _gather_163[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_164")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_197_rule(p)) // expression ['as' star_target] + (elem = _tmp_198_rule(p)) // expression ['as' star_target] && - (seq = _loop0_163_rule(p)) // _loop0_163 + (seq = _loop0_164_rule(p)) // _loop0_164 ) { - D(fprintf(stderr, "%*c+ _gather_162[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_163")); + D(fprintf(stderr, "%*c+ _gather_163[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_164")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_162[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_163")); + D(fprintf(stderr, "%*c%s _gather_163[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_164")); } _res = NULL; done: @@ -32401,9 +32391,9 @@ _gather_162_rule(Parser *p) return _res; } -// _loop0_165: ',' (expressions ['as' star_target]) +// _loop0_166: ',' (expressions ['as' star_target]) static asdl_seq * -_loop0_165_rule(Parser *p) +_loop0_166_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32430,13 +32420,13 @@ _loop0_165_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_165[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_198_rule(p)) // expressions ['as' star_target] + (elem = _tmp_199_rule(p)) // expressions ['as' star_target] ) { _res = elem; @@ -32461,7 +32451,7 @@ _loop0_165_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_165[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_166[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expressions ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32474,14 +32464,14 @@ _loop0_165_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_165_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_166_type, _seq); p->level--; return _seq; } -// _gather_164: (expressions ['as' star_target]) _loop0_165 +// _gather_165: (expressions ['as' star_target]) _loop0_166 static asdl_seq * -_gather_164_rule(Parser *p) +_gather_165_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32493,27 +32483,27 @@ _gather_164_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expressions ['as' star_target]) _loop0_165 + { // (expressions ['as' star_target]) _loop0_166 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_164[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_165")); + D(fprintf(stderr, "%*c> _gather_165[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_166")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_198_rule(p)) // expressions ['as' star_target] + (elem = _tmp_199_rule(p)) // expressions ['as' star_target] && - (seq = _loop0_165_rule(p)) // _loop0_165 + (seq = _loop0_166_rule(p)) // _loop0_166 ) { - D(fprintf(stderr, "%*c+ _gather_164[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_165")); + D(fprintf(stderr, "%*c+ _gather_165[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_166")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_164[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_165")); + D(fprintf(stderr, "%*c%s _gather_165[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_166")); } _res = NULL; done: @@ -32521,9 +32511,9 @@ _gather_164_rule(Parser *p) return _res; } -// _loop0_167: ',' (expression ['as' star_target]) +// _loop0_168: ',' (expression ['as' star_target]) static asdl_seq * -_loop0_167_rule(Parser *p) +_loop0_168_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32550,13 +32540,13 @@ _loop0_167_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expression ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_199_rule(p)) // expression ['as' star_target] + (elem = _tmp_200_rule(p)) // expression ['as' star_target] ) { _res = elem; @@ -32581,7 +32571,7 @@ _loop0_167_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_167[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_168[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expression ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32594,14 +32584,14 @@ _loop0_167_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_167_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_168_type, _seq); p->level--; return _seq; } -// _gather_166: (expression ['as' star_target]) _loop0_167 +// _gather_167: (expression ['as' star_target]) _loop0_168 static asdl_seq * -_gather_166_rule(Parser *p) +_gather_167_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32613,27 +32603,27 @@ _gather_166_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expression ['as' star_target]) _loop0_167 + { // (expression ['as' star_target]) _loop0_168 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_166[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_167")); + D(fprintf(stderr, "%*c> _gather_167[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_168")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_199_rule(p)) // expression ['as' star_target] + (elem = _tmp_200_rule(p)) // expression ['as' star_target] && - (seq = _loop0_167_rule(p)) // _loop0_167 + (seq = _loop0_168_rule(p)) // _loop0_168 ) { - D(fprintf(stderr, "%*c+ _gather_166[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_167")); + D(fprintf(stderr, "%*c+ _gather_167[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expression ['as' star_target]) _loop0_168")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_166[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_167")); + D(fprintf(stderr, "%*c%s _gather_167[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expression ['as' star_target]) _loop0_168")); } _res = NULL; done: @@ -32641,9 +32631,9 @@ _gather_166_rule(Parser *p) return _res; } -// _loop0_169: ',' (expressions ['as' star_target]) +// _loop0_170: ',' (expressions ['as' star_target]) static asdl_seq * -_loop0_169_rule(Parser *p) +_loop0_170_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32670,13 +32660,13 @@ _loop0_169_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); + D(fprintf(stderr, "%*c> _loop0_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (expressions ['as' star_target])")); Token * _literal; void *elem; while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (elem = _tmp_200_rule(p)) // expressions ['as' star_target] + (elem = _tmp_201_rule(p)) // expressions ['as' star_target] ) { _res = elem; @@ -32701,7 +32691,7 @@ _loop0_169_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_169[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_170[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (expressions ['as' star_target])")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -32714,14 +32704,14 @@ _loop0_169_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_169_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_170_type, _seq); p->level--; return _seq; } -// _gather_168: (expressions ['as' star_target]) _loop0_169 +// _gather_169: (expressions ['as' star_target]) _loop0_170 static asdl_seq * -_gather_168_rule(Parser *p) +_gather_169_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32733,27 +32723,27 @@ _gather_168_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // (expressions ['as' star_target]) _loop0_169 + { // (expressions ['as' star_target]) _loop0_170 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_168[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_169")); + D(fprintf(stderr, "%*c> _gather_169[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_170")); void *elem; asdl_seq * seq; if ( - (elem = _tmp_200_rule(p)) // expressions ['as' star_target] + (elem = _tmp_201_rule(p)) // expressions ['as' star_target] && - (seq = _loop0_169_rule(p)) // _loop0_169 + (seq = _loop0_170_rule(p)) // _loop0_170 ) { - D(fprintf(stderr, "%*c+ _gather_168[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_169")); + D(fprintf(stderr, "%*c+ _gather_169[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(expressions ['as' star_target]) _loop0_170")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_168[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_169")); + D(fprintf(stderr, "%*c%s _gather_169[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(expressions ['as' star_target]) _loop0_170")); } _res = NULL; done: @@ -32761,9 +32751,9 @@ _gather_168_rule(Parser *p) return _res; } -// _tmp_170: 'except' | 'finally' +// _tmp_171: 'except' | 'finally' static void * -_tmp_170_rule(Parser *p) +_tmp_171_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32780,18 +32770,18 @@ _tmp_170_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except'")); + D(fprintf(stderr, "%*c> _tmp_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except'")); Token * _keyword; if ( (_keyword = _PyPegen_expect_token(p, 521)) // token='except' ) { - D(fprintf(stderr, "%*c+ _tmp_170[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except'")); + D(fprintf(stderr, "%*c+ _tmp_171[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'except'")); _res = _keyword; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_170[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_171[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'except'")); } { // 'finally' @@ -32799,18 +32789,18 @@ _tmp_170_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_170[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally'")); + D(fprintf(stderr, "%*c> _tmp_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally'")); Token * _keyword; if ( (_keyword = _PyPegen_expect_token(p, 522)) // token='finally' ) { - D(fprintf(stderr, "%*c+ _tmp_170[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'finally'")); + D(fprintf(stderr, "%*c+ _tmp_171[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'finally'")); _res = _keyword; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_170[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_171[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'finally'")); } _res = NULL; @@ -32819,9 +32809,9 @@ _tmp_170_rule(Parser *p) return _res; } -// _tmp_171: 'as' NAME +// _tmp_172: 'as' NAME static void * -_tmp_171_rule(Parser *p) +_tmp_172_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32838,7 +32828,7 @@ _tmp_171_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_171[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_172[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -32847,12 +32837,12 @@ _tmp_171_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_171[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_172[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_171[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_172[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -32861,9 +32851,9 @@ _tmp_171_rule(Parser *p) return _res; } -// _tmp_172: 'as' NAME +// _tmp_173: 'as' NAME static void * -_tmp_172_rule(Parser *p) +_tmp_173_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32880,7 +32870,7 @@ _tmp_172_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_172[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_173[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -32889,12 +32879,12 @@ _tmp_172_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_172[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_173[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_172[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_173[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -32903,9 +32893,9 @@ _tmp_172_rule(Parser *p) return _res; } -// _tmp_173: 'as' NAME +// _tmp_174: 'as' NAME static void * -_tmp_173_rule(Parser *p) +_tmp_174_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32922,7 +32912,7 @@ _tmp_173_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_173[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c> _tmp_174[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' NAME")); Token * _keyword; expr_ty name_var; if ( @@ -32931,12 +32921,12 @@ _tmp_173_rule(Parser *p) (name_var = _PyPegen_name_token(p)) // NAME ) { - D(fprintf(stderr, "%*c+ _tmp_173[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); + D(fprintf(stderr, "%*c+ _tmp_174[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' NAME")); _res = _PyPegen_dummy_name(p, _keyword, name_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_173[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_174[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' NAME")); } _res = NULL; @@ -32945,9 +32935,9 @@ _tmp_173_rule(Parser *p) return _res; } -// _tmp_174: positional_patterns ',' +// _tmp_175: positional_patterns ',' static void * -_tmp_174_rule(Parser *p) +_tmp_175_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -32964,7 +32954,7 @@ _tmp_174_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_174[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); + D(fprintf(stderr, "%*c> _tmp_175[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); Token * _literal; asdl_pattern_seq* positional_patterns_var; if ( @@ -32973,12 +32963,12 @@ _tmp_174_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_174[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); + D(fprintf(stderr, "%*c+ _tmp_175[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "positional_patterns ','")); _res = _PyPegen_dummy_name(p, positional_patterns_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_174[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_175[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "positional_patterns ','")); } _res = NULL; @@ -32987,9 +32977,9 @@ _tmp_174_rule(Parser *p) return _res; } -// _tmp_175: '->' expression +// _tmp_176: '->' expression static void * -_tmp_175_rule(Parser *p) +_tmp_176_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33006,7 +32996,7 @@ _tmp_175_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_175[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'->' expression")); + D(fprintf(stderr, "%*c> _tmp_176[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'->' expression")); Token * _literal; expr_ty expression_var; if ( @@ -33015,12 +33005,12 @@ _tmp_175_rule(Parser *p) (expression_var = expression_rule(p)) // expression ) { - D(fprintf(stderr, "%*c+ _tmp_175[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'->' expression")); + D(fprintf(stderr, "%*c+ _tmp_176[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'->' expression")); _res = _PyPegen_dummy_name(p, _literal, expression_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_175[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_176[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'->' expression")); } _res = NULL; @@ -33029,9 +33019,9 @@ _tmp_175_rule(Parser *p) return _res; } -// _tmp_176: '(' arguments? ')' +// _tmp_177: '(' arguments? ')' static void * -_tmp_176_rule(Parser *p) +_tmp_177_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33048,7 +33038,7 @@ _tmp_176_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_176[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); + D(fprintf(stderr, "%*c> _tmp_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); Token * _literal; Token * _literal_1; void *_opt_var; @@ -33061,12 +33051,12 @@ _tmp_176_rule(Parser *p) (_literal_1 = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_176[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); + D(fprintf(stderr, "%*c+ _tmp_177[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'(' arguments? ')'")); _res = _PyPegen_dummy_name(p, _literal, _opt_var, _literal_1); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_176[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_177[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'(' arguments? ')'")); } _res = NULL; @@ -33075,9 +33065,9 @@ _tmp_176_rule(Parser *p) return _res; } -// _loop0_178: ',' double_starred_kvpair +// _loop0_179: ',' double_starred_kvpair static asdl_seq * -_loop0_178_rule(Parser *p) +_loop0_179_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33104,7 +33094,7 @@ _loop0_178_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _loop0_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' double_starred_kvpair")); + D(fprintf(stderr, "%*c> _loop0_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' double_starred_kvpair")); Token * _literal; KeyValuePair* elem; while ( @@ -33135,7 +33125,7 @@ _loop0_178_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_178[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_179[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' double_starred_kvpair")); } asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); @@ -33148,14 +33138,14 @@ _loop0_178_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_178_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_179_type, _seq); p->level--; return _seq; } -// _gather_177: double_starred_kvpair _loop0_178 +// _gather_178: double_starred_kvpair _loop0_179 static asdl_seq * -_gather_177_rule(Parser *p) +_gather_178_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33167,27 +33157,27 @@ _gather_177_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // double_starred_kvpair _loop0_178 + { // double_starred_kvpair _loop0_179 if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> _gather_177[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_178")); + D(fprintf(stderr, "%*c> _gather_178[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_179")); KeyValuePair* elem; asdl_seq * seq; if ( (elem = double_starred_kvpair_rule(p)) // double_starred_kvpair && - (seq = _loop0_178_rule(p)) // _loop0_178 + (seq = _loop0_179_rule(p)) // _loop0_179 ) { - D(fprintf(stderr, "%*c+ _gather_177[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_178")); + D(fprintf(stderr, "%*c+ _gather_178[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "double_starred_kvpair _loop0_179")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_177[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "double_starred_kvpair _loop0_178")); + D(fprintf(stderr, "%*c%s _gather_178[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "double_starred_kvpair _loop0_179")); } _res = NULL; done: @@ -33195,9 +33185,9 @@ _gather_177_rule(Parser *p) return _res; } -// _tmp_179: '}' | ',' +// _tmp_180: '}' | ',' static void * -_tmp_179_rule(Parser *p) +_tmp_180_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33214,18 +33204,18 @@ _tmp_179_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'}'")); + D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'}'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 26)) // token='}' ) { - D(fprintf(stderr, "%*c+ _tmp_179[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'}'")); + D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'}'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_179[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'}'")); } { // ',' @@ -33233,18 +33223,18 @@ _tmp_179_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_179[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' ) { - D(fprintf(stderr, "%*c+ _tmp_179[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); + D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_179[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "','")); } _res = NULL; @@ -33253,9 +33243,9 @@ _tmp_179_rule(Parser *p) return _res; } -// _tmp_180: star_targets '=' +// _tmp_181: star_targets '=' static void * -_tmp_180_rule(Parser *p) +_tmp_181_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33272,7 +33262,7 @@ _tmp_180_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_180[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty z; if ( @@ -33281,7 +33271,7 @@ _tmp_180_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_180[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_181[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33291,7 +33281,7 @@ _tmp_180_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_180[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_181[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -33300,9 +33290,9 @@ _tmp_180_rule(Parser *p) return _res; } -// _tmp_181: '.' | '...' +// _tmp_182: '.' | '...' static void * -_tmp_181_rule(Parser *p) +_tmp_182_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33319,18 +33309,18 @@ _tmp_181_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_182[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_181[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_182[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_181[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_182[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -33338,18 +33328,18 @@ _tmp_181_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_181[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_182[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_181[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_182[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_181[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_182[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -33358,9 +33348,9 @@ _tmp_181_rule(Parser *p) return _res; } -// _tmp_182: '.' | '...' +// _tmp_183: '.' | '...' static void * -_tmp_182_rule(Parser *p) +_tmp_183_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33377,18 +33367,18 @@ _tmp_182_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_182[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_182[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_182[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -33396,18 +33386,18 @@ _tmp_182_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_182[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_182[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_182[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -33416,9 +33406,9 @@ _tmp_182_rule(Parser *p) return _res; } -// _tmp_183: '@' named_expression NEWLINE +// _tmp_184: '@' named_expression NEWLINE static void * -_tmp_183_rule(Parser *p) +_tmp_184_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33435,7 +33425,7 @@ _tmp_183_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_183[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); Token * _literal; expr_ty f; Token * newline_var; @@ -33447,7 +33437,7 @@ _tmp_183_rule(Parser *p) (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ _tmp_183[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); _res = f; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33457,7 +33447,7 @@ _tmp_183_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_183[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'@' named_expression NEWLINE")); } _res = NULL; @@ -33466,9 +33456,9 @@ _tmp_183_rule(Parser *p) return _res; } -// _tmp_184: ',' star_expression +// _tmp_185: ',' star_expression static void * -_tmp_184_rule(Parser *p) +_tmp_185_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33485,7 +33475,7 @@ _tmp_184_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_184[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c> _tmp_185[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); Token * _literal; expr_ty c; if ( @@ -33494,7 +33484,7 @@ _tmp_184_rule(Parser *p) (c = star_expression_rule(p)) // star_expression ) { - D(fprintf(stderr, "%*c+ _tmp_184[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c+ _tmp_185[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33504,7 +33494,7 @@ _tmp_184_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_184[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_185[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_expression")); } _res = NULL; @@ -33513,9 +33503,9 @@ _tmp_184_rule(Parser *p) return _res; } -// _tmp_185: ',' expression +// _tmp_186: ',' expression static void * -_tmp_185_rule(Parser *p) +_tmp_186_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33532,7 +33522,7 @@ _tmp_185_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_185[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c> _tmp_186[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); Token * _literal; expr_ty c; if ( @@ -33541,7 +33531,7 @@ _tmp_185_rule(Parser *p) (c = expression_rule(p)) // expression ) { - D(fprintf(stderr, "%*c+ _tmp_185[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c+ _tmp_186[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33551,7 +33541,7 @@ _tmp_185_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_185[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_186[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' expression")); } _res = NULL; @@ -33560,9 +33550,9 @@ _tmp_185_rule(Parser *p) return _res; } -// _tmp_186: 'or' conjunction +// _tmp_187: 'or' conjunction static void * -_tmp_186_rule(Parser *p) +_tmp_187_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33579,7 +33569,7 @@ _tmp_186_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_186[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c> _tmp_187[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); Token * _keyword; expr_ty c; if ( @@ -33588,7 +33578,7 @@ _tmp_186_rule(Parser *p) (c = conjunction_rule(p)) // conjunction ) { - D(fprintf(stderr, "%*c+ _tmp_186[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c+ _tmp_187[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33598,7 +33588,7 @@ _tmp_186_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_186[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_187[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'or' conjunction")); } _res = NULL; @@ -33607,9 +33597,9 @@ _tmp_186_rule(Parser *p) return _res; } -// _tmp_187: 'and' inversion +// _tmp_188: 'and' inversion static void * -_tmp_187_rule(Parser *p) +_tmp_188_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33626,7 +33616,7 @@ _tmp_187_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_187[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c> _tmp_188[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); Token * _keyword; expr_ty c; if ( @@ -33635,7 +33625,7 @@ _tmp_187_rule(Parser *p) (c = inversion_rule(p)) // inversion ) { - D(fprintf(stderr, "%*c+ _tmp_187[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c+ _tmp_188[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33645,7 +33635,7 @@ _tmp_187_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_187[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_188[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'and' inversion")); } _res = NULL; @@ -33654,9 +33644,9 @@ _tmp_187_rule(Parser *p) return _res; } -// _tmp_188: 'if' disjunction +// _tmp_189: 'if' disjunction static void * -_tmp_188_rule(Parser *p) +_tmp_189_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33673,7 +33663,7 @@ _tmp_188_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_188[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_189[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -33682,7 +33672,7 @@ _tmp_188_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_188[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_189[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33692,7 +33682,7 @@ _tmp_188_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_188[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_189[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -33701,9 +33691,9 @@ _tmp_188_rule(Parser *p) return _res; } -// _tmp_189: 'if' disjunction +// _tmp_190: 'if' disjunction static void * -_tmp_189_rule(Parser *p) +_tmp_190_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33720,7 +33710,7 @@ _tmp_189_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_189[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -33729,7 +33719,7 @@ _tmp_189_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_189[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_190[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33739,7 +33729,7 @@ _tmp_189_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_189[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_190[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -33748,9 +33738,9 @@ _tmp_189_rule(Parser *p) return _res; } -// _tmp_190: starred_expression | (assignment_expression | expression !':=') !'=' +// _tmp_191: starred_expression | (assignment_expression | expression !':=') !'=' static void * -_tmp_190_rule(Parser *p) +_tmp_191_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33767,18 +33757,18 @@ _tmp_190_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression")); + D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression")); expr_ty starred_expression_var; if ( (starred_expression_var = starred_expression_rule(p)) // starred_expression ) { - D(fprintf(stderr, "%*c+ _tmp_190[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression")); + D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression")); _res = starred_expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_190[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "starred_expression")); } { // (assignment_expression | expression !':=') !'=' @@ -33786,20 +33776,20 @@ _tmp_190_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_190[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); - void *_tmp_201_var; + D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); + void *_tmp_202_var; if ( - (_tmp_201_var = _tmp_201_rule(p)) // assignment_expression | expression !':=' + (_tmp_202_var = _tmp_202_rule(p)) // assignment_expression | expression !':=' && _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 22) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_190[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); - _res = _tmp_201_var; + D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(assignment_expression | expression !':=') !'='")); + _res = _tmp_202_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_190[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(assignment_expression | expression !':=') !'='")); } _res = NULL; @@ -33808,9 +33798,9 @@ _tmp_190_rule(Parser *p) return _res; } -// _tmp_191: ',' star_target +// _tmp_192: ',' star_target static void * -_tmp_191_rule(Parser *p) +_tmp_192_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33827,7 +33817,7 @@ _tmp_191_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_191[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty c; if ( @@ -33836,7 +33826,7 @@ _tmp_191_rule(Parser *p) (c = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_191[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33846,7 +33836,7 @@ _tmp_191_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_191[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } _res = NULL; @@ -33855,9 +33845,9 @@ _tmp_191_rule(Parser *p) return _res; } -// _tmp_192: ',' star_target +// _tmp_193: ',' star_target static void * -_tmp_192_rule(Parser *p) +_tmp_193_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33874,7 +33864,7 @@ _tmp_192_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_192[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty c; if ( @@ -33883,7 +33873,7 @@ _tmp_192_rule(Parser *p) (c = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_192[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -33893,7 +33883,7 @@ _tmp_192_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_192[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } _res = NULL; @@ -33902,9 +33892,9 @@ _tmp_192_rule(Parser *p) return _res; } -// _tmp_193: star_targets '=' +// _tmp_194: star_targets '=' static void * -_tmp_193_rule(Parser *p) +_tmp_194_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33921,7 +33911,7 @@ _tmp_193_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_193[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -33930,12 +33920,12 @@ _tmp_193_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_193[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_193[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -33944,9 +33934,9 @@ _tmp_193_rule(Parser *p) return _res; } -// _tmp_194: star_targets '=' +// _tmp_195: star_targets '=' static void * -_tmp_194_rule(Parser *p) +_tmp_195_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -33963,7 +33953,7 @@ _tmp_194_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_194[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -33972,12 +33962,12 @@ _tmp_194_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_194[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_194[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -33986,9 +33976,9 @@ _tmp_194_rule(Parser *p) return _res; } -// _tmp_195: ')' | '**' +// _tmp_196: ')' | '**' static void * -_tmp_195_rule(Parser *p) +_tmp_196_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34005,18 +33995,18 @@ _tmp_195_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // '**' @@ -34024,18 +34014,18 @@ _tmp_195_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_195[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_195[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_195[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; @@ -34044,9 +34034,9 @@ _tmp_195_rule(Parser *p) return _res; } -// _tmp_196: ':' | '**' +// _tmp_197: ':' | '**' static void * -_tmp_196_rule(Parser *p) +_tmp_197_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34063,18 +34053,18 @@ _tmp_196_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // '**' @@ -34082,18 +34072,18 @@ _tmp_196_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_196[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_196[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_196[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; @@ -34102,9 +34092,9 @@ _tmp_196_rule(Parser *p) return _res; } -// _tmp_197: expression ['as' star_target] +// _tmp_198: expression ['as' star_target] static void * -_tmp_197_rule(Parser *p) +_tmp_198_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34121,22 +34111,22 @@ _tmp_197_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_197[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_198[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_202_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_203_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_197[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_198[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); _res = _PyPegen_dummy_name(p, expression_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_197[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_198[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression ['as' star_target]")); } _res = NULL; @@ -34145,9 +34135,9 @@ _tmp_197_rule(Parser *p) return _res; } -// _tmp_198: expressions ['as' star_target] +// _tmp_199: expressions ['as' star_target] static void * -_tmp_198_rule(Parser *p) +_tmp_199_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34164,22 +34154,22 @@ _tmp_198_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_198[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_199[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expressions_var; if ( (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_203_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_204_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_198[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_199[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); _res = _PyPegen_dummy_name(p, expressions_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_198[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_199[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expressions ['as' star_target]")); } _res = NULL; @@ -34188,9 +34178,9 @@ _tmp_198_rule(Parser *p) return _res; } -// _tmp_199: expression ['as' star_target] +// _tmp_200: expression ['as' star_target] static void * -_tmp_199_rule(Parser *p) +_tmp_200_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34207,22 +34197,22 @@ _tmp_199_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_199[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_200[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression && - (_opt_var = _tmp_204_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_205_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_199[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_200[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression ['as' star_target]")); _res = _PyPegen_dummy_name(p, expression_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_199[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_200[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression ['as' star_target]")); } _res = NULL; @@ -34231,9 +34221,9 @@ _tmp_199_rule(Parser *p) return _res; } -// _tmp_200: expressions ['as' star_target] +// _tmp_201: expressions ['as' star_target] static void * -_tmp_200_rule(Parser *p) +_tmp_201_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34250,22 +34240,22 @@ _tmp_200_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_200[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty expressions_var; if ( (expressions_var = expressions_rule(p)) // expressions && - (_opt_var = _tmp_205_rule(p), !p->error_indicator) // ['as' star_target] + (_opt_var = _tmp_206_rule(p), !p->error_indicator) // ['as' star_target] ) { - D(fprintf(stderr, "%*c+ _tmp_200[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); + D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expressions ['as' star_target]")); _res = _PyPegen_dummy_name(p, expressions_var, _opt_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_200[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expressions ['as' star_target]")); } _res = NULL; @@ -34274,9 +34264,9 @@ _tmp_200_rule(Parser *p) return _res; } -// _tmp_201: assignment_expression | expression !':=' +// _tmp_202: assignment_expression | expression !':=' static void * -_tmp_201_rule(Parser *p) +_tmp_202_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34293,18 +34283,18 @@ _tmp_201_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "assignment_expression")); + D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "assignment_expression")); expr_ty assignment_expression_var; if ( (assignment_expression_var = assignment_expression_rule(p)) // assignment_expression ) { - D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "assignment_expression")); + D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "assignment_expression")); _res = assignment_expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "assignment_expression")); } { // expression !':=' @@ -34312,7 +34302,7 @@ _tmp_201_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_201[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression !':='")); + D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression !':='")); expr_ty expression_var; if ( (expression_var = expression_rule(p)) // expression @@ -34320,12 +34310,12 @@ _tmp_201_rule(Parser *p) _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 53) // token=':=' ) { - D(fprintf(stderr, "%*c+ _tmp_201[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression !':='")); + D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression !':='")); _res = expression_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_201[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression !':='")); } _res = NULL; @@ -34334,9 +34324,9 @@ _tmp_201_rule(Parser *p) return _res; } -// _tmp_202: 'as' star_target +// _tmp_203: 'as' star_target static void * -_tmp_202_rule(Parser *p) +_tmp_203_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34353,7 +34343,7 @@ _tmp_202_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_202[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -34362,12 +34352,12 @@ _tmp_202_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_202[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_202[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -34376,9 +34366,9 @@ _tmp_202_rule(Parser *p) return _res; } -// _tmp_203: 'as' star_target +// _tmp_204: 'as' star_target static void * -_tmp_203_rule(Parser *p) +_tmp_204_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34395,7 +34385,7 @@ _tmp_203_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_203[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_204[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -34404,12 +34394,12 @@ _tmp_203_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_203[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_204[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_203[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_204[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -34418,9 +34408,9 @@ _tmp_203_rule(Parser *p) return _res; } -// _tmp_204: 'as' star_target +// _tmp_205: 'as' star_target static void * -_tmp_204_rule(Parser *p) +_tmp_205_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34437,7 +34427,7 @@ _tmp_204_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_204[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_205[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -34446,12 +34436,12 @@ _tmp_204_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_204[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_205[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_204[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_205[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; @@ -34460,9 +34450,9 @@ _tmp_204_rule(Parser *p) return _res; } -// _tmp_205: 'as' star_target +// _tmp_206: 'as' star_target static void * -_tmp_205_rule(Parser *p) +_tmp_206_rule(Parser *p) { if (p->level++ == MAXSTACK) { p->error_indicator = 1; @@ -34479,7 +34469,7 @@ _tmp_205_rule(Parser *p) p->level--; return NULL; } - D(fprintf(stderr, "%*c> _tmp_205[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c> _tmp_206[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'as' star_target")); Token * _keyword; expr_ty star_target_var; if ( @@ -34488,12 +34478,12 @@ _tmp_205_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_205[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); + D(fprintf(stderr, "%*c+ _tmp_206[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'as' star_target")); _res = _PyPegen_dummy_name(p, _keyword, star_target_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_205[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_206[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'as' star_target")); } _res = NULL; diff --git a/Parser/pegen.c b/Parser/pegen.c index d7a498393753e..df17476013447 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -1259,7 +1259,6 @@ _PyPegen_Parser_New(struct tok_state *tok, int start_rule, int flags, p->known_err_token = NULL; p->level = 0; p->call_invalid_rules = 0; - p->in_raw_rule = 0; return p; } diff --git a/Parser/pegen.h b/Parser/pegen.h index bf58f1e1f2b7e..0e5a057c1f196 100644 --- a/Parser/pegen.h +++ b/Parser/pegen.h @@ -76,7 +76,6 @@ typedef struct { Token *known_err_token; int level; int call_invalid_rules; - int in_raw_rule; } Parser; typedef struct { diff --git a/Tools/peg_generator/pegen/c_generator.py b/Tools/peg_generator/pegen/c_generator.py index 49cfb720b1b24..2f0d829d795f6 100644 --- a/Tools/peg_generator/pegen/c_generator.py +++ b/Tools/peg_generator/pegen/c_generator.py @@ -115,6 +115,7 @@ def __init__( self.cache: Dict[Any, FunctionCall] = {} self.keyword_cache: Dict[str, int] = {} self.soft_keywords: Set[str] = set() + self.cleanup_statements: List[str] = [] def keyword_helper(self, keyword: str) -> FunctionCall: if keyword not in self.keyword_cache: @@ -357,6 +358,7 @@ def __init__( self._varname_counter = 0 self.debug = debug self.skip_actions = skip_actions + self.cleanup_statements: List[str] = [] def add_level(self) -> None: self.print("if (p->level++ == MAXSTACK) {") @@ -369,6 +371,8 @@ def remove_level(self) -> None: self.print("p->level--;") def add_return(self, ret_val: str) -> None: + for stmt in self.cleanup_statements: + self.print(stmt) self.remove_level() self.print(f"return {ret_val};") @@ -539,9 +543,7 @@ def _set_up_rule_memoization(self, node: Rule, result_type: str) -> None: f"_PyPegen_update_memo(p, _mark, {node.name}_type, _res)", "_res" ) self.print("p->mark = _mark;") - self.print("p->in_raw_rule++;") self.print(f"void *_raw = {node.name}_raw(p);") - self.print("p->in_raw_rule--;") self.print("if (p->error_indicator) {") with self.indent(): self.add_return("NULL") @@ -649,10 +651,21 @@ def visit_Rule(self, node: Rule) -> None: self._set_up_rule_memoization(node, result_type) self.print("{") + + if node.name.endswith("without_invalid"): + with self.indent(): + self.print("int _prev_call_invalid = p->call_invalid_rules;") + self.print("p->call_invalid_rules = 0;") + self.cleanup_statements.append("p->call_invalid_rules = _prev_call_invalid;") + if is_loop: self._handle_loop_rule_body(node, rhs) else: self._handle_default_rule_body(node, rhs, result_type) + + if node.name.endswith("without_invalid"): + self.cleanup_statements.pop() + self.print("}") def visit_NamedItem(self, node: NamedItem) -> None: From webhook-mailer at python.org Thu Feb 10 10:56:04 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 10 Feb 2022 15:56:04 -0000 Subject: [Python-checkins] Gather stats for PRECALL_METHOD. (GH-31259) Message-ID: https://github.com/python/cpython/commit/1a6411f5738895fa48d35a93435f7c7b6c17bdb9 commit: 1a6411f5738895fa48d35a93435f7c7b6c17bdb9 branch: main author: Mark Shannon committer: markshannon date: 2022-02-10T15:55:52Z summary: Gather stats for PRECALL_METHOD. (GH-31259) files: M Python/ceval.c M Python/specialize.c diff --git a/Python/ceval.c b/Python/ceval.c index c3703a75ce0ca..ffce6b735c1c2 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4451,8 +4451,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr assert(call_shape.kwnames == NULL); #ifdef Py_STATS extern int _PySpecialization_ClassifyCallable(PyObject *); - _py_stats.opcode_stats[PRECALL_FUNCTION].specialization.failure++; - _py_stats.opcode_stats[PRECALL_FUNCTION].specialization.failure_kinds[_PySpecialization_ClassifyCallable(call_shape.callable)]++; + SpecializationStats *stats = + &_py_stats.opcode_stats[PRECALL_FUNCTION].specialization; + stats->failure++; + int kind = _PySpecialization_ClassifyCallable(call_shape.callable); + stats->failure_kinds[kind]++; #endif DISPATCH(); } @@ -4493,6 +4496,14 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr call_shape.total_args = nargs; assert(call_shape.kwnames == NULL); +#ifdef Py_STATS + extern int _PySpecialization_ClassifyCallable(PyObject *); + SpecializationStats *stats = + &_py_stats.opcode_stats[PRECALL_METHOD].specialization; + stats->failure++; + int kind = _PySpecialization_ClassifyCallable(call_shape.callable); + stats->failure_kinds[kind]++; +#endif DISPATCH(); } diff --git a/Python/specialize.c b/Python/specialize.c index 940ab172d55fe..b051f45157ad3 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -176,6 +176,7 @@ print_spec_stats(FILE *out, OpcodeStats *stats) * even though we don't specialize them yet. */ fprintf(out, " opcode[%d].specializable : 1\n", FOR_ITER); fprintf(out, " opcode[%d].specializable : 1\n", PRECALL_FUNCTION); + fprintf(out, " opcode[%d].specializable : 1\n", PRECALL_METHOD); fprintf(out, " opcode[%d].specializable : 1\n", UNPACK_SEQUENCE); for (int i = 0; i < 256; i++) { if (adaptive_opcodes[i]) { From webhook-mailer at python.org Thu Feb 10 12:50:24 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 10 Feb 2022 17:50:24 -0000 Subject: [Python-checkins] bpo-45923: Add `RESUME_QUICK` (GH-31244) Message-ID: https://github.com/python/cpython/commit/d7a5aca982def155a9255893cefcc1493c127c9c commit: d7a5aca982def155a9255893cefcc1493c127c9c branch: main author: Brandt Bucher committer: markshannon date: 2022-02-10T17:50:02Z summary: bpo-45923: Add `RESUME_QUICK` (GH-31244) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-09-20-21-43.bpo-45923.tJ4gDX.rst M Include/opcode.h M Lib/opcode.py M Python/ceval.c M Python/opcode_targets.h M Python/specialize.c diff --git a/Include/opcode.h b/Include/opcode.h index bce7010ab186b..58fc62808937f 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -164,15 +164,16 @@ extern "C" { #define LOAD_METHOD_CLASS 76 #define LOAD_METHOD_MODULE 77 #define LOAD_METHOD_NO_DICT 78 -#define STORE_ATTR_ADAPTIVE 79 -#define STORE_ATTR_INSTANCE_VALUE 80 -#define STORE_ATTR_SLOT 81 -#define STORE_ATTR_WITH_HINT 131 -#define LOAD_FAST__LOAD_FAST 140 -#define STORE_FAST__LOAD_FAST 141 -#define LOAD_FAST__LOAD_CONST 143 -#define LOAD_CONST__LOAD_FAST 150 -#define STORE_FAST__STORE_FAST 153 +#define RESUME_QUICK 79 +#define STORE_ATTR_ADAPTIVE 80 +#define STORE_ATTR_INSTANCE_VALUE 81 +#define STORE_ATTR_SLOT 131 +#define STORE_ATTR_WITH_HINT 140 +#define LOAD_FAST__LOAD_FAST 141 +#define STORE_FAST__LOAD_FAST 143 +#define LOAD_FAST__LOAD_CONST 150 +#define LOAD_CONST__LOAD_FAST 153 +#define STORE_FAST__STORE_FAST 154 #define DO_TRACING 255 #ifdef NEED_OPCODE_JUMP_TABLES static uint32_t _PyOpcode_RelativeJump[8] = { diff --git a/Lib/opcode.py b/Lib/opcode.py index c672aa59f8ec3..a1f0c6e4326d1 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -278,6 +278,7 @@ def jabs_op(name, op): "LOAD_METHOD_CLASS", "LOAD_METHOD_MODULE", "LOAD_METHOD_NO_DICT", + "RESUME_QUICK", "STORE_ATTR_ADAPTIVE", "STORE_ATTR_INSTANCE_VALUE", "STORE_ATTR_SLOT", diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-09-20-21-43.bpo-45923.tJ4gDX.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-09-20-21-43.bpo-45923.tJ4gDX.rst new file mode 100644 index 0000000000000..5ab5d59e50f00 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-09-20-21-43.bpo-45923.tJ4gDX.rst @@ -0,0 +1 @@ +Add a quickened form of :opcode:`RESUME` that skips quickening checks. diff --git a/Python/ceval.c b/Python/ceval.c index ffce6b735c1c2..5eb91502c3010 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1734,9 +1734,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(RESUME) { - assert(tstate->cframe == &cframe); - assert(frame == cframe.current_frame); - int err = _Py_IncrementCountAndMaybeQuicken(frame->f_code); if (err) { if (err < 0) { @@ -1747,6 +1744,13 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr first_instr = frame->f_code->co_firstinstr; next_instr = first_instr + nexti; } + JUMP_TO_INSTRUCTION(RESUME_QUICK); + } + + TARGET(RESUME_QUICK) { + PREDICTED(RESUME_QUICK); + assert(tstate->cframe == &cframe); + assert(frame == cframe.current_frame); frame->f_state = FRAME_EXECUTING; if (_Py_atomic_load_relaxed(eval_breaker) && oparg < 2) { goto handle_eval_breaker; @@ -4004,7 +4008,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(JUMP_ABSOLUTE) { PREDICTED(JUMP_ABSOLUTE); - assert(oparg < INSTR_OFFSET()); int err = _Py_IncrementCountAndMaybeQuicken(frame->f_code); if (err) { if (err < 0) { @@ -4015,9 +4018,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr first_instr = frame->f_code->co_firstinstr; next_instr = first_instr + nexti; } - JUMPTO(oparg); - CHECK_EVAL_BREAKER(); - DISPATCH(); + JUMP_TO_INSTRUCTION(JUMP_ABSOLUTE_QUICK); } TARGET(JUMP_NO_INTERRUPT) { @@ -4032,6 +4033,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(JUMP_ABSOLUTE_QUICK) { + PREDICTED(JUMP_ABSOLUTE_QUICK); assert(oparg < INSTR_OFFSET()); JUMPTO(oparg); CHECK_EVAL_BREAKER(); diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index 1a809ed409d58..f47da2bbb1ef0 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -78,9 +78,9 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_METHOD_CLASS, &&TARGET_LOAD_METHOD_MODULE, &&TARGET_LOAD_METHOD_NO_DICT, + &&TARGET_RESUME_QUICK, &&TARGET_STORE_ATTR_ADAPTIVE, &&TARGET_STORE_ATTR_INSTANCE_VALUE, - &&TARGET_STORE_ATTR_SLOT, &&TARGET_LIST_TO_TUPLE, &&TARGET_RETURN_VALUE, &&TARGET_IMPORT_STAR, @@ -130,7 +130,7 @@ static void *opcode_targets[256] = { &&TARGET_POP_JUMP_IF_NOT_NONE, &&TARGET_POP_JUMP_IF_NONE, &&TARGET_RAISE_VARARGS, - &&TARGET_STORE_ATTR_WITH_HINT, + &&TARGET_STORE_ATTR_SLOT, &&TARGET_MAKE_FUNCTION, &&TARGET_BUILD_SLICE, &&TARGET_JUMP_NO_INTERRUPT, @@ -139,21 +139,21 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_DEREF, &&TARGET_STORE_DEREF, &&TARGET_DELETE_DEREF, + &&TARGET_STORE_ATTR_WITH_HINT, &&TARGET_LOAD_FAST__LOAD_FAST, - &&TARGET_STORE_FAST__LOAD_FAST, &&TARGET_CALL_FUNCTION_EX, - &&TARGET_LOAD_FAST__LOAD_CONST, + &&TARGET_STORE_FAST__LOAD_FAST, &&TARGET_EXTENDED_ARG, &&TARGET_LIST_APPEND, &&TARGET_SET_ADD, &&TARGET_MAP_ADD, &&TARGET_LOAD_CLASSDEREF, &&TARGET_COPY_FREE_VARS, - &&TARGET_LOAD_CONST__LOAD_FAST, + &&TARGET_LOAD_FAST__LOAD_CONST, &&TARGET_RESUME, &&TARGET_MATCH_CLASS, + &&TARGET_LOAD_CONST__LOAD_FAST, &&TARGET_STORE_FAST__STORE_FAST, - &&_unknown_opcode, &&TARGET_FORMAT_VALUE, &&TARGET_BUILD_CONST_KEY_MAP, &&TARGET_BUILD_STRING, diff --git a/Python/specialize.c b/Python/specialize.c index b051f45157ad3..e610a2d85fe58 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -407,6 +407,9 @@ optimize(SpecializedCacheOrInstruction *quickened, int len) case JUMP_ABSOLUTE: instructions[i] = _Py_MAKECODEUNIT(JUMP_ABSOLUTE_QUICK, oparg); break; + case RESUME: + instructions[i] = _Py_MAKECODEUNIT(RESUME_QUICK, oparg); + break; case LOAD_FAST: switch(previous_opcode) { case LOAD_FAST: From webhook-mailer at python.org Thu Feb 10 16:58:08 2022 From: webhook-mailer at python.org (sweeneyde) Date: Thu, 10 Feb 2022 21:58:08 -0000 Subject: [Python-checkins] bpo-44953: Add vectorcall for itemgetter and attrgetter instances (GH-27828) Message-ID: https://github.com/python/cpython/commit/0a145069e807fdafd1fa0315b9bc22da363d2d39 commit: 0a145069e807fdafd1fa0315b9bc22da363d2d39 branch: main author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2022-02-10T16:57:47-05:00 summary: bpo-44953: Add vectorcall for itemgetter and attrgetter instances (GH-27828) files: A Misc/NEWS.d/next/Library/2021-08-19-09-29-43.bpo-44953.27ZyUd.rst M Modules/_operator.c diff --git a/Misc/NEWS.d/next/Library/2021-08-19-09-29-43.bpo-44953.27ZyUd.rst b/Misc/NEWS.d/next/Library/2021-08-19-09-29-43.bpo-44953.27ZyUd.rst new file mode 100644 index 0000000000000..0d54ef006af3e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-08-19-09-29-43.bpo-44953.27ZyUd.rst @@ -0,0 +1 @@ +Calling ``operator.itemgetter`` objects and ``operator.attrgetter`` objects is now faster due to use of the vectorcall calling convention. \ No newline at end of file diff --git a/Modules/_operator.c b/Modules/_operator.c index 10adabcbf0e85..fdc3176c8395f 100644 --- a/Modules/_operator.c +++ b/Modules/_operator.c @@ -1,5 +1,6 @@ #include "Python.h" #include "pycore_moduleobject.h" // _PyModule_GetState() +#include "structmember.h" // PyMemberDef #include "pycore_runtime.h" // _Py_ID() #include "clinic/_operator.c.h" @@ -974,8 +975,15 @@ typedef struct { Py_ssize_t nitems; PyObject *item; Py_ssize_t index; // -1 unless *item* is a single non-negative integer index + vectorcallfunc vectorcall; } itemgetterobject; +// Forward declarations +static PyObject * +itemgetter_vectorcall(PyObject *, PyObject *const *, size_t, PyObject *); +static PyObject * +itemgetter_call_impl(itemgetterobject *, PyObject *); + /* AC 3.5: treats first argument as an iterable, otherwise uses *args */ static PyObject * itemgetter_new(PyTypeObject *type, PyObject *args, PyObject *kwds) @@ -1021,6 +1029,7 @@ itemgetter_new(PyTypeObject *type, PyObject *args, PyObject *kwds) } } + ig->vectorcall = (vectorcallfunc)itemgetter_vectorcall; PyObject_GC_Track(ig); return (PyObject *)ig; } @@ -1053,16 +1062,33 @@ itemgetter_traverse(itemgetterobject *ig, visitproc visit, void *arg) static PyObject * itemgetter_call(itemgetterobject *ig, PyObject *args, PyObject *kw) { - PyObject *obj, *result; - Py_ssize_t i, nitems=ig->nitems; - assert(PyTuple_CheckExact(args)); if (!_PyArg_NoKeywords("itemgetter", kw)) return NULL; if (!_PyArg_CheckPositional("itemgetter", PyTuple_GET_SIZE(args), 1, 1)) return NULL; + return itemgetter_call_impl(ig, PyTuple_GET_ITEM(args, 0)); +} - obj = PyTuple_GET_ITEM(args, 0); +static PyObject * +itemgetter_vectorcall(PyObject *ig, PyObject *const *args, + size_t nargsf, PyObject *kwnames) +{ + if (!_PyArg_NoKwnames("itemgetter", kwnames)) { + return NULL; + } + Py_ssize_t nargs = PyVectorcall_NARGS(nargsf); + if (!_PyArg_CheckPositional("itemgetter", nargs, 1, 1)) { + return NULL; + } + return itemgetter_call_impl((itemgetterobject *)ig, args[0]); +} + +static PyObject * +itemgetter_call_impl(itemgetterobject *ig, PyObject *obj) +{ + PyObject *result; + Py_ssize_t i, nitems=ig->nitems; if (nitems == 1) { if (ig->index >= 0 && PyTuple_CheckExact(obj) @@ -1130,6 +1156,11 @@ static PyMethodDef itemgetter_methods[] = { {NULL} }; +static PyMemberDef itemgetter_members[] = { + {"__vectorcalloffset__", T_PYSSIZET, offsetof(itemgetterobject, vectorcall), READONLY}, + {NULL} /* Sentinel */ +}; + PyDoc_STRVAR(itemgetter_doc, "itemgetter(item, ...) --> itemgetter object\n\ \n\ @@ -1144,6 +1175,7 @@ static PyType_Slot itemgetter_type_slots[] = { {Py_tp_traverse, itemgetter_traverse}, {Py_tp_clear, itemgetter_clear}, {Py_tp_methods, itemgetter_methods}, + {Py_tp_members, itemgetter_members}, {Py_tp_new, itemgetter_new}, {Py_tp_getattro, PyObject_GenericGetAttr}, {Py_tp_repr, itemgetter_repr}, @@ -1155,7 +1187,7 @@ static PyType_Spec itemgetter_type_spec = { .basicsize = sizeof(itemgetterobject), .itemsize = 0, .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | - Py_TPFLAGS_IMMUTABLETYPE), + Py_TPFLAGS_IMMUTABLETYPE | Py_TPFLAGS_HAVE_VECTORCALL), .slots = itemgetter_type_slots, }; @@ -1165,8 +1197,15 @@ typedef struct { PyObject_HEAD Py_ssize_t nattrs; PyObject *attr; + vectorcallfunc vectorcall; } attrgetterobject; +// Forward declarations +static PyObject * +attrgetter_vectorcall(PyObject *, PyObject *const *, size_t, PyObject *); +static PyObject * +attrgetter_call_impl(attrgetterobject *, PyObject *); + /* AC 3.5: treats first argument as an iterable, otherwise uses *args */ static PyObject * attrgetter_new(PyTypeObject *type, PyObject *args, PyObject *kwds) @@ -1210,7 +1249,7 @@ attrgetter_new(PyTypeObject *type, PyObject *args, PyObject *kwds) kind = PyUnicode_KIND(item); data = PyUnicode_DATA(item); - /* check whethere the string is dotted */ + /* check whether the string is dotted */ dot_count = 0; for (char_idx = 0; char_idx < item_len; ++char_idx) { if (PyUnicode_READ(kind, data, char_idx) == '.') @@ -1276,6 +1315,7 @@ attrgetter_new(PyTypeObject *type, PyObject *args, PyObject *kwds) ag->attr = attr; ag->nattrs = nattrs; + ag->vectorcall = (vectorcallfunc)attrgetter_vectorcall; PyObject_GC_Track(ag); return (PyObject *)ag; @@ -1342,16 +1382,36 @@ dotted_getattr(PyObject *obj, PyObject *attr) static PyObject * attrgetter_call(attrgetterobject *ag, PyObject *args, PyObject *kw) { - PyObject *obj, *result; - Py_ssize_t i, nattrs=ag->nattrs; - if (!_PyArg_NoKeywords("attrgetter", kw)) return NULL; if (!_PyArg_CheckPositional("attrgetter", PyTuple_GET_SIZE(args), 1, 1)) return NULL; - obj = PyTuple_GET_ITEM(args, 0); - if (ag->nattrs == 1) /* ag->attr is always a tuple */ + return attrgetter_call_impl(ag, PyTuple_GET_ITEM(args, 0)); +} + +static PyObject * +attrgetter_vectorcall(PyObject *ag, PyObject *const *args, size_t nargsf, PyObject *kwnames) +{ + if (!_PyArg_NoKwnames("attrgetter", kwnames)) { + return NULL; + } + Py_ssize_t nargs = PyVectorcall_NARGS(nargsf); + if (!_PyArg_CheckPositional("attrgetter", nargs, 1, 1)) { + return NULL; + } + return attrgetter_call_impl((attrgetterobject *)ag, args[0]); +} + +static PyObject * +attrgetter_call_impl(attrgetterobject *ag, PyObject *obj) +{ + PyObject *result; + Py_ssize_t i, nattrs=ag->nattrs; + + if (ag->nattrs == 1) { + /* ag->attr is always a tuple */ return dotted_getattr(obj, PyTuple_GET_ITEM(ag->attr, 0)); + } assert(PyTuple_Check(ag->attr)); assert(PyTuple_GET_SIZE(ag->attr) == nattrs); @@ -1460,6 +1520,11 @@ static PyMethodDef attrgetter_methods[] = { {NULL} }; +static PyMemberDef attrgetter_members[] = { + {"__vectorcalloffset__", T_PYSSIZET, offsetof(attrgetterobject, vectorcall), READONLY}, + {NULL} /* Sentinel*/ +}; + PyDoc_STRVAR(attrgetter_doc, "attrgetter(attr, ...) --> attrgetter object\n\ \n\ @@ -1476,6 +1541,7 @@ static PyType_Slot attrgetter_type_slots[] = { {Py_tp_traverse, attrgetter_traverse}, {Py_tp_clear, attrgetter_clear}, {Py_tp_methods, attrgetter_methods}, + {Py_tp_members, attrgetter_members}, {Py_tp_new, attrgetter_new}, {Py_tp_getattro, PyObject_GenericGetAttr}, {Py_tp_repr, attrgetter_repr}, @@ -1487,7 +1553,7 @@ static PyType_Spec attrgetter_type_spec = { .basicsize = sizeof(attrgetterobject), .itemsize = 0, .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | - Py_TPFLAGS_IMMUTABLETYPE), + Py_TPFLAGS_IMMUTABLETYPE | Py_TPFLAGS_HAVE_VECTORCALL), .slots = attrgetter_type_slots, }; From webhook-mailer at python.org Thu Feb 10 17:57:29 2022 From: webhook-mailer at python.org (sweeneyde) Date: Thu, 10 Feb 2022 22:57:29 -0000 Subject: [Python-checkins] bpo-44953: Add newline at end of NEWS entry (GH-31265) Message-ID: https://github.com/python/cpython/commit/035414a878a772d1d293cdecdc4470bcce5e5d7a commit: 035414a878a772d1d293cdecdc4470bcce5e5d7a branch: main author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2022-02-10T17:57:24-05:00 summary: bpo-44953: Add newline at end of NEWS entry (GH-31265) files: M Misc/NEWS.d/next/Library/2021-08-19-09-29-43.bpo-44953.27ZyUd.rst diff --git a/Misc/NEWS.d/next/Library/2021-08-19-09-29-43.bpo-44953.27ZyUd.rst b/Misc/NEWS.d/next/Library/2021-08-19-09-29-43.bpo-44953.27ZyUd.rst index 0d54ef006af3e..0eb235eca4fc0 100644 --- a/Misc/NEWS.d/next/Library/2021-08-19-09-29-43.bpo-44953.27ZyUd.rst +++ b/Misc/NEWS.d/next/Library/2021-08-19-09-29-43.bpo-44953.27ZyUd.rst @@ -1 +1 @@ -Calling ``operator.itemgetter`` objects and ``operator.attrgetter`` objects is now faster due to use of the vectorcall calling convention. \ No newline at end of file +Calling ``operator.itemgetter`` objects and ``operator.attrgetter`` objects is now faster due to use of the vectorcall calling convention. From webhook-mailer at python.org Thu Feb 10 18:14:24 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Thu, 10 Feb 2022 23:14:24 -0000 Subject: [Python-checkins] bpo-36876: Make sure the c-analyzer is checking all the source files.' (gh-31264) Message-ID: https://github.com/python/cpython/commit/80e4f262aa27a39abf3fadc19a6323fea4607a8f commit: 80e4f262aa27a39abf3fadc19a6323fea4607a8f branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-10T16:14:19-07:00 summary: bpo-36876: Make sure the c-analyzer is checking all the source files.' (gh-31264) https://bugs.python.org/issue36876 files: M Tools/c-analyzer/cpython/_analyzer.py M Tools/c-analyzer/cpython/_files.py M Tools/c-analyzer/cpython/_parser.py M Tools/c-analyzer/cpython/globals-to-fix.tsv M Tools/c-analyzer/cpython/ignored.tsv diff --git a/Tools/c-analyzer/cpython/_analyzer.py b/Tools/c-analyzer/cpython/_analyzer.py index 49de9d887284a..741fbaddf16d9 100644 --- a/Tools/c-analyzer/cpython/_analyzer.py +++ b/Tools/c-analyzer/cpython/_analyzer.py @@ -307,7 +307,7 @@ def render(self, fmt='line', *, itemonly=False): # XXX ??? #if itemonly: # yield from rendered - supported = self._supported + supported = self.supported if fmt in ('line', 'brief'): rendered, = rendered parts = [ diff --git a/Tools/c-analyzer/cpython/_files.py b/Tools/c-analyzer/cpython/_files.py index 3e397880977ab..ee9e46f7e5e95 100644 --- a/Tools/c-analyzer/cpython/_files.py +++ b/Tools/c-analyzer/cpython/_files.py @@ -6,15 +6,17 @@ GLOBS = [ 'Include/*.h', + # Technically, this is covered by "Include/*.h": + #'Include/cpython/*.h', 'Include/internal/*.h', 'Modules/**/*.h', 'Modules/**/*.c', 'Objects/**/*.h', 'Objects/**/*.c', - 'Python/**/*.h', + 'Parser/**/*.h', 'Parser/**/*.c', 'Python/**/*.h', - 'Parser/**/*.c', + 'Python/**/*.c', ] LEVEL_GLOBS = { 'stable': 'Include/*.h', diff --git a/Tools/c-analyzer/cpython/_parser.py b/Tools/c-analyzer/cpython/_parser.py index 6b55c41c2c71e..e58871cf73dce 100644 --- a/Tools/c-analyzer/cpython/_parser.py +++ b/Tools/c-analyzer/cpython/_parser.py @@ -81,6 +81,7 @@ def clean_lines(text): Objects/unicodetype_db.h # generated +Python/deepfreeze/*.c Python/frozen_modules/*.h Python/opcode_targets.h Python/stdlib_module_names.h @@ -157,6 +158,7 @@ def clean_lines(text): Modules/faulthandler.c Py_BUILD_CORE 1 Modules/gcmodule.c Py_BUILD_CORE 1 Modules/getpath.c Py_BUILD_CORE 1 +Modules/getpath_noop.c Py_BUILD_CORE 1 Modules/itertoolsmodule.c Py_BUILD_CORE 1 Modules/main.c Py_BUILD_CORE 1 Modules/mathmodule.c Py_BUILD_CORE 1 @@ -169,9 +171,10 @@ def clean_lines(text): Modules/unicodedata.c Py_BUILD_CORE 1 Objects/stringlib/codecs.h Py_BUILD_CORE 1 Objects/stringlib/unicode_format.h Py_BUILD_CORE 1 +Parser/string_parser.h Py_BUILD_CORE 1 +Parser/pegen.h Py_BUILD_CORE 1 Python/ceval_gil.h Py_BUILD_CORE 1 Python/condvar.h Py_BUILD_CORE 1 -Modules/getpath_noop.c Py_BUILD_CORE 1 Modules/_json.c Py_BUILD_CORE_BUILTIN 1 Modules/_pickle.c Py_BUILD_CORE_BUILTIN 1 @@ -293,6 +296,9 @@ def clean_lines(text): _abs('Modules/expat/expat.h'): (10_000, 400), _abs('Objects/stringlib/unicode_format.h'): (10_000, 400), _abs('Objects/typeobject.c'): (20_000, 200), + _abs('Python/compile.c'): (20_000, 500), + _abs('Python/pylifecycle.c'): (200_000, 5000), + _abs('Python/pystate.c'): (200_000, 5000), } diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 5690481485831..4ef0a6c848b7e 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -106,6 +106,28 @@ Objects/unionobject.c - _Py_UnionType - Objects/weakrefobject.c - _PyWeakref_CallableProxyType - Objects/weakrefobject.c - _PyWeakref_ProxyType - Objects/weakrefobject.c - _PyWeakref_RefType - +Python/bltinmodule.c - PyFilter_Type - +Python/bltinmodule.c - PyMap_Type - +Python/bltinmodule.c - PyZip_Type - +Python/context.c - PyContextToken_Type - +Python/context.c - PyContextVar_Type - +Python/context.c - PyContext_Type - +Python/context.c - _PyContextTokenMissing_Type - +Python/errors.c - UnraisableHookArgsType - +Python/hamt.c - _PyHamtItems_Type - +Python/hamt.c - _PyHamtKeys_Type - +Python/hamt.c - _PyHamtValues_Type - +Python/hamt.c - _PyHamt_ArrayNode_Type - +Python/hamt.c - _PyHamt_BitmapNode_Type - +Python/hamt.c - _PyHamt_CollisionNode_Type - +Python/hamt.c - _PyHamt_Type - +Python/symtable.c - PySTEntry_Type - +Python/sysmodule.c - AsyncGenHooksType - +Python/sysmodule.c - FlagsType - +Python/sysmodule.c - Hash_InfoType - +Python/sysmodule.c - VersionInfoType - +Python/thread.c - ThreadInfoType - +Python/traceback.c - PyTraceBack_Type - #----------------------- # builtin exception types @@ -306,12 +328,20 @@ Python/compile.c compiler_visit_annotations return_str - Python/import.c PyImport_Import builtins_str - # XXX This should have been found by the analyzer but wasn't: Python/import.c PyImport_Import import_str - +Python/symtable.c - __class__ - +Python/symtable.c - _annotation - +Python/symtable.c - dictcomp - +Python/symtable.c - genexpr - +Python/symtable.c - lambda - +Python/symtable.c - listcomp - +Python/symtable.c - setcomp - +Python/symtable.c - top - # XXX This should have been found by the analyzer but wasn't: Python/sysmodule.c - whatstrings - # XXX This should have been found by the analyzer but wasn't: Python/sysmodule.c sys_displayhook newline - -# _PyArg_Parser +# _PyArg_Parser (holds tuple of strings) Objects/clinic/bytearrayobject.c.h bytearray___init__ _parser - Objects/clinic/bytearrayobject.c.h bytearray_decode _parser - Objects/clinic/bytearrayobject.c.h bytearray_hex _parser - @@ -367,6 +397,9 @@ Python/clinic/sysmodule.c.h sys_addaudithook _parser - Python/clinic/sysmodule.c.h sys_set_coroutine_origin_tracking_depth _parser - Python/clinic/traceback.c.h tb_new _parser - +# holds strings +Objects/typeobject.c - slotdefs - + # other Objects/typeobject.c - method_cache - Objects/unicodeobject.c - _string_module - @@ -400,29 +433,73 @@ Python/import.c - extensions - # global non-objects to fix in core code #----------------------- -# initialized once +# initialized/set once # pre-allocated buffer Modules/getbuildinfo.c Py_GetBuildInfo buildinfo - # during init +Objects/typeobject.c - slotdefs_initialized - +Objects/unicodeobject.c - bloom_linebreak - Parser/parser.c - Py_DebugFlag - - -# other -Objects/codeobject.c PyCode_NewEmpty nulltuple - +Python/bootstrap_hash.c - _Py_HashSecret_Initialized - +Python/bootstrap_hash.c py_getrandom getrandom_works - +Python/pyhash.c - _Py_HashSecret - +Python/pylifecycle.c - runtime_initialized - +Python/sysmodule.c - _PySys_ImplCacheTag - +Python/sysmodule.c - _PySys_ImplName - +Python/sysmodule.c - _preinit_warnoptions - +Python/sysmodule.c - _preinit_xoptions - +Python/thread.c - thread_debug - +Python/thread.c - initialized - + +# lazy +Objects/floatobject.c - double_format - +Objects/floatobject.c - float_format - +Objects/floatobject.c - detected_double_format - +Objects/floatobject.c - detected_float_format - Objects/longobject.c PyLong_FromString log_base_BASE - Objects/longobject.c PyLong_FromString convwidth_base - Objects/longobject.c PyLong_FromString convmultmax_base - -Objects/typeobject.c - slotdefs - -Objects/typeobject.c - slotdefs_initialized - -Objects/unicodeobject.c - bloom_linebreak - Objects/unicodeobject.c - ucnhash_capi - Parser/action_helpers.c _PyPegen_dummy_name cache - -Parser/pegen.c _PyPegen_dummy_name cache - +Python/dtoa.c - p5s - +Python/fileutils.c - force_ascii - +Python/fileutils.c set_inheritable ioctl_works - # XXX This should have been found by the analyzer but wasn't: Python/import.c - import_lock - # XXX This should have been found by the analyzer but wasn't: Python/import.c import_find_and_load header - +Python/specialize.c - _list_append - + +#----------------------- +# unlikely to change after init (or main thread) + +# through C-API +Python/frozen.c - PyImport_FrozenModules - +Python/frozen.c - _PyImport_FrozenAliases - +Python/frozen.c - _PyImport_FrozenBootstrap - +Python/frozen.c - _PyImport_FrozenStdlib - +Python/frozen.c - _PyImport_FrozenTest - +Python/import.c - inittab_copy - +Python/import.c - PyImport_Inittab - +Python/preconfig.c - Py_FileSystemDefaultEncoding - +Python/preconfig.c - Py_HasFileSystemDefaultEncoding - +Python/preconfig.c - Py_FileSystemDefaultEncodeErrors - +Python/preconfig.c - _Py_HasFileSystemDefaultEncodeErrors - + +# REPL +Parser/myreadline.c - _PyOS_ReadlineLock - +Parser/myreadline.c - _PyOS_ReadlineTState - +Parser/myreadline.c - PyOS_InputHook - +Parser/myreadline.c - PyOS_ReadlineFunctionPointer - + +# handling C argv +Python/getopt.c - _PyOS_optarg - +Python/getopt.c - _PyOS_opterr - +Python/getopt.c - _PyOS_optind - +Python/getopt.c - opt_ptr - +Python/pathconfig.c - _Py_path_config - #----------------------- # state @@ -444,20 +521,22 @@ Objects/obmalloc.c - unused_arena_objects - Objects/obmalloc.c - usable_arenas - Objects/obmalloc.c new_arena debug_stats - -# REPL -Parser/myreadline.c - _PyOS_ReadlineLock - -Parser/myreadline.c - _PyOS_ReadlineTState - -Parser/myreadline.c - PyOS_InputHook - -Parser/myreadline.c - PyOS_ReadlineFunctionPointer - +# pre-allocated memory +Python/dtoa.c - freelist - +Python/dtoa.c - private_mem - + +# local buffer +Python/getversion.c Py_GetVersion version - +Python/suggestions.c levenshtein_distance buffer - + +# linked list +Python/dtoa.c - pmem_next - +Python/getargs.c - static_arg_parsers - # other Objects/dictobject.c - _pydict_global_version - Objects/dictobject.c - next_dict_keys_version - Objects/dictobject.c - pydict_global_version - -Objects/floatobject.c - double_format - -Objects/floatobject.c - float_format - -Objects/floatobject.c - detected_double_format - -Objects/floatobject.c - detected_float_format - Objects/funcobject.c - next_func_version - Objects/moduleobject.c - max_module_number - Objects/object.c - _Py_RefTotal - @@ -480,10 +559,14 @@ Python/import.c - import_lock_thread - Python/import.c import_find_and_load accumulated - # XXX This should have been found by the analyzer but wasn't: Python/import.c import_find_and_load import_level - +Python/modsupport.c - _Py_PackageContext - +Python/pyfpe.c - PyFPE_counter - +Python/pylifecycle.c _Py_FatalErrorFormat reentrant - # XXX This should have been found by the analyzer but wasn't: Python/pylifecycle.c - _Py_UnhandledKeyboardInterrupt - # XXX This should have been found by the analyzer but wasn't: Python/pylifecycle.c fatal_error reentrant - +Python/specialize.c - _Py_QuickenedCount - ################################## diff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsv index 9b8eff3685a68..c945542e29aa5 100644 --- a/Tools/c-analyzer/cpython/ignored.tsv +++ b/Tools/c-analyzer/cpython/ignored.tsv @@ -1,6 +1,12 @@ filename funcname name reason #??? - somevar ??? +################################## +# ignored by design + +Python/pylifecycle.c - _PyRuntime - + + ################################## # forward/extern references # XXX The analyzer should have ignored these. @@ -384,6 +390,13 @@ Objects/interpreteridobject.c interpid_new kwlist - Objects/weakrefobject.c weakref_call kwlist - Objects/exceptions.c NameError_init kwlist - Objects/exceptions.c AttributeError_init kwlist - +Python/_warnings.c warnings_warn_explicit kwd_list - +Python/bltinmodule.c builtin___import__ kwlist - +Python/bltinmodule.c min_max kwlist - +Python/bltinmodule.c zip_new kwlist - +Python/context.c contextvar_tp_new kwlist - +Python/sysmodule.c sys_getsizeof kwlist - +Python/sysmodule.c sys_set_asyncgen_hooks keywords - #----------------------- # PyModuleDef @@ -397,6 +410,13 @@ Modules/_threadmodule.c - thread_module - Modules/_typingmodule.c - typingmodule - Modules/signalmodule.c - signal_module - Modules/xxlimited_35.c - xxmodule - +Python/Python-ast.c - _astmodule - +Python/Python-tokenize.c - _tokenizemodule - +Python/_warnings.c - warnings_module - +Python/bltinmodule.c - builtinsmodule - +Python/import.c - imp_module - +Python/marshal.c - marshalmodule - +Python/sysmodule.c - sysmodule - #----------------------- # PyModuleDef_Slot @@ -478,6 +498,8 @@ Modules/xxmodule.c - xx_slots - Modules/xxsubtype.c - xxsubtype_slots - Modules/zlibmodule.c - zlib_slots - Python/Python-ast.c - astmodule_slots - +Python/Python-tokenize.c - tokenizemodule_slots - +Python/_warnings.c - warnings_slots - Python/marshal.c - marshalmodule_slots - #----------------------- @@ -798,6 +820,7 @@ Objects/unionobject.c - union_methods - Objects/weakrefobject.c - proxy_methods - Objects/weakrefobject.c - weakref_methods - Python/Python-ast.c - ast_type_methods - +Python/Python-tokenize.c - tokenize_methods - Python/_warnings.c - warnings_functions - Python/bltinmodule.c - builtin_methods - Python/bltinmodule.c - filter_methods - @@ -807,6 +830,7 @@ Python/context.c - PyContextTokenType_methods - Python/context.c - PyContextVar_methods - Python/context.c - PyContext_methods - Python/hamt.c - PyHamt_methods - +Python/import.c - imp_slots - Python/import.c - imp_methods - Python/marshal.c - marshal_methods - Python/sysmodule.c - sys_methods - @@ -1218,6 +1242,7 @@ Modules/xxlimited_35.c - Xxo_Type_slots - Modules/zlibmodule.c - Comptype_slots - Modules/zlibmodule.c - Decomptype_slots - Python/Python-ast.c - AST_type_slots - +Python/Python-tokenize.c - tokenizeriter_slots - # PyType_Spec Modules/_abc.c - _abc_data_type_spec - @@ -1317,6 +1342,7 @@ Modules/xxlimited_35.c - Xxo_Type_spec - Modules/zlibmodule.c - Comptype_spec - Modules/zlibmodule.c - Decomptype_spec - Python/Python-ast.c - AST_type_spec - +Python/Python-tokenize.c - tokenizeriter_spec - #----------------------- # for structseq @@ -1470,4 +1496,24 @@ Objects/unicodeobject.c unicode_translate_call_errorhandler argparse - Parser/parser.c - reserved_keywords - Parser/parser.c - soft_keywords - Parser/tokenizer.c - type_comment_prefix - +Python/ast_opt.c fold_unaryop ops - +Python/codecs.c - Py_hexdigits - +Python/codecs.c - ucnhash_capi - +Python/codecs.c _PyCodecRegistry_Init methods - +Python/dynload_shlib.c - _PyImport_DynLoadFiletab - +Python/dynload_stub.c - _PyImport_DynLoadFiletab - +Python/frozen.c - aliases - +Python/frozen.c - bootstrap_modules - +Python/frozen.c - stdlib_modules - +Python/frozen.c - test_modules - +Python/getopt.c - longopts - +Python/import.c - _PyImport_Inittab - +Python/import.c - _PySys_ImplCacheTag - Python/opcode_targets.h - opcode_targets - +Python/pyhash.c - PyHash_Func - +Python/pylifecycle.c - _C_LOCALE_WARNING - +Python/pylifecycle.c - _PyOS_mystrnicmp_hack - +Python/pylifecycle.c - _TARGET_LOCALES - +Python/specialize.c - adaptive_opcodes - +Python/specialize.c - cache_requirements - +Python/specialize.c - compare_masks - From webhook-mailer at python.org Thu Feb 10 19:56:25 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 11 Feb 2022 00:56:25 -0000 Subject: [Python-checkins] bpo-46246: add missing __slots__ to importlib.metadata.DeprecatedList (GH-30452) Message-ID: https://github.com/python/cpython/commit/dd76b3f7d332dd6eced5cbc2ad2adfc397700b3d commit: dd76b3f7d332dd6eced5cbc2ad2adfc397700b3d branch: main author: Arie Bovenberg committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-10T16:56:21-08:00 summary: bpo-46246: add missing __slots__ to importlib.metadata.DeprecatedList (GH-30452) Confirmed with @jaraco that this indeed needs a fix. A question that came up while I was digging into the code: I think `SelectableGroups` could similarly use `__slots__ = ()`, since its purpose seems only for convenience around `dict`, not to have attributes of its own. Automerge-Triggered-By: GH:jaraco files: A Misc/NEWS.d/next/Library/2022-01-07-13-27-53.bpo-46246.CTLx32.rst M Lib/importlib/metadata/__init__.py M Lib/test/test_importlib/test_metadata_api.py diff --git a/Lib/importlib/metadata/__init__.py b/Lib/importlib/metadata/__init__.py index 371c482209586..9fd6e04ae9abf 100644 --- a/Lib/importlib/metadata/__init__.py +++ b/Lib/importlib/metadata/__init__.py @@ -278,6 +278,8 @@ class DeprecatedList(list): 1 """ + __slots__ = () + _warn = functools.partial( warnings.warn, "EntryPoints list interface is deprecated. Cast to list if needed.", diff --git a/Lib/test/test_importlib/test_metadata_api.py b/Lib/test/test_importlib/test_metadata_api.py index e16773a7e87ef..78287c314ced7 100644 --- a/Lib/test/test_importlib/test_metadata_api.py +++ b/Lib/test/test_importlib/test_metadata_api.py @@ -172,6 +172,11 @@ def test_entry_points_groups_get(self): entry_points().get('entries', 'default') == entry_points()['entries'] entry_points().get('missing', ()) == () + def test_entry_points_allows_no_attributes(self): + ep = entry_points().select(group='entries', name='main') + with self.assertRaises(AttributeError): + ep.foo = 4 + def test_metadata_for_this_package(self): md = metadata('egginfo-pkg') assert md['author'] == 'Steven Ma' diff --git a/Misc/NEWS.d/next/Library/2022-01-07-13-27-53.bpo-46246.CTLx32.rst b/Misc/NEWS.d/next/Library/2022-01-07-13-27-53.bpo-46246.CTLx32.rst new file mode 100644 index 0000000000000..4850171439459 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-07-13-27-53.bpo-46246.CTLx32.rst @@ -0,0 +1,2 @@ +Add missing ``__slots__`` to ``importlib.metadata.DeprecatedList``. Patch by +Arie Bovenberg. From webhook-mailer at python.org Thu Feb 10 20:18:32 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 11 Feb 2022 01:18:32 -0000 Subject: [Python-checkins] bpo-46246: add missing __slots__ to importlib.metadata.DeprecatedList (GH-30452) Message-ID: https://github.com/python/cpython/commit/1124ab6d1d15dc5058e03b63fd1d95e6f1009cc3 commit: 1124ab6d1d15dc5058e03b63fd1d95e6f1009cc3 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-10T17:18:23-08:00 summary: bpo-46246: add missing __slots__ to importlib.metadata.DeprecatedList (GH-30452) Confirmed with @jaraco that this indeed needs a fix. A question that came up while I was digging into the code: I think `SelectableGroups` could similarly use `__slots__ = ()`, since its purpose seems only for convenience around `dict`, not to have attributes of its own. Automerge-Triggered-By: GH:jaraco (cherry picked from commit dd76b3f7d332dd6eced5cbc2ad2adfc397700b3d) Co-authored-by: Arie Bovenberg files: A Misc/NEWS.d/next/Library/2022-01-07-13-27-53.bpo-46246.CTLx32.rst M Lib/importlib/metadata/__init__.py M Lib/test/test_importlib/test_metadata_api.py diff --git a/Lib/importlib/metadata/__init__.py b/Lib/importlib/metadata/__init__.py index 33ce1b6b56964..c7462f31c7a0f 100644 --- a/Lib/importlib/metadata/__init__.py +++ b/Lib/importlib/metadata/__init__.py @@ -236,6 +236,8 @@ class DeprecatedList(list): 1 """ + __slots__ = () + _warn = functools.partial( warnings.warn, "EntryPoints list interface is deprecated. Cast to list if needed.", diff --git a/Lib/test/test_importlib/test_metadata_api.py b/Lib/test/test_importlib/test_metadata_api.py index 4a45312e31cd5..0890d6cb72e88 100644 --- a/Lib/test/test_importlib/test_metadata_api.py +++ b/Lib/test/test_importlib/test_metadata_api.py @@ -172,6 +172,11 @@ def test_entry_points_groups_get(self): entry_points().get('entries', 'default') == entry_points()['entries'] entry_points().get('missing', ()) == () + def test_entry_points_allows_no_attributes(self): + ep = entry_points().select(group='entries', name='main') + with self.assertRaises(AttributeError): + ep.foo = 4 + def test_metadata_for_this_package(self): md = metadata('egginfo-pkg') assert md['author'] == 'Steven Ma' diff --git a/Misc/NEWS.d/next/Library/2022-01-07-13-27-53.bpo-46246.CTLx32.rst b/Misc/NEWS.d/next/Library/2022-01-07-13-27-53.bpo-46246.CTLx32.rst new file mode 100644 index 0000000000000..4850171439459 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-07-13-27-53.bpo-46246.CTLx32.rst @@ -0,0 +1,2 @@ +Add missing ``__slots__`` to ``importlib.metadata.DeprecatedList``. Patch by +Arie Bovenberg. From webhook-mailer at python.org Fri Feb 11 03:49:09 2022 From: webhook-mailer at python.org (corona10) Date: Fri, 11 Feb 2022 08:49:09 -0000 Subject: [Python-checkins] bpo-46323: Fix double-free issue for borrowed refs (GH-31272) Message-ID: https://github.com/python/cpython/commit/0ac5372bf6b937ed44a8f9c4e402d024fcd80870 commit: 0ac5372bf6b937ed44a8f9c4e402d024fcd80870 branch: main author: Dong-hee Na committer: corona10 date: 2022-02-11T17:48:47+09:00 summary: bpo-46323: Fix double-free issue for borrowed refs (GH-31272) files: M Modules/_ctypes/callbacks.c diff --git a/Modules/_ctypes/callbacks.c b/Modules/_ctypes/callbacks.c index 6037f5fd84286..bb59456a7321a 100644 --- a/Modules/_ctypes/callbacks.c +++ b/Modules/_ctypes/callbacks.c @@ -189,7 +189,6 @@ static void _CallPythonObject(void *mem, CDataObject *obj = (CDataObject *)_PyObject_CallNoArgs(cnv); if (!obj) { PrintError("create argument %zd:\n", i); - Py_DECREF(cnv); goto Done; } if (!CDataObject_Check(obj)) { From webhook-mailer at python.org Fri Feb 11 06:25:49 2022 From: webhook-mailer at python.org (encukou) Date: Fri, 11 Feb 2022 11:25:49 -0000 Subject: [Python-checkins] [3.10] bpo-46433: _PyType_GetModuleByDef: handle static types in MRO (GH-30696) (GH-31262) Message-ID: https://github.com/python/cpython/commit/8b8673fe940c4ebc4512bff5af180b66def3d1ae commit: 8b8673fe940c4ebc4512bff5af180b66def3d1ae branch: 3.10 author: Petr Viktorin committer: encukou date: 2022-02-11T12:25:25+01:00 summary: [3.10] bpo-46433: _PyType_GetModuleByDef: handle static types in MRO (GH-30696) (GH-31262) (cherry picked from commit 0ef08530124c5ca13a9394f4ac18bee8e6c66409) files: A Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst M Lib/test/test_capi.py M Modules/_testmultiphase.c M Modules/clinic/_testmultiphase.c.h M Objects/typeobject.c diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py index ccb9d5383d6be..7b208c90063fd 100644 --- a/Lib/test/test_capi.py +++ b/Lib/test/test_capi.py @@ -1030,6 +1030,22 @@ def test_state_access(self): with self.assertRaises(TypeError): increment_count(1, 2, 3) + def test_get_module_bad_def(self): + # _PyType_GetModuleByDef fails gracefully if it doesn't + # find what it's looking for. + # see bpo-46433 + instance = self.module.StateAccessType() + with self.assertRaises(TypeError): + instance.getmodulebydef_bad_def() + + def test_get_module_static_in_mro(self): + # Here, the class _PyType_GetModuleByDef is looking for + # appears in the MRO after a static type (Exception). + # see bpo-46433 + class Subclass(BaseException, self.module.StateAccessType): + pass + self.assertIs(Subclass().get_defining_module(), self.module) + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst b/Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst new file mode 100644 index 0000000000000..e1987c4536b5c --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-01-19-16-51-54.bpo-46433.Er9ApS.rst @@ -0,0 +1,2 @@ +The internal function _PyType_GetModuleByDef now correctly handles +inheritance patterns involving static types. diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c index e0ed77d265cdc..58282cec1087d 100644 --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -122,6 +122,8 @@ static PyType_Spec Example_Type_spec = { static PyModuleDef def_meth_state_access; +static PyModuleDef def_nonmodule; +static PyModuleDef def_nonmodule_with_methods; /*[clinic input] _testmultiphase.StateAccessType.get_defining_module @@ -149,6 +151,24 @@ _testmultiphase_StateAccessType_get_defining_module_impl(StateAccessTypeObject * return retval; } +/*[clinic input] +_testmultiphase.StateAccessType.getmodulebydef_bad_def + + cls: defining_class + +Test that result of _PyType_GetModuleByDef with a bad def is NULL. +[clinic start generated code]*/ + +static PyObject * +_testmultiphase_StateAccessType_getmodulebydef_bad_def_impl(StateAccessTypeObject *self, + PyTypeObject *cls) +/*[clinic end generated code: output=64509074dfcdbd31 input=906047715ee293cd]*/ +{ + _PyType_GetModuleByDef(Py_TYPE(self), &def_nonmodule); // should raise + assert(PyErr_Occurred()); + return NULL; +} + /*[clinic input] _testmultiphase.StateAccessType.increment_count_clinic @@ -245,6 +265,7 @@ _testmultiphase_StateAccessType_get_count_impl(StateAccessTypeObject *self, static PyMethodDef StateAccessType_methods[] = { _TESTMULTIPHASE_STATEACCESSTYPE_GET_DEFINING_MODULE_METHODDEF + _TESTMULTIPHASE_STATEACCESSTYPE_GETMODULEBYDEF_BAD_DEF_METHODDEF _TESTMULTIPHASE_STATEACCESSTYPE_GET_COUNT_METHODDEF _TESTMULTIPHASE_STATEACCESSTYPE_INCREMENT_COUNT_CLINIC_METHODDEF { @@ -433,9 +454,6 @@ PyInit__testmultiphase(PyObject *spec) /**** Importing a non-module object ****/ -static PyModuleDef def_nonmodule; -static PyModuleDef def_nonmodule_with_methods; - /* Create a SimpleNamespace(three=3) */ static PyObject* createfunc_nonmodule(PyObject *spec, PyModuleDef *def) diff --git a/Modules/clinic/_testmultiphase.c.h b/Modules/clinic/_testmultiphase.c.h index 55f934be8c6c1..17c28d54ce575 100644 --- a/Modules/clinic/_testmultiphase.c.h +++ b/Modules/clinic/_testmultiphase.c.h @@ -35,6 +35,36 @@ _testmultiphase_StateAccessType_get_defining_module(StateAccessTypeObject *self, return return_value; } +PyDoc_STRVAR(_testmultiphase_StateAccessType_getmodulebydef_bad_def__doc__, +"getmodulebydef_bad_def($self, /)\n" +"--\n" +"\n" +"Test that result of _PyType_GetModuleByDef with a bad def is NULL."); + +#define _TESTMULTIPHASE_STATEACCESSTYPE_GETMODULEBYDEF_BAD_DEF_METHODDEF \ + {"getmodulebydef_bad_def", (PyCFunction)(void(*)(void))_testmultiphase_StateAccessType_getmodulebydef_bad_def, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _testmultiphase_StateAccessType_getmodulebydef_bad_def__doc__}, + +static PyObject * +_testmultiphase_StateAccessType_getmodulebydef_bad_def_impl(StateAccessTypeObject *self, + PyTypeObject *cls); + +static PyObject * +_testmultiphase_StateAccessType_getmodulebydef_bad_def(StateAccessTypeObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":getmodulebydef_bad_def", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = _testmultiphase_StateAccessType_getmodulebydef_bad_def_impl(self, cls); + +exit: + return return_value; +} + PyDoc_STRVAR(_testmultiphase_StateAccessType_increment_count_clinic__doc__, "increment_count_clinic($self, /, n=1, *, twice=False)\n" "--\n" @@ -101,4 +131,4 @@ _testmultiphase_StateAccessType_get_count(StateAccessTypeObject *self, PyTypeObj exit: return return_value; } -/*[clinic end generated code: output=f01137bb3b373e14 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=eb1b8c2ee6290be3 input=a9049054013a1b77]*/ diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 10b69fe1196db..deffeb2ccc25d 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3707,22 +3707,20 @@ _PyType_GetModuleByDef(PyTypeObject *type, struct PyModuleDef *def) // to check i < PyTuple_GET_SIZE(mro) at the first loop iteration. assert(PyTuple_GET_SIZE(mro) >= 1); - Py_ssize_t i = 0; - do { + Py_ssize_t n = PyTuple_GET_SIZE(mro); + for (Py_ssize_t i = 0; i < n; i++) { PyObject *super = PyTuple_GET_ITEM(mro, i); - // _PyType_GetModuleByDef() must only be called on a heap type created - // by PyType_FromModuleAndSpec() or on its subclasses. - // type_ready_mro() ensures that a static type cannot inherit from a - // heap type. - assert(_PyType_HasFeature((PyTypeObject *)type, Py_TPFLAGS_HEAPTYPE)); + if(!_PyType_HasFeature((PyTypeObject *)super, Py_TPFLAGS_HEAPTYPE)) { + // Static types in the MRO need to be skipped + continue; + } PyHeapTypeObject *ht = (PyHeapTypeObject*)super; PyObject *module = ht->ht_module; if (module && _PyModule_GetDef(module) == def) { return module; } - i++; - } while (i < PyTuple_GET_SIZE(mro)); + } PyErr_Format( PyExc_TypeError, From webhook-mailer at python.org Fri Feb 11 09:43:10 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 11 Feb 2022 14:43:10 -0000 Subject: [Python-checkins] Fix the signature of multiprocessing.set_executable (GH-31276) Message-ID: https://github.com/python/cpython/commit/4f9386661d51b78348395e78710f3bfbee9fd1de commit: 4f9386661d51b78348395e78710f3bfbee9fd1de branch: main author: G?ry Ogam committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-11T06:42:54-08:00 summary: Fix the signature of multiprocessing.set_executable (GH-31276) Automerge-Triggered-By: GH:merwok files: M Doc/library/multiprocessing.rst diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 9bb7dd3d703ab..cbbe1840fc351 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1047,9 +1047,9 @@ Miscellaneous .. versionadded:: 3.4 -.. function:: set_executable() +.. function:: set_executable(executable) - Sets the path of the Python interpreter to use when starting a child process. + Set the path of the Python interpreter to use when starting a child process. (By default :data:`sys.executable` is used). Embedders will probably need to do some thing like :: From webhook-mailer at python.org Fri Feb 11 10:08:52 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 11 Feb 2022 15:08:52 -0000 Subject: [Python-checkins] Fix the signature of multiprocessing.set_executable (GH-31276) Message-ID: https://github.com/python/cpython/commit/c23f4ab3f57c48dffbcf773b3907fdc36ffc98b7 commit: c23f4ab3f57c48dffbcf773b3907fdc36ffc98b7 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-11T07:08:44-08:00 summary: Fix the signature of multiprocessing.set_executable (GH-31276) Automerge-Triggered-By: GH:merwok (cherry picked from commit 4f9386661d51b78348395e78710f3bfbee9fd1de) Co-authored-by: G?ry Ogam files: M Doc/library/multiprocessing.rst diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 4966be303f425..b62e1f8dba1bf 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1047,9 +1047,9 @@ Miscellaneous .. versionadded:: 3.4 -.. function:: set_executable() +.. function:: set_executable(executable) - Sets the path of the Python interpreter to use when starting a child process. + Set the path of the Python interpreter to use when starting a child process. (By default :data:`sys.executable` is used). Embedders will probably need to do some thing like :: From webhook-mailer at python.org Fri Feb 11 11:01:27 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 11 Feb 2022 16:01:27 -0000 Subject: [Python-checkins] bpo-45490: Rename static inline functions (GH-31217) Message-ID: https://github.com/python/cpython/commit/e0bcfd0e4db193743d4bafc48d10f15ae9ed7b2b commit: e0bcfd0e4db193743d4bafc48d10f15ae9ed7b2b branch: main author: Victor Stinner committer: vstinner date: 2022-02-11T17:01:10+01:00 summary: bpo-45490: Rename static inline functions (GH-31217) When a static inline function is wrapped by a macro which casts its arguments to the expected type, there is no need that the function has a different name than the macro. Use the same name for the macro and the function to avoid confusion. Rename _PyUnicode_get_wstr_length() to PyUnicode_WSTR_LENGTH(). Don't rename static inline _Py_NewRef() and _Py_XNewRef() functions, since the C API exports Py_NewRef() and Py_XNewRef() functions as regular functions. The name cannot be reused in this case. files: M Include/cpython/unicodeobject.h M Include/object.h M Python/specialize.c diff --git a/Include/cpython/unicodeobject.h b/Include/cpython/unicodeobject.h index be5647c7d263c..77a171b86bff4 100644 --- a/Include/cpython/unicodeobject.h +++ b/Include/cpython/unicodeobject.h @@ -434,12 +434,12 @@ enum PyUnicode_Kind { (0x10ffffU))))) Py_DEPRECATED(3.3) -static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { +static inline Py_ssize_t PyUnicode_WSTR_LENGTH(PyObject *op) { return PyUnicode_IS_COMPACT_ASCII(op) ? ((PyASCIIObject*)op)->length : ((PyCompactUnicodeObject*)op)->wstr_length; } -#define PyUnicode_WSTR_LENGTH(op) _PyUnicode_get_wstr_length((PyObject*)op) +#define PyUnicode_WSTR_LENGTH(op) PyUnicode_WSTR_LENGTH(_PyObject_CAST(op)) /* === Public API ========================================================= */ diff --git a/Include/object.h b/Include/object.h index 4fd16616ce705..3566c736a535c 100644 --- a/Include/object.h +++ b/Include/object.h @@ -127,49 +127,49 @@ PyAPI_FUNC(int) Py_Is(PyObject *x, PyObject *y); #define Py_Is(x, y) ((x) == (y)) -static inline Py_ssize_t _Py_REFCNT(const PyObject *ob) { +static inline Py_ssize_t Py_REFCNT(const PyObject *ob) { return ob->ob_refcnt; } -#define Py_REFCNT(ob) _Py_REFCNT(_PyObject_CAST_CONST(ob)) +#define Py_REFCNT(ob) Py_REFCNT(_PyObject_CAST_CONST(ob)) // bpo-39573: The Py_SET_TYPE() function must be used to set an object type. -static inline PyTypeObject* _Py_TYPE(const PyObject *ob) { +static inline PyTypeObject* Py_TYPE(const PyObject *ob) { return ob->ob_type; } -#define Py_TYPE(ob) _Py_TYPE(_PyObject_CAST_CONST(ob)) +#define Py_TYPE(ob) Py_TYPE(_PyObject_CAST_CONST(ob)) // bpo-39573: The Py_SET_SIZE() function must be used to set an object size. -static inline Py_ssize_t _Py_SIZE(const PyVarObject *ob) { +static inline Py_ssize_t Py_SIZE(const PyVarObject *ob) { return ob->ob_size; } -#define Py_SIZE(ob) _Py_SIZE(_PyVarObject_CAST_CONST(ob)) +#define Py_SIZE(ob) Py_SIZE(_PyVarObject_CAST_CONST(ob)) -static inline int _Py_IS_TYPE(const PyObject *ob, const PyTypeObject *type) { +static inline int Py_IS_TYPE(const PyObject *ob, const PyTypeObject *type) { // bpo-44378: Don't use Py_TYPE() since Py_TYPE() requires a non-const // object. return ob->ob_type == type; } -#define Py_IS_TYPE(ob, type) _Py_IS_TYPE(_PyObject_CAST_CONST(ob), type) +#define Py_IS_TYPE(ob, type) Py_IS_TYPE(_PyObject_CAST_CONST(ob), type) -static inline void _Py_SET_REFCNT(PyObject *ob, Py_ssize_t refcnt) { +static inline void Py_SET_REFCNT(PyObject *ob, Py_ssize_t refcnt) { ob->ob_refcnt = refcnt; } -#define Py_SET_REFCNT(ob, refcnt) _Py_SET_REFCNT(_PyObject_CAST(ob), refcnt) +#define Py_SET_REFCNT(ob, refcnt) Py_SET_REFCNT(_PyObject_CAST(ob), refcnt) -static inline void _Py_SET_TYPE(PyObject *ob, PyTypeObject *type) { +static inline void Py_SET_TYPE(PyObject *ob, PyTypeObject *type) { ob->ob_type = type; } -#define Py_SET_TYPE(ob, type) _Py_SET_TYPE(_PyObject_CAST(ob), type) +#define Py_SET_TYPE(ob, type) Py_SET_TYPE(_PyObject_CAST(ob), type) -static inline void _Py_SET_SIZE(PyVarObject *ob, Py_ssize_t size) { +static inline void Py_SET_SIZE(PyVarObject *ob, Py_ssize_t size) { ob->ob_size = size; } -#define Py_SET_SIZE(ob, size) _Py_SET_SIZE(_PyVarObject_CAST(ob), size) +#define Py_SET_SIZE(ob, size) Py_SET_SIZE(_PyVarObject_CAST(ob), size) /* @@ -253,10 +253,10 @@ PyAPI_FUNC(PyObject *) PyType_GetQualName(PyTypeObject *); /* Generic type check */ PyAPI_FUNC(int) PyType_IsSubtype(PyTypeObject *, PyTypeObject *); -static inline int _PyObject_TypeCheck(PyObject *ob, PyTypeObject *type) { +static inline int PyObject_TypeCheck(PyObject *ob, PyTypeObject *type) { return Py_IS_TYPE(ob, type) || PyType_IsSubtype(Py_TYPE(ob), type); } -#define PyObject_TypeCheck(ob, type) _PyObject_TypeCheck(_PyObject_CAST(ob), type) +#define PyObject_TypeCheck(ob, type) PyObject_TypeCheck(_PyObject_CAST(ob), type) PyAPI_DATA(PyTypeObject) PyType_Type; /* built-in 'type' */ PyAPI_DATA(PyTypeObject) PyBaseObject_Type; /* built-in 'object' */ @@ -475,7 +475,7 @@ PyAPI_FUNC(void) Py_DecRef(PyObject *); PyAPI_FUNC(void) _Py_IncRef(PyObject *); PyAPI_FUNC(void) _Py_DecRef(PyObject *); -static inline void _Py_INCREF(PyObject *op) +static inline void Py_INCREF(PyObject *op) { #if defined(Py_REF_DEBUG) && defined(Py_LIMITED_API) && Py_LIMITED_API+0 >= 0x030A0000 // Stable ABI for Python 3.10 built in debug mode. @@ -489,9 +489,9 @@ static inline void _Py_INCREF(PyObject *op) op->ob_refcnt++; #endif } -#define Py_INCREF(op) _Py_INCREF(_PyObject_CAST(op)) +#define Py_INCREF(op) Py_INCREF(_PyObject_CAST(op)) -static inline void _Py_DECREF( +static inline void Py_DECREF( #if defined(Py_REF_DEBUG) && !(defined(Py_LIMITED_API) && Py_LIMITED_API+0 >= 0x030A0000) const char *filename, int lineno, #endif @@ -519,9 +519,9 @@ static inline void _Py_DECREF( #endif } #if defined(Py_REF_DEBUG) && !(defined(Py_LIMITED_API) && Py_LIMITED_API+0 >= 0x030A0000) -# define Py_DECREF(op) _Py_DECREF(__FILE__, __LINE__, _PyObject_CAST(op)) +# define Py_DECREF(op) Py_DECREF(__FILE__, __LINE__, _PyObject_CAST(op)) #else -# define Py_DECREF(op) _Py_DECREF(_PyObject_CAST(op)) +# define Py_DECREF(op) Py_DECREF(_PyObject_CAST(op)) #endif @@ -569,23 +569,23 @@ static inline void _Py_DECREF( } while (0) /* Function to use in case the object pointer can be NULL: */ -static inline void _Py_XINCREF(PyObject *op) +static inline void Py_XINCREF(PyObject *op) { if (op != NULL) { Py_INCREF(op); } } -#define Py_XINCREF(op) _Py_XINCREF(_PyObject_CAST(op)) +#define Py_XINCREF(op) Py_XINCREF(_PyObject_CAST(op)) -static inline void _Py_XDECREF(PyObject *op) +static inline void Py_XDECREF(PyObject *op) { if (op != NULL) { Py_DECREF(op); } } -#define Py_XDECREF(op) _Py_XDECREF(_PyObject_CAST(op)) +#define Py_XDECREF(op) Py_XDECREF(_PyObject_CAST(op)) // Create a new strong reference to an object: // increment the reference count of the object and return the object. @@ -750,17 +750,17 @@ PyType_HasFeature(PyTypeObject *type, unsigned long feature) #define PyType_FastSubclass(type, flag) PyType_HasFeature(type, flag) -static inline int _PyType_Check(PyObject *op) { +static inline int PyType_Check(PyObject *op) { return PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_TYPE_SUBCLASS); } -#define PyType_Check(op) _PyType_Check(_PyObject_CAST(op)) +#define PyType_Check(op) PyType_Check(_PyObject_CAST(op)) #define _PyType_CAST(op) (assert(PyType_Check(op)), (PyTypeObject*)(op)) -static inline int _PyType_CheckExact(PyObject *op) { +static inline int PyType_CheckExact(PyObject *op) { return Py_IS_TYPE(op, &PyType_Type); } -#define PyType_CheckExact(op) _PyType_CheckExact(_PyObject_CAST(op)) +#define PyType_CheckExact(op) PyType_CheckExact(_PyObject_CAST(op)) #ifdef __cplusplus } diff --git a/Python/specialize.c b/Python/specialize.c index e610a2d85fe58..1259a3ca3eccc 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -1939,7 +1939,7 @@ int if (PyAsyncGen_CheckExact(iter)) { return SPEC_FAIL_FOR_ITER_ASYNC_GENERATOR; } - PyTypeObject *t = _Py_TYPE(iter); + PyTypeObject *t = Py_TYPE(iter); if (t == &PyListIter_Type) { return SPEC_FAIL_FOR_ITER_LIST; } From webhook-mailer at python.org Fri Feb 11 11:22:17 2022 From: webhook-mailer at python.org (encukou) Date: Fri, 11 Feb 2022 16:22:17 -0000 Subject: [Python-checkins] bpo-46613: Add PyType_GetModuleByDef to the public API (GH-31081) Message-ID: https://github.com/python/cpython/commit/204946986feee7bc80b233350377d24d20fcb1b8 commit: 204946986feee7bc80b233350377d24d20fcb1b8 branch: main author: Petr Viktorin committer: encukou date: 2022-02-11T17:22:11+01:00 summary: bpo-46613: Add PyType_GetModuleByDef to the public API (GH-31081) * Make PyType_GetModuleByDef public (remove underscore) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/C API/2022-02-02-17-58-49.bpo-46613.__ZdpH.rst M Doc/c-api/type.rst M Doc/howto/clinic.rst M Doc/whatsnew/3.11.rst M Include/cpython/object.h M Lib/test/test_capi.py M Modules/_csv.c M Modules/_functoolsmodule.c M Modules/_queuemodule.c M Modules/_randommodule.c M Modules/_sqlite/module.h M Modules/_ssl.c M Modules/_ssl.h M Modules/_struct.c M Modules/_testmultiphase.c M Modules/_threadmodule.c M Modules/arraymodule.c M Modules/cjkcodecs/multibytecodec.c M Modules/clinic/_testmultiphase.c.h M Objects/typeobject.c M Python/Python-tokenize.c diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index 97a818ab2ccd0..e6a5a0ea9c121 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -149,6 +149,8 @@ Type Objects ``Py_TYPE(self)`` may be a *subclass* of the intended class, and subclasses are not necessarily defined in the same module as their superclass. See :c:type:`PyCMethod` to get the class that defines the method. + See :c:func:`PyType_GetModuleByDef` for cases when ``PyCMethod`` cannot + be used. .. versionadded:: 3.9 @@ -166,6 +168,21 @@ Type Objects .. versionadded:: 3.9 +.. c:function:: PyObject* PyType_GetModuleByDef(PyTypeObject *type, struct PyModuleDef *def) + + Find the first superclass whose module was created from + the given :c:type:`PyModuleDef` *def*, and return that module. + + If no module is found, raises a :py:class:`TypeError` and returns ``NULL``. + + This function is intended to be used together with + :c:func:`PyModule_GetState()` to get module state from slot methods (such as + :c:member:`~PyTypeObject.tp_init` or :c:member:`~PyNumberMethods.nb_add`) + and other places where a method's defining class cannot be passed using the + :c:type:`PyCMethod` calling convention. + + .. versionadded:: 3.11 + Creating Heap-Allocated Types ............................. diff --git a/Doc/howto/clinic.rst b/Doc/howto/clinic.rst index a3c4330296bed..04b1a2cac0b04 100644 --- a/Doc/howto/clinic.rst +++ b/Doc/howto/clinic.rst @@ -1249,15 +1249,15 @@ The ``defining_class`` converter is not compatible with ``__init__`` and ``__new methods, which cannot use the ``METH_METHOD`` convention. It is not possible to use ``defining_class`` with slot methods. In order to -fetch the module state from such methods, use ``_PyType_GetModuleByDef`` to -look up the module and then :c:func:`PyModule_GetState` to fetch the module +fetch the module state from such methods, use :c:func:`PyType_GetModuleByDef` +to look up the module and then :c:func:`PyModule_GetState` to fetch the module state. Example from the ``setattro`` slot method in ``Modules/_threadmodule.c``:: static int local_setattro(localobject *self, PyObject *name, PyObject *v) { - PyObject *module = _PyType_GetModuleByDef(Py_TYPE(self), &thread_module); + PyObject *module = PyType_GetModuleByDef(Py_TYPE(self), &thread_module); thread_module_state *state = get_thread_state(module); ... } diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 5738745ba1323..95fce1e8e1082 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -708,6 +708,11 @@ New Features (Contributed by Christian Heimes in :issue:`45459`.) +* Added the :c:data:`PyType_GetModuleByDef` function, used to get the module + in which a method was defined, in cases where this information is not + available directly (via :c:type:`PyCMethod`). + (Contributed by Petr Viktorin in :issue:`46613`.) + Porting to Python 3.11 ---------------------- diff --git a/Include/cpython/object.h b/Include/cpython/object.h index a3632cf1dcf81..2ee97ba0b40af 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -265,7 +265,7 @@ PyAPI_FUNC(PyTypeObject *) _PyType_CalculateMetaclass(PyTypeObject *, PyObject * PyAPI_FUNC(PyObject *) _PyType_GetDocFromInternalDoc(const char *, const char *); PyAPI_FUNC(PyObject *) _PyType_GetTextSignatureFromInternalDoc(const char *, const char *); struct PyModuleDef; -PyAPI_FUNC(PyObject *) _PyType_GetModuleByDef(PyTypeObject *, struct PyModuleDef *); +PyAPI_FUNC(PyObject *) PyType_GetModuleByDef(PyTypeObject *, struct PyModuleDef *); struct _Py_Identifier; PyAPI_FUNC(int) PyObject_Print(PyObject *, FILE *, int); diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py index 089088d97a66e..8832292a9991a 100644 --- a/Lib/test/test_capi.py +++ b/Lib/test/test_capi.py @@ -1071,7 +1071,7 @@ def test_state_access(self): increment_count(1, 2, 3) def test_get_module_bad_def(self): - # _PyType_GetModuleByDef fails gracefully if it doesn't + # PyType_GetModuleByDef fails gracefully if it doesn't # find what it's looking for. # see bpo-46433 instance = self.module.StateAccessType() @@ -1079,7 +1079,7 @@ def test_get_module_bad_def(self): instance.getmodulebydef_bad_def() def test_get_module_static_in_mro(self): - # Here, the class _PyType_GetModuleByDef is looking for + # Here, the class PyType_GetModuleByDef is looking for # appears in the MRO after a static type (Exception). # see bpo-46433 class Subclass(BaseException, self.module.StateAccessType): diff --git a/Misc/NEWS.d/next/C API/2022-02-02-17-58-49.bpo-46613.__ZdpH.rst b/Misc/NEWS.d/next/C API/2022-02-02-17-58-49.bpo-46613.__ZdpH.rst new file mode 100644 index 0000000000000..9d0fca7a06b89 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-02-02-17-58-49.bpo-46613.__ZdpH.rst @@ -0,0 +1,2 @@ +Added function :c:func:`PyType_GetModuleByDef`, which allows accesss to +module state when a method's defining class is not available. diff --git a/Modules/_csv.c b/Modules/_csv.c index d2cb77f9f0334..f59d42a022e41 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -374,7 +374,7 @@ static char *dialect_kws[] = { static _csvstate * _csv_state_from_type(PyTypeObject *type, const char *name) { - PyObject *module = _PyType_GetModuleByDef(type, &_csvmodule); + PyObject *module = PyType_GetModuleByDef(type, &_csvmodule); if (module == NULL) { return NULL; } diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c index 1f6b852f6d99b..0974fc704ffbd 100644 --- a/Modules/_functoolsmodule.c +++ b/Modules/_functoolsmodule.c @@ -51,7 +51,7 @@ partial_call(partialobject *pto, PyObject *args, PyObject *kwargs); static inline _functools_state * get_functools_state_by_type(PyTypeObject *type) { - PyObject *module = _PyType_GetModuleByDef(type, &_functools_module); + PyObject *module = PyType_GetModuleByDef(type, &_functools_module); if (module == NULL) { return NULL; } diff --git a/Modules/_queuemodule.c b/Modules/_queuemodule.c index 413387fecc4cd..af19dd6c198b6 100644 --- a/Modules/_queuemodule.c +++ b/Modules/_queuemodule.c @@ -21,7 +21,7 @@ simplequeue_get_state(PyObject *module) } static struct PyModuleDef queuemodule; #define simplequeue_get_state_by_type(type) \ - (simplequeue_get_state(_PyType_GetModuleByDef(type, &queuemodule))) + (simplequeue_get_state(PyType_GetModuleByDef(type, &queuemodule))) typedef struct { PyObject_HEAD diff --git a/Modules/_randommodule.c b/Modules/_randommodule.c index 45860e342eb43..d96c0371ec7f8 100644 --- a/Modules/_randommodule.c +++ b/Modules/_randommodule.c @@ -99,7 +99,7 @@ get_random_state(PyObject *module) static struct PyModuleDef _randommodule; #define _randomstate_type(type) \ - (get_random_state(_PyType_GetModuleByDef(type, &_randommodule))) + (get_random_state(PyType_GetModuleByDef(type, &_randommodule))) typedef struct { PyObject_HEAD diff --git a/Modules/_sqlite/module.h b/Modules/_sqlite/module.h index 1d319f1ed5541..a248d044af8fd 100644 --- a/Modules/_sqlite/module.h +++ b/Modules/_sqlite/module.h @@ -74,7 +74,7 @@ extern struct PyModuleDef _sqlite3module; static inline pysqlite_state * pysqlite_get_state_by_type(PyTypeObject *tp) { - PyObject *module = _PyType_GetModuleByDef(tp, &_sqlite3module); + PyObject *module = PyType_GetModuleByDef(tp, &_sqlite3module); assert(module != NULL); return pysqlite_get_state(module); } diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 07f0580ef6fc5..d7e041fed3cb3 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -2988,8 +2988,8 @@ _ssl__SSLContext_impl(PyTypeObject *type, int proto_version) int result; /* slower approach, walk MRO and get borrowed reference to module. - * _PyType_GetModuleByDef is required for SSLContext subclasses */ - PyObject *module = _PyType_GetModuleByDef(type, &_sslmodule_def); + * PyType_GetModuleByDef is required for SSLContext subclasses */ + PyObject *module = PyType_GetModuleByDef(type, &_sslmodule_def); if (module == NULL) { PyErr_SetString(PyExc_RuntimeError, "Cannot find internal module state"); diff --git a/Modules/_ssl.h b/Modules/_ssl.h index 5fe6504a9dd7b..5593a455de4dc 100644 --- a/Modules/_ssl.h +++ b/Modules/_ssl.h @@ -42,7 +42,7 @@ get_ssl_state(PyObject *module) } #define get_state_type(type) \ - (get_ssl_state(_PyType_GetModuleByDef(type, &_sslmodule_def))) + (get_ssl_state(PyType_GetModuleByDef(type, &_sslmodule_def))) #define get_state_ctx(c) (((PySSLContext *)(c))->state) #define get_state_sock(s) (((PySSLSocket *)(s))->ctx->state) #define get_state_obj(o) ((_sslmodulestate *)PyType_GetModuleState(Py_TYPE(o))) diff --git a/Modules/_struct.c b/Modules/_struct.c index 7d05ec5ac7c50..a2e14e89d26d4 100644 --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -38,7 +38,7 @@ get_struct_state(PyObject *module) static struct PyModuleDef _structmodule; #define get_struct_state_structinst(self) \ - (get_struct_state(_PyType_GetModuleByDef(Py_TYPE(self), &_structmodule))) + (get_struct_state(PyType_GetModuleByDef(Py_TYPE(self), &_structmodule))) #define get_struct_state_iterinst(self) \ (get_struct_state(PyType_GetModule(Py_TYPE(self)))) diff --git a/Modules/_testmultiphase.c b/Modules/_testmultiphase.c index f7bde9895eb09..4905269177bf3 100644 --- a/Modules/_testmultiphase.c +++ b/Modules/_testmultiphase.c @@ -136,21 +136,21 @@ _testmultiphase.StateAccessType.get_defining_module Return the module of the defining class. -Also tests that result of _PyType_GetModuleByDef matches defining_class's +Also tests that result of PyType_GetModuleByDef matches defining_class's module. [clinic start generated code]*/ static PyObject * _testmultiphase_StateAccessType_get_defining_module_impl(StateAccessTypeObject *self, PyTypeObject *cls) -/*[clinic end generated code: output=ba2a14284a5d0921 input=356f999fc16e0933]*/ +/*[clinic end generated code: output=ba2a14284a5d0921 input=d2c7245c8a9d06f8]*/ { PyObject *retval; retval = PyType_GetModule(cls); if (retval == NULL) { return NULL; } - assert(_PyType_GetModuleByDef(Py_TYPE(self), &def_meth_state_access) == retval); + assert(PyType_GetModuleByDef(Py_TYPE(self), &def_meth_state_access) == retval); Py_INCREF(retval); return retval; } @@ -160,15 +160,15 @@ _testmultiphase.StateAccessType.getmodulebydef_bad_def cls: defining_class -Test that result of _PyType_GetModuleByDef with a bad def is NULL. +Test that result of PyType_GetModuleByDef with a bad def is NULL. [clinic start generated code]*/ static PyObject * _testmultiphase_StateAccessType_getmodulebydef_bad_def_impl(StateAccessTypeObject *self, PyTypeObject *cls) -/*[clinic end generated code: output=64509074dfcdbd31 input=906047715ee293cd]*/ +/*[clinic end generated code: output=64509074dfcdbd31 input=edaff09aa4788204]*/ { - _PyType_GetModuleByDef(Py_TYPE(self), &def_nonmodule); // should raise + PyType_GetModuleByDef(Py_TYPE(self), &def_nonmodule); // should raise assert(PyErr_Occurred()); return NULL; } diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index 072b79a75f3af..ca74f9d6977e2 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -786,7 +786,7 @@ local_new(PyTypeObject *type, PyObject *args, PyObject *kw) } } - PyObject *module = _PyType_GetModuleByDef(type, &thread_module); + PyObject *module = PyType_GetModuleByDef(type, &thread_module); thread_module_state *state = get_thread_state(module); localobject *self = (localobject *)type->tp_alloc(type, 0); @@ -925,7 +925,7 @@ _ldict(localobject *self, thread_module_state *state) static int local_setattro(localobject *self, PyObject *name, PyObject *v) { - PyObject *module = _PyType_GetModuleByDef(Py_TYPE(self), &thread_module); + PyObject *module = PyType_GetModuleByDef(Py_TYPE(self), &thread_module); thread_module_state *state = get_thread_state(module); PyObject *ldict = _ldict(self, state); @@ -977,7 +977,7 @@ static PyType_Spec local_type_spec = { static PyObject * local_getattro(localobject *self, PyObject *name) { - PyObject *module = _PyType_GetModuleByDef(Py_TYPE(self), &thread_module); + PyObject *module = PyType_GetModuleByDef(Py_TYPE(self), &thread_module); thread_module_state *state = get_thread_state(module); PyObject *ldict = _ldict(self, state); diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c index 4aaab0c531435..e516f54ab61e6 100644 --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -67,7 +67,7 @@ get_array_state(PyObject *module) } #define find_array_state_by_type(tp) \ - (get_array_state(_PyType_GetModuleByDef(tp, &arraymodule))) + (get_array_state(PyType_GetModuleByDef(tp, &arraymodule))) #define get_array_state_by_class(cls) \ (get_array_state(PyType_GetModule(cls))) diff --git a/Modules/cjkcodecs/multibytecodec.c b/Modules/cjkcodecs/multibytecodec.c index df48a0deec7e9..acfe96654a226 100644 --- a/Modules/cjkcodecs/multibytecodec.c +++ b/Modules/cjkcodecs/multibytecodec.c @@ -33,7 +33,7 @@ static struct PyModuleDef _multibytecodecmodule; static _multibytecodec_state * _multibyte_codec_find_state_by_type(PyTypeObject *type) { - PyObject *module = _PyType_GetModuleByDef(type, &_multibytecodecmodule); + PyObject *module = PyType_GetModuleByDef(type, &_multibytecodecmodule); assert(module != NULL); return _multibytecodec_get_state(module); } diff --git a/Modules/clinic/_testmultiphase.c.h b/Modules/clinic/_testmultiphase.c.h index 17c28d54ce575..b8ee93c6e19ea 100644 --- a/Modules/clinic/_testmultiphase.c.h +++ b/Modules/clinic/_testmultiphase.c.h @@ -8,7 +8,7 @@ PyDoc_STRVAR(_testmultiphase_StateAccessType_get_defining_module__doc__, "\n" "Return the module of the defining class.\n" "\n" -"Also tests that result of _PyType_GetModuleByDef matches defining_class\'s\n" +"Also tests that result of PyType_GetModuleByDef matches defining_class\'s\n" "module."); #define _TESTMULTIPHASE_STATEACCESSTYPE_GET_DEFINING_MODULE_METHODDEF \ @@ -39,7 +39,7 @@ PyDoc_STRVAR(_testmultiphase_StateAccessType_getmodulebydef_bad_def__doc__, "getmodulebydef_bad_def($self, /)\n" "--\n" "\n" -"Test that result of _PyType_GetModuleByDef with a bad def is NULL."); +"Test that result of PyType_GetModuleByDef with a bad def is NULL."); #define _TESTMULTIPHASE_STATEACCESSTYPE_GETMODULEBYDEF_BAD_DEF_METHODDEF \ {"getmodulebydef_bad_def", (PyCFunction)(void(*)(void))_testmultiphase_StateAccessType_getmodulebydef_bad_def, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _testmultiphase_StateAccessType_getmodulebydef_bad_def__doc__}, @@ -131,4 +131,4 @@ _testmultiphase_StateAccessType_get_count(StateAccessTypeObject *self, PyTypeObj exit: return return_value; } -/*[clinic end generated code: output=eb1b8c2ee6290be3 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=e8d074b4e6437438 input=a9049054013a1b77]*/ diff --git a/Objects/typeobject.c b/Objects/typeobject.c index ef3549ce68f7f..3f8f36a9c4648 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3714,13 +3714,9 @@ PyType_GetModuleState(PyTypeObject *type) /* Get the module of the first superclass where the module has the * given PyModuleDef. - * Implemented by walking the MRO, is relatively slow. - * - * This is internal API for experimentation within stdlib. Discussion: - * https://mail.python.org/archives/list/capi-sig at python.org/thread/T3P2QNLNLBRFHWSKYSTPMVEIL2EEKFJU/ */ PyObject * -_PyType_GetModuleByDef(PyTypeObject *type, struct PyModuleDef *def) +PyType_GetModuleByDef(PyTypeObject *type, struct PyModuleDef *def) { assert(PyType_Check(type)); @@ -3749,7 +3745,7 @@ _PyType_GetModuleByDef(PyTypeObject *type, struct PyModuleDef *def) PyErr_Format( PyExc_TypeError, - "_PyType_GetModuleByDef: No superclass of '%s' has the given module", + "PyType_GetModuleByDef: No superclass of '%s' has the given module", type->tp_name); return NULL; } diff --git a/Python/Python-tokenize.c b/Python/Python-tokenize.c index d3ebbe1331a15..6acfc2a7cfd26 100644 --- a/Python/Python-tokenize.c +++ b/Python/Python-tokenize.c @@ -13,7 +13,7 @@ get_tokenize_state(PyObject *module) { } #define _tokenize_get_state_by_type(type) \ - get_tokenize_state(_PyType_GetModuleByDef(type, &_tokenizemodule)) + get_tokenize_state(PyType_GetModuleByDef(type, &_tokenizemodule)) #include "clinic/Python-tokenize.c.h" From webhook-mailer at python.org Fri Feb 11 11:25:18 2022 From: webhook-mailer at python.org (sweeneyde) Date: Fri, 11 Feb 2022 16:25:18 -0000 Subject: [Python-checkins] bpo-46615: Don't crash when set operations mutate the sets (GH-31120) Message-ID: https://github.com/python/cpython/commit/4a66615ba736f84eadf9456bfd5d32a94cccf117 commit: 4a66615ba736f84eadf9456bfd5d32a94cccf117 branch: main author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2022-02-11T11:25:08-05:00 summary: bpo-46615: Don't crash when set operations mutate the sets (GH-31120) Ensure strong references are acquired whenever using `set_next()`. Added randomized test cases for `__eq__` methods that sometimes mutate sets when called. files: A Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst M Lib/test/test_set.py M Objects/setobject.c diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py index 77f3da40c063a..03b911920e179 100644 --- a/Lib/test/test_set.py +++ b/Lib/test/test_set.py @@ -1815,6 +1815,192 @@ def __eq__(self, o): s = {0} s.update(other) + +class TestOperationsMutating: + """Regression test for bpo-46615""" + + constructor1 = None + constructor2 = None + + def make_sets_of_bad_objects(self): + class Bad: + def __eq__(self, other): + if not enabled: + return False + if randrange(20) == 0: + set1.clear() + if randrange(20) == 0: + set2.clear() + return bool(randrange(2)) + def __hash__(self): + return randrange(2) + # Don't behave poorly during construction. + enabled = False + set1 = self.constructor1(Bad() for _ in range(randrange(50))) + set2 = self.constructor2(Bad() for _ in range(randrange(50))) + # Now start behaving poorly + enabled = True + return set1, set2 + + def check_set_op_does_not_crash(self, function): + for _ in range(100): + set1, set2 = self.make_sets_of_bad_objects() + try: + function(set1, set2) + except RuntimeError as e: + # Just make sure we don't crash here. + self.assertIn("changed size during iteration", str(e)) + + +class TestBinaryOpsMutating(TestOperationsMutating): + + def test_eq_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a == b) + + def test_ne_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a != b) + + def test_lt_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a < b) + + def test_le_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a <= b) + + def test_gt_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a > b) + + def test_ge_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a >= b) + + def test_and_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a & b) + + def test_or_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a | b) + + def test_sub_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a - b) + + def test_xor_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a ^ b) + + def test_iadd_with_mutation(self): + def f(a, b): + a &= b + self.check_set_op_does_not_crash(f) + + def test_ior_with_mutation(self): + def f(a, b): + a |= b + self.check_set_op_does_not_crash(f) + + def test_isub_with_mutation(self): + def f(a, b): + a -= b + self.check_set_op_does_not_crash(f) + + def test_ixor_with_mutation(self): + def f(a, b): + a ^= b + self.check_set_op_does_not_crash(f) + + def test_iteration_with_mutation(self): + def f1(a, b): + for x in a: + pass + for y in b: + pass + def f2(a, b): + for y in b: + pass + for x in a: + pass + def f3(a, b): + for x, y in zip(a, b): + pass + self.check_set_op_does_not_crash(f1) + self.check_set_op_does_not_crash(f2) + self.check_set_op_does_not_crash(f3) + + +class TestBinaryOpsMutating_Set_Set(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = set + constructor2 = set + +class TestBinaryOpsMutating_Subclass_Subclass(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = SetSubclass + +class TestBinaryOpsMutating_Set_Subclass(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = set + constructor2 = SetSubclass + +class TestBinaryOpsMutating_Subclass_Set(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = set + + +class TestMethodsMutating(TestOperationsMutating): + + def test_issubset_with_mutation(self): + self.check_set_op_does_not_crash(set.issubset) + + def test_issuperset_with_mutation(self): + self.check_set_op_does_not_crash(set.issuperset) + + def test_intersection_with_mutation(self): + self.check_set_op_does_not_crash(set.intersection) + + def test_union_with_mutation(self): + self.check_set_op_does_not_crash(set.union) + + def test_difference_with_mutation(self): + self.check_set_op_does_not_crash(set.difference) + + def test_symmetric_difference_with_mutation(self): + self.check_set_op_does_not_crash(set.symmetric_difference) + + def test_isdisjoint_with_mutation(self): + self.check_set_op_does_not_crash(set.isdisjoint) + + def test_difference_update_with_mutation(self): + self.check_set_op_does_not_crash(set.difference_update) + + def test_intersection_update_with_mutation(self): + self.check_set_op_does_not_crash(set.intersection_update) + + def test_symmetric_difference_update_with_mutation(self): + self.check_set_op_does_not_crash(set.symmetric_difference_update) + + def test_update_with_mutation(self): + self.check_set_op_does_not_crash(set.update) + + +class TestMethodsMutating_Set_Set(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = set + +class TestMethodsMutating_Subclass_Subclass(TestMethodsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = SetSubclass + +class TestMethodsMutating_Set_Subclass(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = SetSubclass + +class TestMethodsMutating_Subclass_Set(TestMethodsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = set + +class TestMethodsMutating_Set_Dict(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = dict.fromkeys + +class TestMethodsMutating_Set_List(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = list + + # Application tests (based on David Eppstein's graph recipes ==================================== def powerset(U): diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst new file mode 100644 index 0000000000000..6dee92a546e33 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst @@ -0,0 +1 @@ +When iterating over sets internally in ``setobject.c``, acquire strong references to the resulting items from the set. This prevents crashes in corner-cases of various set operations where the set gets mutated. diff --git a/Objects/setobject.c b/Objects/setobject.c index fe124945b1c7e..0dd28402afbd2 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -1205,17 +1205,21 @@ set_intersection(PySetObject *so, PyObject *other) while (set_next((PySetObject *)other, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = set_contains_entry(so, key, hash); if (rv < 0) { Py_DECREF(result); + Py_DECREF(key); return NULL; } if (rv) { if (set_add_entry(result, key, hash)) { Py_DECREF(result); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } return (PyObject *)result; } @@ -1355,11 +1359,16 @@ set_isdisjoint(PySetObject *so, PyObject *other) other = tmp; } while (set_next((PySetObject *)other, &pos, &entry)) { - rv = set_contains_entry(so, entry->key, entry->hash); - if (rv < 0) + PyObject *key = entry->key; + Py_INCREF(key); + rv = set_contains_entry(so, key, entry->hash); + Py_DECREF(key); + if (rv < 0) { return NULL; - if (rv) + } + if (rv) { Py_RETURN_FALSE; + } } Py_RETURN_TRUE; } @@ -1418,11 +1427,16 @@ set_difference_update_internal(PySetObject *so, PyObject *other) Py_INCREF(other); } - while (set_next((PySetObject *)other, &pos, &entry)) - if (set_discard_entry(so, entry->key, entry->hash) < 0) { + while (set_next((PySetObject *)other, &pos, &entry)) { + PyObject *key = entry->key; + Py_INCREF(key); + if (set_discard_entry(so, key, entry->hash) < 0) { Py_DECREF(other); + Py_DECREF(key); return -1; } + Py_DECREF(key); + } Py_DECREF(other); } else { @@ -1513,17 +1527,21 @@ set_difference(PySetObject *so, PyObject *other) while (set_next(so, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = _PyDict_Contains_KnownHash(other, key, hash); if (rv < 0) { Py_DECREF(result); + Py_DECREF(key); return NULL; } if (!rv) { if (set_add_entry((PySetObject *)result, key, hash)) { Py_DECREF(result); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } return result; } @@ -1532,17 +1550,21 @@ set_difference(PySetObject *so, PyObject *other) while (set_next(so, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = set_contains_entry((PySetObject *)other, key, hash); if (rv < 0) { Py_DECREF(result); + Py_DECREF(key); return NULL; } if (!rv) { if (set_add_entry((PySetObject *)result, key, hash)) { Py_DECREF(result); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } return result; } @@ -1639,17 +1661,21 @@ set_symmetric_difference_update(PySetObject *so, PyObject *other) while (set_next(otherset, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = set_discard_entry(so, key, hash); if (rv < 0) { Py_DECREF(otherset); + Py_DECREF(key); return NULL; } if (rv == DISCARD_NOTFOUND) { if (set_add_entry(so, key, hash)) { Py_DECREF(otherset); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } Py_DECREF(otherset); Py_RETURN_NONE; @@ -1724,11 +1750,16 @@ set_issubset(PySetObject *so, PyObject *other) Py_RETURN_FALSE; while (set_next(so, &pos, &entry)) { - rv = set_contains_entry((PySetObject *)other, entry->key, entry->hash); - if (rv < 0) + PyObject *key = entry->key; + Py_INCREF(key); + rv = set_contains_entry((PySetObject *)other, key, entry->hash); + Py_DECREF(key); + if (rv < 0) { return NULL; - if (!rv) + } + if (!rv) { Py_RETURN_FALSE; + } } Py_RETURN_TRUE; } From webhook-mailer at python.org Fri Feb 11 15:39:49 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 11 Feb 2022 20:39:49 -0000 Subject: [Python-checkins] bpo-46483: [doc] pathlib classes no longer support parameterized generics (GH-31281) Message-ID: https://github.com/python/cpython/commit/e0bc8ee945af96f9395659bbe3cc30b082e7a361 commit: e0bc8ee945af96f9395659bbe3cc30b082e7a361 branch: main author: Alex Waygood committer: serhiy-storchaka date: 2022-02-11T22:39:37+02:00 summary: bpo-46483: [doc] pathlib classes no longer support parameterized generics (GH-31281) Remove pathlib classes from the list in stdtypes.rst of classes that can be parameterized at runtime. files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 3465320c87d8b..cdc4dad067814 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4997,10 +4997,6 @@ list is non-exhaustive. * :class:`functools.cached_property` * :class:`functools.partialmethod` * :class:`os.PathLike` -* :class:`pathlib.Path` -* :class:`pathlib.PurePath` -* :class:`pathlib.PurePosixPath` -* :class:`pathlib.PureWindowsPath` * :class:`queue.LifoQueue` * :class:`queue.Queue` * :class:`queue.PriorityQueue` From webhook-mailer at python.org Fri Feb 11 15:44:25 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 11 Feb 2022 20:44:25 -0000 Subject: [Python-checkins] bpo-46615: Don't crash when set operations mutate the sets (GH-31120) Message-ID: https://github.com/python/cpython/commit/1f5fe9962f768c8bfd4ed06a22532d31d3424dc9 commit: 1f5fe9962f768c8bfd4ed06a22532d31d3424dc9 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-11T12:44:17-08:00 summary: bpo-46615: Don't crash when set operations mutate the sets (GH-31120) Ensure strong references are acquired whenever using `set_next()`. Added randomized test cases for `__eq__` methods that sometimes mutate sets when called. (cherry picked from commit 4a66615ba736f84eadf9456bfd5d32a94cccf117) Co-authored-by: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> files: A Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst M Lib/test/test_set.py M Objects/setobject.c diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py index 29bb39df76c8a..824eddbed775a 100644 --- a/Lib/test/test_set.py +++ b/Lib/test/test_set.py @@ -1766,6 +1766,192 @@ def __eq__(self, o): s = {0} s.update(other) + +class TestOperationsMutating: + """Regression test for bpo-46615""" + + constructor1 = None + constructor2 = None + + def make_sets_of_bad_objects(self): + class Bad: + def __eq__(self, other): + if not enabled: + return False + if randrange(20) == 0: + set1.clear() + if randrange(20) == 0: + set2.clear() + return bool(randrange(2)) + def __hash__(self): + return randrange(2) + # Don't behave poorly during construction. + enabled = False + set1 = self.constructor1(Bad() for _ in range(randrange(50))) + set2 = self.constructor2(Bad() for _ in range(randrange(50))) + # Now start behaving poorly + enabled = True + return set1, set2 + + def check_set_op_does_not_crash(self, function): + for _ in range(100): + set1, set2 = self.make_sets_of_bad_objects() + try: + function(set1, set2) + except RuntimeError as e: + # Just make sure we don't crash here. + self.assertIn("changed size during iteration", str(e)) + + +class TestBinaryOpsMutating(TestOperationsMutating): + + def test_eq_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a == b) + + def test_ne_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a != b) + + def test_lt_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a < b) + + def test_le_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a <= b) + + def test_gt_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a > b) + + def test_ge_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a >= b) + + def test_and_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a & b) + + def test_or_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a | b) + + def test_sub_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a - b) + + def test_xor_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a ^ b) + + def test_iadd_with_mutation(self): + def f(a, b): + a &= b + self.check_set_op_does_not_crash(f) + + def test_ior_with_mutation(self): + def f(a, b): + a |= b + self.check_set_op_does_not_crash(f) + + def test_isub_with_mutation(self): + def f(a, b): + a -= b + self.check_set_op_does_not_crash(f) + + def test_ixor_with_mutation(self): + def f(a, b): + a ^= b + self.check_set_op_does_not_crash(f) + + def test_iteration_with_mutation(self): + def f1(a, b): + for x in a: + pass + for y in b: + pass + def f2(a, b): + for y in b: + pass + for x in a: + pass + def f3(a, b): + for x, y in zip(a, b): + pass + self.check_set_op_does_not_crash(f1) + self.check_set_op_does_not_crash(f2) + self.check_set_op_does_not_crash(f3) + + +class TestBinaryOpsMutating_Set_Set(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = set + constructor2 = set + +class TestBinaryOpsMutating_Subclass_Subclass(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = SetSubclass + +class TestBinaryOpsMutating_Set_Subclass(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = set + constructor2 = SetSubclass + +class TestBinaryOpsMutating_Subclass_Set(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = set + + +class TestMethodsMutating(TestOperationsMutating): + + def test_issubset_with_mutation(self): + self.check_set_op_does_not_crash(set.issubset) + + def test_issuperset_with_mutation(self): + self.check_set_op_does_not_crash(set.issuperset) + + def test_intersection_with_mutation(self): + self.check_set_op_does_not_crash(set.intersection) + + def test_union_with_mutation(self): + self.check_set_op_does_not_crash(set.union) + + def test_difference_with_mutation(self): + self.check_set_op_does_not_crash(set.difference) + + def test_symmetric_difference_with_mutation(self): + self.check_set_op_does_not_crash(set.symmetric_difference) + + def test_isdisjoint_with_mutation(self): + self.check_set_op_does_not_crash(set.isdisjoint) + + def test_difference_update_with_mutation(self): + self.check_set_op_does_not_crash(set.difference_update) + + def test_intersection_update_with_mutation(self): + self.check_set_op_does_not_crash(set.intersection_update) + + def test_symmetric_difference_update_with_mutation(self): + self.check_set_op_does_not_crash(set.symmetric_difference_update) + + def test_update_with_mutation(self): + self.check_set_op_does_not_crash(set.update) + + +class TestMethodsMutating_Set_Set(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = set + +class TestMethodsMutating_Subclass_Subclass(TestMethodsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = SetSubclass + +class TestMethodsMutating_Set_Subclass(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = SetSubclass + +class TestMethodsMutating_Subclass_Set(TestMethodsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = set + +class TestMethodsMutating_Set_Dict(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = dict.fromkeys + +class TestMethodsMutating_Set_List(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = list + + # Application tests (based on David Eppstein's graph recipes ==================================== def powerset(U): diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst new file mode 100644 index 0000000000000..6dee92a546e33 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst @@ -0,0 +1 @@ +When iterating over sets internally in ``setobject.c``, acquire strong references to the resulting items from the set. This prevents crashes in corner-cases of various set operations where the set gets mutated. diff --git a/Objects/setobject.c b/Objects/setobject.c index 6524963b8abc3..e8ba32e5781a8 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -1204,17 +1204,21 @@ set_intersection(PySetObject *so, PyObject *other) while (set_next((PySetObject *)other, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = set_contains_entry(so, key, hash); if (rv < 0) { Py_DECREF(result); + Py_DECREF(key); return NULL; } if (rv) { if (set_add_entry(result, key, hash)) { Py_DECREF(result); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } return (PyObject *)result; } @@ -1354,11 +1358,16 @@ set_isdisjoint(PySetObject *so, PyObject *other) other = tmp; } while (set_next((PySetObject *)other, &pos, &entry)) { - rv = set_contains_entry(so, entry->key, entry->hash); - if (rv < 0) + PyObject *key = entry->key; + Py_INCREF(key); + rv = set_contains_entry(so, key, entry->hash); + Py_DECREF(key); + if (rv < 0) { return NULL; - if (rv) + } + if (rv) { Py_RETURN_FALSE; + } } Py_RETURN_TRUE; } @@ -1417,11 +1426,16 @@ set_difference_update_internal(PySetObject *so, PyObject *other) Py_INCREF(other); } - while (set_next((PySetObject *)other, &pos, &entry)) - if (set_discard_entry(so, entry->key, entry->hash) < 0) { + while (set_next((PySetObject *)other, &pos, &entry)) { + PyObject *key = entry->key; + Py_INCREF(key); + if (set_discard_entry(so, key, entry->hash) < 0) { Py_DECREF(other); + Py_DECREF(key); return -1; } + Py_DECREF(key); + } Py_DECREF(other); } else { @@ -1512,17 +1526,21 @@ set_difference(PySetObject *so, PyObject *other) while (set_next(so, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = _PyDict_Contains_KnownHash(other, key, hash); if (rv < 0) { Py_DECREF(result); + Py_DECREF(key); return NULL; } if (!rv) { if (set_add_entry((PySetObject *)result, key, hash)) { Py_DECREF(result); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } return result; } @@ -1531,17 +1549,21 @@ set_difference(PySetObject *so, PyObject *other) while (set_next(so, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = set_contains_entry((PySetObject *)other, key, hash); if (rv < 0) { Py_DECREF(result); + Py_DECREF(key); return NULL; } if (!rv) { if (set_add_entry((PySetObject *)result, key, hash)) { Py_DECREF(result); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } return result; } @@ -1638,17 +1660,21 @@ set_symmetric_difference_update(PySetObject *so, PyObject *other) while (set_next(otherset, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = set_discard_entry(so, key, hash); if (rv < 0) { Py_DECREF(otherset); + Py_DECREF(key); return NULL; } if (rv == DISCARD_NOTFOUND) { if (set_add_entry(so, key, hash)) { Py_DECREF(otherset); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } Py_DECREF(otherset); Py_RETURN_NONE; @@ -1723,11 +1749,16 @@ set_issubset(PySetObject *so, PyObject *other) Py_RETURN_FALSE; while (set_next(so, &pos, &entry)) { - rv = set_contains_entry((PySetObject *)other, entry->key, entry->hash); - if (rv < 0) + PyObject *key = entry->key; + Py_INCREF(key); + rv = set_contains_entry((PySetObject *)other, key, entry->hash); + Py_DECREF(key); + if (rv < 0) { return NULL; - if (!rv) + } + if (!rv) { Py_RETURN_FALSE; + } } Py_RETURN_TRUE; } From webhook-mailer at python.org Fri Feb 11 16:03:58 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 11 Feb 2022 21:03:58 -0000 Subject: [Python-checkins] bpo-46483: [doc] pathlib classes no longer support parameterized generics (GH-31281) Message-ID: https://github.com/python/cpython/commit/a7c1cc41696740bb528f4d24816c59e38b8be345 commit: a7c1cc41696740bb528f4d24816c59e38b8be345 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-11T13:03:45-08:00 summary: bpo-46483: [doc] pathlib classes no longer support parameterized generics (GH-31281) Remove pathlib classes from the list in stdtypes.rst of classes that can be parameterized at runtime. (cherry picked from commit e0bc8ee945af96f9395659bbe3cc30b082e7a361) Co-authored-by: Alex Waygood files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index ba5e0d5a164e8..b10909d81af0a 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4903,10 +4903,6 @@ list is non-exhaustive. * :class:`functools.cached_property` * :class:`functools.partialmethod` * :class:`os.PathLike` -* :class:`pathlib.Path` -* :class:`pathlib.PurePath` -* :class:`pathlib.PurePosixPath` -* :class:`pathlib.PureWindowsPath` * :class:`queue.LifoQueue` * :class:`queue.Queue` * :class:`queue.PriorityQueue` From webhook-mailer at python.org Fri Feb 11 16:04:06 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 11 Feb 2022 21:04:06 -0000 Subject: [Python-checkins] bpo-46483: [doc] pathlib classes no longer support parameterized generics (GH-31281) Message-ID: https://github.com/python/cpython/commit/f2fbfbe0ec57ec39b3099dae62571a48d5fe8729 commit: f2fbfbe0ec57ec39b3099dae62571a48d5fe8729 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-11T13:04:01-08:00 summary: bpo-46483: [doc] pathlib classes no longer support parameterized generics (GH-31281) Remove pathlib classes from the list in stdtypes.rst of classes that can be parameterized at runtime. (cherry picked from commit e0bc8ee945af96f9395659bbe3cc30b082e7a361) Co-authored-by: Alex Waygood files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 08e7c0db8cc97..b330335773103 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4959,10 +4959,6 @@ list is non-exhaustive. * :class:`functools.cached_property` * :class:`functools.partialmethod` * :class:`os.PathLike` -* :class:`pathlib.Path` -* :class:`pathlib.PurePath` -* :class:`pathlib.PurePosixPath` -* :class:`pathlib.PureWindowsPath` * :class:`queue.LifoQueue` * :class:`queue.Queue` * :class:`queue.PriorityQueue` From webhook-mailer at python.org Fri Feb 11 19:48:34 2022 From: webhook-mailer at python.org (vstinner) Date: Sat, 12 Feb 2022 00:48:34 -0000 Subject: [Python-checkins] bpo-46355: Amend What's New in Python 3.11 C API wording (GH-31288) Message-ID: https://github.com/python/cpython/commit/ba5725171d9c411fc4764349205eff5cfc028797 commit: ba5725171d9c411fc4764349205eff5cfc028797 branch: main author: Erlend Egeberg Aasland committer: vstinner date: 2022-02-12T01:48:26+01:00 summary: bpo-46355: Amend What's New in Python 3.11 C API wording (GH-31288) files: M Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 95fce1e8e1082..5e7c89e68426e 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -874,8 +874,8 @@ Porting to Python 3.11 #endif Or use `the pythoncapi_compat project - `__ to get these functions - on old Python functions. + `__ to get these APIs + on older Python versions. * Changes of the :c:type:`PyThreadState` structure members: From webhook-mailer at python.org Fri Feb 11 23:21:42 2022 From: webhook-mailer at python.org (ethanfurman) Date: Sat, 12 Feb 2022 04:21:42 -0000 Subject: [Python-checkins] bpo-46586: Fix documentation links (GH-31216) Message-ID: https://github.com/python/cpython/commit/9d9cfd61ec3cbe84dbc25c74f664877f3d02b8ef commit: 9d9cfd61ec3cbe84dbc25c74f664877f3d02b8ef branch: main author: Meer Suri <46469858+meersuri at users.noreply.github.com> committer: ethanfurman date: 2022-02-11T20:21:38-08:00 summary: bpo-46586: Fix documentation links (GH-31216) * Fix enum.property documentation link files: M Doc/library/enum.rst diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 672e256c77c0d..52ef0094cb71f 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -110,7 +110,7 @@ Module Contents :class:`StrEnum` defaults to the lower-cased version of the member name, while other Enums default to 1 and increase from there. - :func:`property` + :func:`~enum.property` Allows :class:`Enum` members to have attributes without conflicting with member names. From webhook-mailer at python.org Sat Feb 12 03:27:10 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 12 Feb 2022 08:27:10 -0000 Subject: [Python-checkins] bpo-45948: Remove constructor discrepancy in C version of ElementTree.XMLParser (GH-31152) Message-ID: https://github.com/python/cpython/commit/168fd6453b5de15236116f9261d64601d92571ac commit: 168fd6453b5de15236116f9261d64601d92571ac branch: main author: Jacob Walls committer: serhiy-storchaka date: 2022-02-12T10:27:02+02:00 summary: bpo-45948: Remove constructor discrepancy in C version of ElementTree.XMLParser (GH-31152) Both implementations accept target=None now. files: A Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst M Lib/test/test_xml_etree.py M Modules/_elementtree.c M Modules/clinic/_elementtree.c.h diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index 285559a872a65..a25f536134c7b 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -758,6 +758,15 @@ def end_ns(self, prefix): ('end-ns', ''), ]) + def test_initialize_parser_without_target(self): + # Explicit None + parser = ET.XMLParser(target=None) + self.assertIsInstance(parser.target, ET.TreeBuilder) + + # Implicit None + parser2 = ET.XMLParser() + self.assertIsInstance(parser2.target, ET.TreeBuilder) + def test_children(self): # Test Element children iteration diff --git a/Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst b/Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst new file mode 100644 index 0000000000000..42dc114b5ad60 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst @@ -0,0 +1,5 @@ +Fixed a discrepancy in the C implementation of the +:mod:`xml.etree.ElementTree` module. Now, instantiating an +:class:`xml.etree.ElementTree.XMLParser` with a ``target=None`` +keyword provides a default :class:`xml.etree.ElementTree.TreeBuilder` +target as the Python implementation does. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index f155dc72152d4..f54c28447ca8f 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3637,7 +3637,7 @@ ignore_attribute_error(PyObject *value) _elementtree.XMLParser.__init__ * - target: object = NULL + target: object = None encoding: str(accept={str, NoneType}) = None [clinic start generated code]*/ @@ -3645,7 +3645,7 @@ _elementtree.XMLParser.__init__ static int _elementtree_XMLParser___init___impl(XMLParserObject *self, PyObject *target, const char *encoding) -/*[clinic end generated code: output=3ae45ec6cdf344e4 input=53e35a829ae043e8]*/ +/*[clinic end generated code: output=3ae45ec6cdf344e4 input=7e716dd6e4f3e439]*/ { self->entity = PyDict_New(); if (!self->entity) @@ -3670,7 +3670,7 @@ _elementtree_XMLParser___init___impl(XMLParserObject *self, PyObject *target, (unsigned long)_Py_HashSecret.expat.hashsalt); } - if (target) { + if (target != Py_None) { Py_INCREF(target); } else { target = treebuilder_new(&TreeBuilder_Type, NULL, NULL); diff --git a/Modules/clinic/_elementtree.c.h b/Modules/clinic/_elementtree.c.h index 2c9ba47823bf4..5ed5a6cadd9d5 100644 --- a/Modules/clinic/_elementtree.c.h +++ b/Modules/clinic/_elementtree.c.h @@ -807,7 +807,7 @@ _elementtree_XMLParser___init__(PyObject *self, PyObject *args, PyObject *kwargs PyObject * const *fastargs; Py_ssize_t nargs = PyTuple_GET_SIZE(args); Py_ssize_t noptargs = nargs + (kwargs ? PyDict_GET_SIZE(kwargs) : 0) - 0; - PyObject *target = NULL; + PyObject *target = Py_None; const char *encoding = NULL; fastargs = _PyArg_UnpackKeywords(_PyTuple_CAST(args)->ob_item, nargs, kwargs, NULL, &_parser, 0, 0, 0, argsbuf); @@ -915,4 +915,4 @@ _elementtree_XMLParser__setevents(XMLParserObject *self, PyObject *const *args, exit: return return_value; } -/*[clinic end generated code: output=1385b5e5688f3614 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=992733cfc7390590 input=a9049054013a1b77]*/ From webhook-mailer at python.org Sat Feb 12 03:50:19 2022 From: webhook-mailer at python.org (miss-islington) Date: Sat, 12 Feb 2022 08:50:19 -0000 Subject: [Python-checkins] bpo-45948: Remove constructor discrepancy in C version of ElementTree.XMLParser (GH-31152) Message-ID: https://github.com/python/cpython/commit/b7a65c939c93e7910abc8c9d4a129ff385714824 commit: b7a65c939c93e7910abc8c9d4a129ff385714824 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-12T00:50:11-08:00 summary: bpo-45948: Remove constructor discrepancy in C version of ElementTree.XMLParser (GH-31152) Both implementations accept target=None now. (cherry picked from commit 168fd6453b5de15236116f9261d64601d92571ac) Co-authored-by: Jacob Walls files: A Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst M Lib/test/test_xml_etree.py M Modules/_elementtree.c M Modules/clinic/_elementtree.c.h diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index 285559a872a65..a25f536134c7b 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -758,6 +758,15 @@ def end_ns(self, prefix): ('end-ns', ''), ]) + def test_initialize_parser_without_target(self): + # Explicit None + parser = ET.XMLParser(target=None) + self.assertIsInstance(parser.target, ET.TreeBuilder) + + # Implicit None + parser2 = ET.XMLParser() + self.assertIsInstance(parser2.target, ET.TreeBuilder) + def test_children(self): # Test Element children iteration diff --git a/Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst b/Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst new file mode 100644 index 0000000000000..42dc114b5ad60 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst @@ -0,0 +1,5 @@ +Fixed a discrepancy in the C implementation of the +:mod:`xml.etree.ElementTree` module. Now, instantiating an +:class:`xml.etree.ElementTree.XMLParser` with a ``target=None`` +keyword provides a default :class:`xml.etree.ElementTree.TreeBuilder` +target as the Python implementation does. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 9dadeef712938..8637bae0ae143 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3636,7 +3636,7 @@ ignore_attribute_error(PyObject *value) _elementtree.XMLParser.__init__ * - target: object = NULL + target: object = None encoding: str(accept={str, NoneType}) = None [clinic start generated code]*/ @@ -3644,7 +3644,7 @@ _elementtree.XMLParser.__init__ static int _elementtree_XMLParser___init___impl(XMLParserObject *self, PyObject *target, const char *encoding) -/*[clinic end generated code: output=3ae45ec6cdf344e4 input=53e35a829ae043e8]*/ +/*[clinic end generated code: output=3ae45ec6cdf344e4 input=7e716dd6e4f3e439]*/ { self->entity = PyDict_New(); if (!self->entity) @@ -3669,7 +3669,7 @@ _elementtree_XMLParser___init___impl(XMLParserObject *self, PyObject *target, (unsigned long)_Py_HashSecret.expat.hashsalt); } - if (target) { + if (target != Py_None) { Py_INCREF(target); } else { target = treebuilder_new(&TreeBuilder_Type, NULL, NULL); diff --git a/Modules/clinic/_elementtree.c.h b/Modules/clinic/_elementtree.c.h index 2c9ba47823bf4..5ed5a6cadd9d5 100644 --- a/Modules/clinic/_elementtree.c.h +++ b/Modules/clinic/_elementtree.c.h @@ -807,7 +807,7 @@ _elementtree_XMLParser___init__(PyObject *self, PyObject *args, PyObject *kwargs PyObject * const *fastargs; Py_ssize_t nargs = PyTuple_GET_SIZE(args); Py_ssize_t noptargs = nargs + (kwargs ? PyDict_GET_SIZE(kwargs) : 0) - 0; - PyObject *target = NULL; + PyObject *target = Py_None; const char *encoding = NULL; fastargs = _PyArg_UnpackKeywords(_PyTuple_CAST(args)->ob_item, nargs, kwargs, NULL, &_parser, 0, 0, 0, argsbuf); @@ -915,4 +915,4 @@ _elementtree_XMLParser__setevents(XMLParserObject *self, PyObject *const *args, exit: return return_value; } -/*[clinic end generated code: output=1385b5e5688f3614 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=992733cfc7390590 input=a9049054013a1b77]*/ From webhook-mailer at python.org Sat Feb 12 10:30:04 2022 From: webhook-mailer at python.org (corona10) Date: Sat, 12 Feb 2022 15:30:04 -0000 Subject: [Python-checkins] bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) Message-ID: https://github.com/python/cpython/commit/8aaaf7e182e22026c3487a3b86d4d7d4f0f5f778 commit: 8aaaf7e182e22026c3487a3b86d4d7d4f0f5f778 branch: main author: Cyril Jouve committer: corona10 date: 2022-02-13T00:29:41+09:00 summary: bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) files: A Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst M Modules/expat/expat.h M Modules/expat/xmlparse.c M Modules/expat/xmlrole.c M Modules/expat/xmltok.c M Modules/expat/xmltok_ns.c diff --git a/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst new file mode 100644 index 0000000000000..9c1f24c0e5171 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst @@ -0,0 +1 @@ +expat: Update libexpat from 2.4.1 to 2.4.4 diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index b7d6d354801b3..4c5704fd9336b 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -11,7 +11,7 @@ Copyright (c) 2000-2005 Fred L. Drake, Jr. Copyright (c) 2001-2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Cristian Rodr?guez Copyright (c) 2016 Thomas Beutlich Copyright (c) 2017 Rhodri James @@ -1041,7 +1041,7 @@ XML_SetBillionLaughsAttackProtectionActivationThreshold( */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 4 -#define XML_MICRO_VERSION 1 +#define XML_MICRO_VERSION 4 #ifdef __cplusplus } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 034a03c30851a..4b43e61321691 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 8539b9040d9d901366a62560a064af7cb99811335784b363abc039c5b0ebc416 (2.4.1+) +/* 2e2c8ce5f11a473d65ec313ab20ceee6afefb355f5405afc06e7204e2e41c8c0 (2.4.4+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -13,7 +13,7 @@ Copyright (c) 2002-2016 Karl Waclawek Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016 Eric Rahm - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Gaurav Copyright (c) 2016 Thomas Beutlich Copyright (c) 2016 Gustavo Grieco @@ -32,6 +32,8 @@ Copyright (c) 2019 David Loffredo Copyright (c) 2019-2020 Ben Wagner Copyright (c) 2019 Vadim Zeitlin + Copyright (c) 2021 Dong-hee Na + Copyright (c) 2022 Samanta Navarro Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -54,6 +56,10 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#define XML_BUILDING_EXPAT 1 + +#include + #if ! defined(_GNU_SOURCE) # define _GNU_SOURCE 1 /* syscall prototype */ #endif @@ -84,14 +90,10 @@ # include #endif -#define XML_BUILDING_EXPAT 1 - #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "ascii.h" #include "expat.h" #include "siphash.h" @@ -973,7 +975,7 @@ parserCreate(const XML_Char *encodingName, if (memsuite) { XML_Memory_Handling_Suite *mtemp; - parser = (XML_Parser)memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); + parser = memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); if (parser != NULL) { mtemp = (XML_Memory_Handling_Suite *)&(parser->m_mem); mtemp->malloc_fcn = memsuite->malloc_fcn; @@ -2066,6 +2068,11 @@ XML_GetBuffer(XML_Parser parser, int len) { keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; + /* Detect and prevent integer overflow */ + if (keep > INT_MAX - neededSize) { + parser->m_errorCode = XML_ERROR_NO_MEMORY; + return NULL; + } neededSize += keep; #endif /* defined XML_CONTEXT_BYTES */ if (neededSize @@ -3260,13 +3267,38 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, /* get the attributes from the tokenizer */ n = XmlGetAttributes(enc, attStr, parser->m_attsSize, parser->m_atts); + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - nDefaultAtts) { + return XML_ERROR_NO_MEMORY; + } + if (n + nDefaultAtts > parser->m_attsSize) { int oldAttsSize = parser->m_attsSize; ATTRIBUTE *temp; #ifdef XML_ATTR_INFO XML_AttrInfo *temp2; #endif + + /* Detect and prevent integer overflow */ + if ((nDefaultAtts > INT_MAX - INIT_ATTS_SIZE) + || (n > INT_MAX - (nDefaultAtts + INIT_ATTS_SIZE))) { + return XML_ERROR_NO_MEMORY; + } + parser->m_attsSize = n + nDefaultAtts + INIT_ATTS_SIZE; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(ATTRIBUTE)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (ATTRIBUTE *)REALLOC(parser, (void *)parser->m_atts, parser->m_attsSize * sizeof(ATTRIBUTE)); if (temp == NULL) { @@ -3275,6 +3307,17 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, } parser->m_atts = temp; #ifdef XML_ATTR_INFO + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +# if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(XML_AttrInfo)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +# endif + temp2 = (XML_AttrInfo *)REALLOC(parser, (void *)parser->m_attInfo, parser->m_attsSize * sizeof(XML_AttrInfo)); if (temp2 == NULL) { @@ -3413,7 +3456,13 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, if (nPrefixes) { int j; /* hash table index */ unsigned long version = parser->m_nsAttsVersion; - int nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(unsigned int) * 8 /* bits per byte */) { + return XML_ERROR_NO_MEMORY; + } + + unsigned int nsAttsSize = 1u << parser->m_nsAttsPower; unsigned char oldNsAttsPower = parser->m_nsAttsPower; /* size of hash table must be at least 2 * (# of prefixed attributes) */ if ((nPrefixes << 1) @@ -3424,7 +3473,28 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, ; if (parser->m_nsAttsPower < 3) parser->m_nsAttsPower = 3; - nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(nsAttsSize) * 8 /* bits per byte */) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } + + nsAttsSize = 1u << parser->m_nsAttsPower; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (nsAttsSize > (size_t)(-1) / sizeof(NS_ATT)) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (NS_ATT *)REALLOC(parser, parser->m_nsAtts, nsAttsSize * sizeof(NS_ATT)); if (! temp) { @@ -3582,9 +3652,31 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, tagNamePtr->prefixLen = prefixLen; for (i = 0; localPart[i++];) ; /* i includes null terminator */ + + /* Detect and prevent integer overflow */ + if (binding->uriLen > INT_MAX - prefixLen + || i > INT_MAX - (binding->uriLen + prefixLen)) { + return XML_ERROR_NO_MEMORY; + } + n = i + binding->uriLen + prefixLen; if (n > binding->uriAlloc) { TAG *p; + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(n + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + uri = (XML_Char *)MALLOC(parser, (n + EXPAND_SPARE) * sizeof(XML_Char)); if (! uri) return XML_ERROR_NO_MEMORY; @@ -3680,6 +3772,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (parser->m_freeBindingList) { b = parser->m_freeBindingList; if (len > b->uriAlloc) { + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + XML_Char *temp = (XML_Char *)REALLOC( parser, b->uri, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (temp == NULL) @@ -3692,6 +3799,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, b = (BINDING *)MALLOC(parser, sizeof(BINDING)); if (! b) return XML_ERROR_NO_MEMORY; + + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + b->uri = (XML_Char *)MALLOC(parser, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (! b->uri) { @@ -3976,7 +4098,7 @@ initializeEncoding(XML_Parser parser) { const char *s; #ifdef XML_UNICODE char encodingBuf[128]; - /* See comments about `protoclEncodingName` in parserInit() */ + /* See comments about `protocolEncodingName` in parserInit() */ if (! parser->m_protocolEncodingName) s = NULL; else { @@ -5018,6 +5140,11 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (parser->m_prologState.level >= parser->m_groupSize) { if (parser->m_groupSize) { { + /* Detect and prevent integer overflow */ + if (parser->m_groupSize > (unsigned int)(-1) / 2u) { + return XML_ERROR_NO_MEMORY; + } + char *const new_connector = (char *)REALLOC( parser, parser->m_groupConnector, parser->m_groupSize *= 2); if (new_connector == NULL) { @@ -5028,6 +5155,16 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, } if (dtd->scaffIndex) { + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (parser->m_groupSize > (size_t)(-1) / sizeof(int)) { + return XML_ERROR_NO_MEMORY; + } +#endif + int *const new_scaff_index = (int *)REALLOC( parser, dtd->scaffIndex, parser->m_groupSize * sizeof(int)); if (new_scaff_index == NULL) @@ -5236,7 +5373,7 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (dtd->in_eldecl) { ELEMENT_TYPE *el; const XML_Char *name; - int nameLen; + size_t nameLen; const char *nxt = (quant == XML_CQUANT_NONE ? next : next - enc->minBytesPerChar); int myindex = nextScaffoldPart(parser); @@ -5252,7 +5389,13 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, nameLen = 0; for (; name[nameLen++];) ; - dtd->contentStringLen += nameLen; + + /* Detect and prevent integer overflow */ + if (nameLen > UINT_MAX - dtd->contentStringLen) { + return XML_ERROR_NO_MEMORY; + } + + dtd->contentStringLen += (unsigned)nameLen; if (parser->m_elementDeclHandler) handleDefault = XML_FALSE; } @@ -6098,7 +6241,24 @@ defineAttribute(ELEMENT_TYPE *type, ATTRIBUTE_ID *attId, XML_Bool isCdata, } } else { DEFAULT_ATTRIBUTE *temp; + + /* Detect and prevent integer overflow */ + if (type->allocDefaultAtts > INT_MAX / 2) { + return 0; + } + int count = type->allocDefaultAtts * 2; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)count > (size_t)(-1) / sizeof(DEFAULT_ATTRIBUTE)) { + return 0; + } +#endif + temp = (DEFAULT_ATTRIBUTE *)REALLOC(parser, type->defaultAtts, (count * sizeof(DEFAULT_ATTRIBUTE))); if (temp == NULL) @@ -6388,7 +6548,7 @@ normalizePublicId(XML_Char *publicId) { static DTD * dtdCreate(const XML_Memory_Handling_Suite *ms) { - DTD *p = (DTD *)ms->malloc_fcn(sizeof(DTD)); + DTD *p = ms->malloc_fcn(sizeof(DTD)); if (p == NULL) return p; poolInit(&(p->pool), ms); @@ -6561,8 +6721,8 @@ dtdCopy(XML_Parser oldParser, DTD *newDtd, const DTD *oldDtd, if (! newE) return 0; if (oldE->nDefaultAtts) { - newE->defaultAtts = (DEFAULT_ATTRIBUTE *)ms->malloc_fcn( - oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); + newE->defaultAtts + = ms->malloc_fcn(oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); if (! newE->defaultAtts) { return 0; } @@ -6724,7 +6884,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* table->size is a power of 2 */ table->size = (size_t)1 << INIT_POWER; tsize = table->size * sizeof(NAMED *); - table->v = (NAMED **)table->mem->malloc_fcn(tsize); + table->v = table->mem->malloc_fcn(tsize); if (! table->v) { table->size = 0; return NULL; @@ -6749,10 +6909,22 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* check for overflow (table is half full) */ if (table->used >> (table->power - 1)) { unsigned char newPower = table->power + 1; + + /* Detect and prevent invalid shift */ + if (newPower >= sizeof(unsigned long) * 8 /* bits per byte */) { + return NULL; + } + size_t newSize = (size_t)1 << newPower; unsigned long newMask = (unsigned long)newSize - 1; + + /* Detect and prevent integer overflow */ + if (newSize > (size_t)(-1) / sizeof(NAMED *)) { + return NULL; + } + size_t tsize = newSize * sizeof(NAMED *); - NAMED **newV = (NAMED **)table->mem->malloc_fcn(tsize); + NAMED **newV = table->mem->malloc_fcn(tsize); if (! newV) return NULL; memset(newV, 0, tsize); @@ -6781,7 +6953,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { } } } - table->v[i] = (NAMED *)table->mem->malloc_fcn(createSize); + table->v[i] = table->mem->malloc_fcn(createSize); if (! table->v[i]) return NULL; memset(table->v[i], 0, createSize); @@ -7069,7 +7241,7 @@ poolGrow(STRING_POOL *pool) { if (bytesToAllocate == 0) return XML_FALSE; - tem = (BLOCK *)pool->mem->malloc_fcn(bytesToAllocate); + tem = pool->mem->malloc_fcn(bytesToAllocate); if (! tem) return XML_FALSE; tem->size = blockSize; @@ -7100,6 +7272,20 @@ nextScaffoldPart(XML_Parser parser) { if (dtd->scaffCount >= dtd->scaffSize) { CONTENT_SCAFFOLD *temp; if (dtd->scaffold) { + /* Detect and prevent integer overflow */ + if (dtd->scaffSize > UINT_MAX / 2u) { + return -1; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffSize > (size_t)(-1) / 2u / sizeof(CONTENT_SCAFFOLD)) { + return -1; + } +#endif + temp = (CONTENT_SCAFFOLD *)REALLOC( parser, dtd->scaffold, dtd->scaffSize * 2 * sizeof(CONTENT_SCAFFOLD)); if (temp == NULL) @@ -7169,8 +7355,26 @@ build_model(XML_Parser parser) { XML_Content *ret; XML_Content *cpos; XML_Char *str; - int allocsize = (dtd->scaffCount * sizeof(XML_Content) - + (dtd->contentStringLen * sizeof(XML_Char))); + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffCount > (size_t)(-1) / sizeof(XML_Content)) { + return NULL; + } + if (dtd->contentStringLen > (size_t)(-1) / sizeof(XML_Char)) { + return NULL; + } +#endif + if (dtd->scaffCount * sizeof(XML_Content) + > (size_t)(-1) - dtd->contentStringLen * sizeof(XML_Char)) { + return NULL; + } + + const size_t allocsize = (dtd->scaffCount * sizeof(XML_Content) + + (dtd->contentStringLen * sizeof(XML_Char))); ret = (XML_Content *)MALLOC(parser, allocsize); if (! ret) diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 08173b0fd541d..77746ee42d10a 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -15,6 +15,7 @@ Copyright (c) 2016-2021 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -37,14 +38,14 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#include + #include #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "expat_external.h" #include "internal.h" #include "xmlrole.h" diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index 5b935718743ad..502ca1adc33b9 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -20,6 +20,7 @@ Copyright (c) 2017 Benbuck Nason Copyright (c) 2017 Jos? Guti?rrez de la Concha Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -42,16 +43,16 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ -#ifdef _WIN32 -# include "winconfig.h" -#endif - #include #include #include /* memcpy */ #include +#ifdef _WIN32 +# include "winconfig.h" +#endif + #include "expat_external.h" #include "internal.h" #include "xmltok.h" diff --git a/Modules/expat/xmltok_ns.c b/Modules/expat/xmltok_ns.c index 5fd8392235940..fbdd3e3c7b799 100644 --- a/Modules/expat/xmltok_ns.c +++ b/Modules/expat/xmltok_ns.c @@ -11,7 +11,7 @@ Copyright (c) 2002 Greg Stein Copyright (c) 2002 Fred L. Drake, Jr. Copyright (c) 2002-2006 Karl Waclawek - Copyright (c) 2017 Sebastian Pipping + Copyright (c) 2017-2021 Sebastian Pipping Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -93,7 +93,7 @@ NS(XmlInitEncoding)(INIT_ENCODING *p, const ENCODING **encPtr, static const ENCODING * NS(findEncoding)(const ENCODING *enc, const char *ptr, const char *end) { # define ENCODING_MAX 128 - char buf[ENCODING_MAX]; + char buf[ENCODING_MAX] = ""; char *p = buf; int i; XmlUtf8Convert(enc, &ptr, end, &p, p + ENCODING_MAX - 1); From webhook-mailer at python.org Sat Feb 12 10:36:01 2022 From: webhook-mailer at python.org (gvanrossum) Date: Sat, 12 Feb 2022 15:36:01 -0000 Subject: [Python-checkins] bpo-46333: include `module` in `ForwardRef.__repr__` (#31283) Message-ID: https://github.com/python/cpython/commit/b70690bb37cc4bac695051484734eede0c1f9ada commit: b70690bb37cc4bac695051484734eede0c1f9ada branch: main author: aha79 <34090357+aha79 at users.noreply.github.com> committer: gvanrossum date: 2022-02-12T07:35:57-08:00 summary: bpo-46333: include `module` in `ForwardRef.__repr__` (#31283) The module parameter carries semantic information about the forward ref. Show to the user that forward refs with same argument but different module are different. Co-authored-by: Andreas Hangauer Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-02-11-20-01-49.bpo-46333.PMTBY9.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 6e2a2b1978ab4..2bb5d61068a78 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -2862,6 +2862,8 @@ def fun(x: a): def test_forward_repr(self): self.assertEqual(repr(List['int']), "typing.List[ForwardRef('int')]") + self.assertEqual(repr(List[ForwardRef('int', module='mod')]), + "typing.List[ForwardRef('int', module='mod')]") def test_union_forward(self): diff --git a/Lib/typing.py b/Lib/typing.py index 1de48cca00d84..4a8bdf8132861 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -779,7 +779,11 @@ def __ror__(self, other): return Union[other, self] def __repr__(self): - return f'ForwardRef({self.__forward_arg__!r})' + if self.__forward_module__ is None: + module_repr = '' + else: + module_repr = f', module={self.__forward_module__!r}' + return f'ForwardRef({self.__forward_arg__!r}{module_repr})' class _TypeVarLike: """Mixin for TypeVar-like types (TypeVar and ParamSpec).""" diff --git a/Misc/NEWS.d/next/Library/2022-02-11-20-01-49.bpo-46333.PMTBY9.rst b/Misc/NEWS.d/next/Library/2022-02-11-20-01-49.bpo-46333.PMTBY9.rst new file mode 100644 index 0000000000000..669217e25f789 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-11-20-01-49.bpo-46333.PMTBY9.rst @@ -0,0 +1,3 @@ +The :meth:`__repr__` method of :class:`typing.ForwardRef` now +includes the ``module`` parameter of :class:`typing.ForwardRef` +when it is set. From webhook-mailer at python.org Sat Feb 12 19:04:57 2022 From: webhook-mailer at python.org (terryjreedy) Date: Sun, 13 Feb 2022 00:04:57 -0000 Subject: [Python-checkins] bpo-45447: Add syntax highlighting for `.pyi` files in IDLE (GH-28950) Message-ID: https://github.com/python/cpython/commit/50cf4991c49e19f917305dd7b9c71085c11edddb commit: 50cf4991c49e19f917305dd7b9c71085c11edddb branch: main author: Alex Waygood committer: terryjreedy date: 2022-02-12T19:04:48-05:00 summary: bpo-45447: Add syntax highlighting for `.pyi` files in IDLE (GH-28950) Also add .pyi to the python extensions in the "File-open" and "File-save" dialogues. Add util.py to contain objects that are used in multiple idlelib modules and have no dependencies on any of them. Co-authored-by: E-Paine <63801254+E-Paine at users.noreply.github.com> Co-authored-by: Terry Jan Reedy files: A Lib/idlelib/idle_test/example_noext A Lib/idlelib/idle_test/example_stub.pyi A Lib/idlelib/idle_test/test_util.py A Lib/idlelib/util.py A Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst M Doc/whatsnew/3.11.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/README.txt M Lib/idlelib/editor.py M Lib/idlelib/idle_test/test_iomenu.py M Lib/idlelib/iomenu.py diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 5e7c89e68426e..43a112a0ca112 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -228,6 +228,12 @@ fractions (Contributed by Mark Dickinson in :issue:`44547`.) +IDLE and idlelib +---------------- + +* IDLE now applies syntax highlighting to `.pyi` files. (Contributed by Alex + Waygood and Terry Jan Reedy in :issue:`45447`.) + inspect ------- * Add :func:`inspect.getmembers_static`: return all members without diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 0bfadfd81e2dd..441ec41ed76b2 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,6 +4,9 @@ Released on 2022-10-03 ========================= +bpo-28950: Apply IDLE syntax highlighting to `.pyi` files. Add util.py +for common components. Patch by Alex Waygood and Terry Jan Reedy. + bpo-46630: Make query dialogs on Windows start with a cursor in the entry box. diff --git a/Lib/idlelib/README.txt b/Lib/idlelib/README.txt index bc3d978f43f1a..8870fda315e39 100644 --- a/Lib/idlelib/README.txt +++ b/Lib/idlelib/README.txt @@ -82,6 +82,7 @@ tabbedpages.py # Define tabbed pages widget (nim). textview.py # Define read-only text widget (nim). tree.py # Define tree widget, used in browsers (nim). undo.py # Manage undo stack. +util.py # Define objects imported elsewhere with no dependencies (nim) windows.py # Manage window list and define listed top level. zoomheight.py # Zoom window to full height of screen. diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py index fcc8a3f08ccfe..6c52efd655e12 100644 --- a/Lib/idlelib/editor.py +++ b/Lib/idlelib/editor.py @@ -27,6 +27,7 @@ from idlelib import replace from idlelib import search from idlelib.tree import wheel_event +from idlelib.util import py_extensions from idlelib import window # The default tab setting for a Text widget, in average-width characters. @@ -757,7 +758,7 @@ def ispythonsource(self, filename): if not filename or os.path.isdir(filename): return True base, ext = os.path.splitext(os.path.basename(filename)) - if os.path.normcase(ext) in (".py", ".pyw"): + if os.path.normcase(ext) in py_extensions: return True line = self.text.get('1.0', '1.0 lineend') return line.startswith('#!') and 'python' in line diff --git a/Lib/idlelib/idle_test/example_noext b/Lib/idlelib/idle_test/example_noext new file mode 100644 index 0000000000000..7d2510e30bc0a --- /dev/null +++ b/Lib/idlelib/idle_test/example_noext @@ -0,0 +1,4 @@ +#!usr/bin/env python + +def example_function(some_argument): + pass diff --git a/Lib/idlelib/idle_test/example_stub.pyi b/Lib/idlelib/idle_test/example_stub.pyi new file mode 100644 index 0000000000000..a9811a78d10a6 --- /dev/null +++ b/Lib/idlelib/idle_test/example_stub.pyi @@ -0,0 +1,2 @@ +class Example: + def method(self, argument1: str, argument2: list[int]) -> None: ... diff --git a/Lib/idlelib/idle_test/test_iomenu.py b/Lib/idlelib/idle_test/test_iomenu.py index 99f4048796712..e338893c09e6a 100644 --- a/Lib/idlelib/idle_test/test_iomenu.py +++ b/Lib/idlelib/idle_test/test_iomenu.py @@ -1,6 +1,6 @@ "Test , coverage 17%." -from idlelib import iomenu +from idlelib import iomenu, util import unittest from test.support import requires from tkinter import Tk @@ -45,5 +45,27 @@ def test_fixnewlines_end(self): eq(fix(), 'a'+io.eol_convention) +def _extension_in_filetypes(extension): + return any( + f'*{extension}' in filetype_tuple[1] + for filetype_tuple in iomenu.IOBinding.filetypes + ) + + +class FiletypesTest(unittest.TestCase): + def test_python_source_files(self): + for extension in util.py_extensions: + with self.subTest(extension=extension): + self.assertTrue( + _extension_in_filetypes(extension) + ) + + def test_text_files(self): + self.assertTrue(_extension_in_filetypes('.txt')) + + def test_all_files(self): + self.assertTrue(_extension_in_filetypes('')) + + if __name__ == '__main__': unittest.main(verbosity=2) diff --git a/Lib/idlelib/idle_test/test_util.py b/Lib/idlelib/idle_test/test_util.py new file mode 100644 index 0000000000000..20721fe980c78 --- /dev/null +++ b/Lib/idlelib/idle_test/test_util.py @@ -0,0 +1,14 @@ +"""Test util, coverage 100%""" + +import unittest +from idlelib import util + + +class UtilTest(unittest.TestCase): + def test_extensions(self): + for extension in {'.pyi', '.py', '.pyw'}: + self.assertIn(extension, util.py_extensions) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/Lib/idlelib/iomenu.py b/Lib/idlelib/iomenu.py index 5ebf7089fb9ab..ad3109df84096 100644 --- a/Lib/idlelib/iomenu.py +++ b/Lib/idlelib/iomenu.py @@ -11,6 +11,9 @@ import idlelib from idlelib.config import idleConf +from idlelib.util import py_extensions + +py_extensions = ' '.join("*"+ext for ext in py_extensions) encoding = 'utf-8' if sys.platform == 'win32': @@ -348,7 +351,7 @@ def print_window(self, event): savedialog = None filetypes = ( - ("Python files", "*.py *.pyw", "TEXT"), + ("Python files", py_extensions, "TEXT"), ("Text files", "*.txt", "TEXT"), ("All files", "*"), ) diff --git a/Lib/idlelib/util.py b/Lib/idlelib/util.py new file mode 100644 index 0000000000000..5480219786bca --- /dev/null +++ b/Lib/idlelib/util.py @@ -0,0 +1,22 @@ +""" +Idlelib objects with no external idlelib dependencies +which are needed in more than one idlelib module. + +They are included here because + a) they don't particularly belong elsewhere; or + b) because inclusion here simplifies the idlelib dependency graph. + +TODO: + * Python versions (editor and help_about), + * tk version and patchlevel (pyshell, help_about, maxos?, editor?), + * std streams (pyshell, run), + * warning stuff (pyshell, run). +""" +from os import path + +# .pyw is for Windows; .pyi is for stub files. +py_extensions = ('.py', '.pyw', '.pyi') # Order needed for open/save dialogs. + +if __name__ == '__main__': + from unittest import main + main('idlelib.idle_test.test_util', verbosity=2) diff --git a/Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst b/Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst new file mode 100644 index 0000000000000..2b5170c7631d2 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst @@ -0,0 +1,2 @@ +Apply IDLE syntax highlighting to `.pyi` files. Patch by Alex Waygood +and Terry Jan Reedy. From webhook-mailer at python.org Sat Feb 12 19:16:40 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 13 Feb 2022 00:16:40 -0000 Subject: [Python-checkins] [3.9] bpo-45948: Remove constructor discrepancy in C version of ElementTree.XMLParser (GH-31152) (GH-31299) Message-ID: https://github.com/python/cpython/commit/cc6d8f88289917d67237a10f7a0e2439fde7a573 commit: cc6d8f88289917d67237a10f7a0e2439fde7a573 branch: 3.9 author: Jacob Walls committer: serhiy-storchaka date: 2022-02-13T02:16:31+02:00 summary: [3.9] bpo-45948: Remove constructor discrepancy in C version of ElementTree.XMLParser (GH-31152) (GH-31299) Both implementations accept target=None now. (cherry picked from commit 168fd6453b5de15236116f9261d64601d92571ac) files: A Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst M Lib/test/test_xml_etree.py M Modules/_elementtree.c M Modules/clinic/_elementtree.c.h diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index 23c4cd5eff0a9..8cdf451fbdbac 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -740,6 +740,15 @@ def end_ns(self, prefix): ('end-ns', ''), ]) + def test_initialize_parser_without_target(self): + # Explicit None + parser = ET.XMLParser(target=None) + self.assertIsInstance(parser.target, ET.TreeBuilder) + + # Implicit None + parser2 = ET.XMLParser() + self.assertIsInstance(parser2.target, ET.TreeBuilder) + def test_children(self): # Test Element children iteration diff --git a/Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst b/Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst new file mode 100644 index 0000000000000..42dc114b5ad60 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-05-18-22-05.bpo-45948.w4mCnE.rst @@ -0,0 +1,5 @@ +Fixed a discrepancy in the C implementation of the +:mod:`xml.etree.ElementTree` module. Now, instantiating an +:class:`xml.etree.ElementTree.XMLParser` with a ``target=None`` +keyword provides a default :class:`xml.etree.ElementTree.TreeBuilder` +target as the Python implementation does. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index d9c2a23ccb858..8d1b56b35d204 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3636,7 +3636,7 @@ ignore_attribute_error(PyObject *value) _elementtree.XMLParser.__init__ * - target: object = NULL + target: object = None encoding: str(accept={str, NoneType}) = None [clinic start generated code]*/ @@ -3644,7 +3644,7 @@ _elementtree.XMLParser.__init__ static int _elementtree_XMLParser___init___impl(XMLParserObject *self, PyObject *target, const char *encoding) -/*[clinic end generated code: output=3ae45ec6cdf344e4 input=53e35a829ae043e8]*/ +/*[clinic end generated code: output=3ae45ec6cdf344e4 input=7e716dd6e4f3e439]*/ { self->entity = PyDict_New(); if (!self->entity) @@ -3669,7 +3669,7 @@ _elementtree_XMLParser___init___impl(XMLParserObject *self, PyObject *target, (unsigned long)_Py_HashSecret.expat.hashsalt); } - if (target) { + if (target != Py_None) { Py_INCREF(target); } else { target = treebuilder_new(&TreeBuilder_Type, NULL, NULL); diff --git a/Modules/clinic/_elementtree.c.h b/Modules/clinic/_elementtree.c.h index 825416f4a3982..034c1081dccdf 100644 --- a/Modules/clinic/_elementtree.c.h +++ b/Modules/clinic/_elementtree.c.h @@ -812,7 +812,7 @@ _elementtree_XMLParser___init__(PyObject *self, PyObject *args, PyObject *kwargs PyObject * const *fastargs; Py_ssize_t nargs = PyTuple_GET_SIZE(args); Py_ssize_t noptargs = nargs + (kwargs ? PyDict_GET_SIZE(kwargs) : 0) - 0; - PyObject *target = NULL; + PyObject *target = Py_None; const char *encoding = NULL; fastargs = _PyArg_UnpackKeywords(_PyTuple_CAST(args)->ob_item, nargs, kwargs, NULL, &_parser, 0, 0, 0, argsbuf); @@ -920,4 +920,4 @@ _elementtree_XMLParser__setevents(XMLParserObject *self, PyObject *const *args, exit: return return_value; } -/*[clinic end generated code: output=b7f6a32462fc42a9 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=18ddc3c84139443d input=a9049054013a1b77]*/ From webhook-mailer at python.org Sat Feb 12 19:52:46 2022 From: webhook-mailer at python.org (terryjreedy) Date: Sun, 13 Feb 2022 00:52:46 -0000 Subject: [Python-checkins] bpo-45447: Add syntax highlighting for `.pyi` files in IDLE (GH-28950) Message-ID: https://github.com/python/cpython/commit/9fabcfbe68ff81ef5f17f86a93daf9cce9d83876 commit: 9fabcfbe68ff81ef5f17f86a93daf9cce9d83876 branch: 3.10 author: Terry Jan Reedy committer: terryjreedy date: 2022-02-12T19:52:37-05:00 summary: bpo-45447: Add syntax highlighting for `.pyi` files in IDLE (GH-28950) Also add .pyi to the python extensions in the "File-open" and "File-save" dialogues. Add util.py to contain objects that are used in multiple idlelib modules and have no dependencies on any of them. Co-authored-by: E-Paine <63801254+E-Paine at users.noreply.github.com> Co-authored-by: Terry Jan Reedy (cherry picked from commit 50cf4991c49e19f917305dd7b9c71085c11edddb) Co-authored-by: Alex Waygood files: A Lib/idlelib/idle_test/example_noext A Lib/idlelib/idle_test/example_stub.pyi A Lib/idlelib/idle_test/test_util.py A Lib/idlelib/util.py A Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/README.txt M Lib/idlelib/editor.py M Lib/idlelib/idle_test/test_iomenu.py M Lib/idlelib/iomenu.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index e8055a054640b..424e971c8bb31 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,6 +4,9 @@ Released on 2022-05-16 ========================= +bpo-28950: Apply IDLE syntax highlighting to `.pyi` files. Add util.py +for common components. Patch by Alex Waygood and Terry Jan Reedy. + bpo-46630: Make query dialogs on Windows start with a cursor in the entry box. diff --git a/Lib/idlelib/README.txt b/Lib/idlelib/README.txt index bc3d978f43f1a..8870fda315e39 100644 --- a/Lib/idlelib/README.txt +++ b/Lib/idlelib/README.txt @@ -82,6 +82,7 @@ tabbedpages.py # Define tabbed pages widget (nim). textview.py # Define read-only text widget (nim). tree.py # Define tree widget, used in browsers (nim). undo.py # Manage undo stack. +util.py # Define objects imported elsewhere with no dependencies (nim) windows.py # Manage window list and define listed top level. zoomheight.py # Zoom window to full height of screen. diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py index fcc8a3f08ccfe..6c52efd655e12 100644 --- a/Lib/idlelib/editor.py +++ b/Lib/idlelib/editor.py @@ -27,6 +27,7 @@ from idlelib import replace from idlelib import search from idlelib.tree import wheel_event +from idlelib.util import py_extensions from idlelib import window # The default tab setting for a Text widget, in average-width characters. @@ -757,7 +758,7 @@ def ispythonsource(self, filename): if not filename or os.path.isdir(filename): return True base, ext = os.path.splitext(os.path.basename(filename)) - if os.path.normcase(ext) in (".py", ".pyw"): + if os.path.normcase(ext) in py_extensions: return True line = self.text.get('1.0', '1.0 lineend') return line.startswith('#!') and 'python' in line diff --git a/Lib/idlelib/idle_test/example_noext b/Lib/idlelib/idle_test/example_noext new file mode 100644 index 0000000000000..7d2510e30bc0a --- /dev/null +++ b/Lib/idlelib/idle_test/example_noext @@ -0,0 +1,4 @@ +#!usr/bin/env python + +def example_function(some_argument): + pass diff --git a/Lib/idlelib/idle_test/example_stub.pyi b/Lib/idlelib/idle_test/example_stub.pyi new file mode 100644 index 0000000000000..a9811a78d10a6 --- /dev/null +++ b/Lib/idlelib/idle_test/example_stub.pyi @@ -0,0 +1,2 @@ +class Example: + def method(self, argument1: str, argument2: list[int]) -> None: ... diff --git a/Lib/idlelib/idle_test/test_iomenu.py b/Lib/idlelib/idle_test/test_iomenu.py index 99f4048796712..e338893c09e6a 100644 --- a/Lib/idlelib/idle_test/test_iomenu.py +++ b/Lib/idlelib/idle_test/test_iomenu.py @@ -1,6 +1,6 @@ "Test , coverage 17%." -from idlelib import iomenu +from idlelib import iomenu, util import unittest from test.support import requires from tkinter import Tk @@ -45,5 +45,27 @@ def test_fixnewlines_end(self): eq(fix(), 'a'+io.eol_convention) +def _extension_in_filetypes(extension): + return any( + f'*{extension}' in filetype_tuple[1] + for filetype_tuple in iomenu.IOBinding.filetypes + ) + + +class FiletypesTest(unittest.TestCase): + def test_python_source_files(self): + for extension in util.py_extensions: + with self.subTest(extension=extension): + self.assertTrue( + _extension_in_filetypes(extension) + ) + + def test_text_files(self): + self.assertTrue(_extension_in_filetypes('.txt')) + + def test_all_files(self): + self.assertTrue(_extension_in_filetypes('')) + + if __name__ == '__main__': unittest.main(verbosity=2) diff --git a/Lib/idlelib/idle_test/test_util.py b/Lib/idlelib/idle_test/test_util.py new file mode 100644 index 0000000000000..20721fe980c78 --- /dev/null +++ b/Lib/idlelib/idle_test/test_util.py @@ -0,0 +1,14 @@ +"""Test util, coverage 100%""" + +import unittest +from idlelib import util + + +class UtilTest(unittest.TestCase): + def test_extensions(self): + for extension in {'.pyi', '.py', '.pyw'}: + self.assertIn(extension, util.py_extensions) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/Lib/idlelib/iomenu.py b/Lib/idlelib/iomenu.py index 5ebf7089fb9ab..ad3109df84096 100644 --- a/Lib/idlelib/iomenu.py +++ b/Lib/idlelib/iomenu.py @@ -11,6 +11,9 @@ import idlelib from idlelib.config import idleConf +from idlelib.util import py_extensions + +py_extensions = ' '.join("*"+ext for ext in py_extensions) encoding = 'utf-8' if sys.platform == 'win32': @@ -348,7 +351,7 @@ def print_window(self, event): savedialog = None filetypes = ( - ("Python files", "*.py *.pyw", "TEXT"), + ("Python files", py_extensions, "TEXT"), ("Text files", "*.txt", "TEXT"), ("All files", "*"), ) diff --git a/Lib/idlelib/util.py b/Lib/idlelib/util.py new file mode 100644 index 0000000000000..5480219786bca --- /dev/null +++ b/Lib/idlelib/util.py @@ -0,0 +1,22 @@ +""" +Idlelib objects with no external idlelib dependencies +which are needed in more than one idlelib module. + +They are included here because + a) they don't particularly belong elsewhere; or + b) because inclusion here simplifies the idlelib dependency graph. + +TODO: + * Python versions (editor and help_about), + * tk version and patchlevel (pyshell, help_about, maxos?, editor?), + * std streams (pyshell, run), + * warning stuff (pyshell, run). +""" +from os import path + +# .pyw is for Windows; .pyi is for stub files. +py_extensions = ('.py', '.pyw', '.pyi') # Order needed for open/save dialogs. + +if __name__ == '__main__': + from unittest import main + main('idlelib.idle_test.test_util', verbosity=2) diff --git a/Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst b/Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst new file mode 100644 index 0000000000000..2b5170c7631d2 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst @@ -0,0 +1,2 @@ +Apply IDLE syntax highlighting to `.pyi` files. Patch by Alex Waygood +and Terry Jan Reedy. From webhook-mailer at python.org Sat Feb 12 20:19:33 2022 From: webhook-mailer at python.org (miss-islington) Date: Sun, 13 Feb 2022 01:19:33 -0000 Subject: [Python-checkins] bpo-45447: Add syntax highlighting for `.pyi` files in IDLE (GH-28950) Message-ID: https://github.com/python/cpython/commit/1d4d44c385414042cbdd2a1e9504cd8e9d69c0ae commit: 1d4d44c385414042cbdd2a1e9504cd8e9d69c0ae branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-12T17:19:25-08:00 summary: bpo-45447: Add syntax highlighting for `.pyi` files in IDLE (GH-28950) Also add .pyi to the python extensions in the "File-open" and "File-save" dialogues. Add util.py to contain objects that are used in multiple idlelib modules and have no dependencies on any of them. Co-authored-by: E-Paine <63801254+E-Paine at users.noreply.github.com> Co-authored-by: Terry Jan Reedy (cherry picked from commit 50cf4991c49e19f917305dd7b9c71085c11edddb) Co-authored-by: Alex Waygood (cherry picked from commit 9fabcfbe68ff81ef5f17f86a93daf9cce9d83876) Co-authored-by: Terry Jan Reedy files: A Lib/idlelib/idle_test/example_noext A Lib/idlelib/idle_test/example_stub.pyi A Lib/idlelib/idle_test/test_util.py A Lib/idlelib/util.py A Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/README.txt M Lib/idlelib/editor.py M Lib/idlelib/idle_test/test_iomenu.py M Lib/idlelib/iomenu.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 73c8b7a32d767..5605c564f9032 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -4,6 +4,9 @@ Released on 2022-05-16 ========================= +bpo-28950: Apply IDLE syntax highlighting to `.pyi` files. Add util.py +for common components. Patch by Alex Waygood and Terry Jan Reedy. + bpo-46630: Make query dialogs on Windows start with a cursor in the entry box. diff --git a/Lib/idlelib/README.txt b/Lib/idlelib/README.txt index bc3d978f43f1a..8870fda315e39 100644 --- a/Lib/idlelib/README.txt +++ b/Lib/idlelib/README.txt @@ -82,6 +82,7 @@ tabbedpages.py # Define tabbed pages widget (nim). textview.py # Define read-only text widget (nim). tree.py # Define tree widget, used in browsers (nim). undo.py # Manage undo stack. +util.py # Define objects imported elsewhere with no dependencies (nim) windows.py # Manage window list and define listed top level. zoomheight.py # Zoom window to full height of screen. diff --git a/Lib/idlelib/editor.py b/Lib/idlelib/editor.py index b9cb50264ff06..3bd0a0cd39c8e 100644 --- a/Lib/idlelib/editor.py +++ b/Lib/idlelib/editor.py @@ -27,6 +27,7 @@ from idlelib import replace from idlelib import search from idlelib.tree import wheel_event +from idlelib.util import py_extensions from idlelib import window # The default tab setting for a Text widget, in average-width characters. @@ -757,7 +758,7 @@ def ispythonsource(self, filename): if not filename or os.path.isdir(filename): return True base, ext = os.path.splitext(os.path.basename(filename)) - if os.path.normcase(ext) in (".py", ".pyw"): + if os.path.normcase(ext) in py_extensions: return True line = self.text.get('1.0', '1.0 lineend') return line.startswith('#!') and 'python' in line diff --git a/Lib/idlelib/idle_test/example_noext b/Lib/idlelib/idle_test/example_noext new file mode 100644 index 0000000000000..7d2510e30bc0a --- /dev/null +++ b/Lib/idlelib/idle_test/example_noext @@ -0,0 +1,4 @@ +#!usr/bin/env python + +def example_function(some_argument): + pass diff --git a/Lib/idlelib/idle_test/example_stub.pyi b/Lib/idlelib/idle_test/example_stub.pyi new file mode 100644 index 0000000000000..a9811a78d10a6 --- /dev/null +++ b/Lib/idlelib/idle_test/example_stub.pyi @@ -0,0 +1,2 @@ +class Example: + def method(self, argument1: str, argument2: list[int]) -> None: ... diff --git a/Lib/idlelib/idle_test/test_iomenu.py b/Lib/idlelib/idle_test/test_iomenu.py index 99f4048796712..e338893c09e6a 100644 --- a/Lib/idlelib/idle_test/test_iomenu.py +++ b/Lib/idlelib/idle_test/test_iomenu.py @@ -1,6 +1,6 @@ "Test , coverage 17%." -from idlelib import iomenu +from idlelib import iomenu, util import unittest from test.support import requires from tkinter import Tk @@ -45,5 +45,27 @@ def test_fixnewlines_end(self): eq(fix(), 'a'+io.eol_convention) +def _extension_in_filetypes(extension): + return any( + f'*{extension}' in filetype_tuple[1] + for filetype_tuple in iomenu.IOBinding.filetypes + ) + + +class FiletypesTest(unittest.TestCase): + def test_python_source_files(self): + for extension in util.py_extensions: + with self.subTest(extension=extension): + self.assertTrue( + _extension_in_filetypes(extension) + ) + + def test_text_files(self): + self.assertTrue(_extension_in_filetypes('.txt')) + + def test_all_files(self): + self.assertTrue(_extension_in_filetypes('')) + + if __name__ == '__main__': unittest.main(verbosity=2) diff --git a/Lib/idlelib/idle_test/test_util.py b/Lib/idlelib/idle_test/test_util.py new file mode 100644 index 0000000000000..20721fe980c78 --- /dev/null +++ b/Lib/idlelib/idle_test/test_util.py @@ -0,0 +1,14 @@ +"""Test util, coverage 100%""" + +import unittest +from idlelib import util + + +class UtilTest(unittest.TestCase): + def test_extensions(self): + for extension in {'.pyi', '.py', '.pyw'}: + self.assertIn(extension, util.py_extensions) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/Lib/idlelib/iomenu.py b/Lib/idlelib/iomenu.py index 5ebf7089fb9ab..ad3109df84096 100644 --- a/Lib/idlelib/iomenu.py +++ b/Lib/idlelib/iomenu.py @@ -11,6 +11,9 @@ import idlelib from idlelib.config import idleConf +from idlelib.util import py_extensions + +py_extensions = ' '.join("*"+ext for ext in py_extensions) encoding = 'utf-8' if sys.platform == 'win32': @@ -348,7 +351,7 @@ def print_window(self, event): savedialog = None filetypes = ( - ("Python files", "*.py *.pyw", "TEXT"), + ("Python files", py_extensions, "TEXT"), ("Text files", "*.txt", "TEXT"), ("All files", "*"), ) diff --git a/Lib/idlelib/util.py b/Lib/idlelib/util.py new file mode 100644 index 0000000000000..5480219786bca --- /dev/null +++ b/Lib/idlelib/util.py @@ -0,0 +1,22 @@ +""" +Idlelib objects with no external idlelib dependencies +which are needed in more than one idlelib module. + +They are included here because + a) they don't particularly belong elsewhere; or + b) because inclusion here simplifies the idlelib dependency graph. + +TODO: + * Python versions (editor and help_about), + * tk version and patchlevel (pyshell, help_about, maxos?, editor?), + * std streams (pyshell, run), + * warning stuff (pyshell, run). +""" +from os import path + +# .pyw is for Windows; .pyi is for stub files. +py_extensions = ('.py', '.pyw', '.pyi') # Order needed for open/save dialogs. + +if __name__ == '__main__': + from unittest import main + main('idlelib.idle_test.test_util', verbosity=2) diff --git a/Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst b/Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst new file mode 100644 index 0000000000000..2b5170c7631d2 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2021-10-14-16-55-03.bpo-45447.FhiH5P.rst @@ -0,0 +1,2 @@ +Apply IDLE syntax highlighting to `.pyi` files. Patch by Alex Waygood +and Terry Jan Reedy. From webhook-mailer at python.org Sat Feb 12 21:10:50 2022 From: webhook-mailer at python.org (terryjreedy) Date: Sun, 13 Feb 2022 02:10:50 -0000 Subject: [Python-checkins] bpo-45447: Add entry to What's new 3.10 (GH-31304) Message-ID: https://github.com/python/cpython/commit/2d98433549be358d1c192e30e51b8d345d618cc7 commit: 2d98433549be358d1c192e30e51b8d345d618cc7 branch: main author: Terry Jan Reedy committer: terryjreedy date: 2022-02-12T21:10:29-05:00 summary: bpo-45447: Add entry to What's new 3.10 (GH-31304) files: M Doc/whatsnew/3.10.rst diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 73c22aafc764d..806170ede1596 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1159,7 +1159,7 @@ Reedy in :issue:`40468`.) Move the indent space setting from the Font tab to the new Windows tab. (Contributed by Mark Roseman and Terry Jan Reedy in :issue:`33962`.) -These changes were backported to a 3.9 maintenance release. +The changes above were backported to a 3.9 maintenance release. Add a Shell sidebar. Move the primary prompt ('>>>') to the sidebar. Add secondary prompts ('...') to the sidebar. Left click and optional @@ -1181,6 +1181,11 @@ pattern-matching statements. However, this highlighting is not perfect and will be incorrect in some rare cases, including some ``_``-s in ``case`` patterns. (Contributed by Tal Einat in :issue:`44010`.) +New in 3.10 maintenance releases. + +Apply syntax highlighting to `.pyi` files. (Contributed by Alex +Waygood and Terry Jan Reedy in :issue:`45447`.) + importlib.metadata ------------------ From webhook-mailer at python.org Sat Feb 12 21:11:24 2022 From: webhook-mailer at python.org (terryjreedy) Date: Sun, 13 Feb 2022 02:11:24 -0000 Subject: [Python-checkins] bpo-45447: Add entry to What's new 3.9 (GH-31305) Message-ID: https://github.com/python/cpython/commit/cef91ca80c41749824eca1d4b2c99731e3d5f64c commit: cef91ca80c41749824eca1d4b2c99731e3d5f64c branch: main author: Terry Jan Reedy committer: terryjreedy date: 2022-02-12T21:11:19-05:00 summary: bpo-45447: Add entry to What's new 3.9 (GH-31305) files: M Doc/whatsnew/3.9.rst diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 4fa3835a39900..c7255b3d41765 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -500,6 +500,9 @@ Reedy in :issue:`40468`.) Move the indent space setting from the Font tab to the new Windows tab. (Contributed by Mark Roseman and Terry Jan Reedy in :issue:`33962`.) +Apply syntax highlighting to `.pyi` files. (Contributed by Alex +Waygood and Terry Jan Reedy in :issue:`45447`.) + imaplib ------- From webhook-mailer at python.org Sat Feb 12 21:12:26 2022 From: webhook-mailer at python.org (terryjreedy) Date: Sun, 13 Feb 2022 02:12:26 -0000 Subject: [Python-checkins] bpo-45447: Fix entry in What's New 3.11 (GH-31307) Message-ID: https://github.com/python/cpython/commit/6331c08d1c3248ff47a7b8e0045c9023c9af672c commit: 6331c08d1c3248ff47a7b8e0045c9023c9af672c branch: main author: Terry Jan Reedy committer: terryjreedy date: 2022-02-12T21:12:21-05:00 summary: bpo-45447: Fix entry in What's New 3.11 (GH-31307) files: M Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 43a112a0ca112..510ef86d42cc7 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -231,7 +231,7 @@ fractions IDLE and idlelib ---------------- -* IDLE now applies syntax highlighting to `.pyi` files. (Contributed by Alex +* Apply syntax highlighting to `.pyi` files. (Contributed by Alex Waygood and Terry Jan Reedy in :issue:`45447`.) inspect From webhook-mailer at python.org Sun Feb 13 01:30:35 2022 From: webhook-mailer at python.org (miss-islington) Date: Sun, 13 Feb 2022 06:30:35 -0000 Subject: [Python-checkins] bpo-45447: Add entry to What's new 3.9 (GH-31305) Message-ID: https://github.com/python/cpython/commit/5698e0be8e77bd4d7a1fda88005662f0b239dec8 commit: 5698e0be8e77bd4d7a1fda88005662f0b239dec8 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-12T22:30:25-08:00 summary: bpo-45447: Add entry to What's new 3.9 (GH-31305) (cherry picked from commit cef91ca80c41749824eca1d4b2c99731e3d5f64c) Co-authored-by: Terry Jan Reedy files: M Doc/whatsnew/3.9.rst diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index a6f83a91f5e14..e2455bce7d368 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -500,6 +500,9 @@ Reedy in :issue:`40468`.) Move the indent space setting from the Font tab to the new Windows tab. (Contributed by Mark Roseman and Terry Jan Reedy in :issue:`33962`.) +Apply syntax highlighting to `.pyi` files. (Contributed by Alex +Waygood and Terry Jan Reedy in :issue:`45447`.) + imaplib ------- From webhook-mailer at python.org Sun Feb 13 01:30:40 2022 From: webhook-mailer at python.org (miss-islington) Date: Sun, 13 Feb 2022 06:30:40 -0000 Subject: [Python-checkins] bpo-45447: Add entry to What's new 3.9 (GH-31305) Message-ID: https://github.com/python/cpython/commit/ebe73e6095fc019fd1e31c9f41bc67d56ee003b6 commit: ebe73e6095fc019fd1e31c9f41bc67d56ee003b6 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-12T22:30:36-08:00 summary: bpo-45447: Add entry to What's new 3.9 (GH-31305) (cherry picked from commit cef91ca80c41749824eca1d4b2c99731e3d5f64c) Co-authored-by: Terry Jan Reedy files: M Doc/whatsnew/3.9.rst diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 044cfc07bce9b..6aed8d2b5f984 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -500,6 +500,9 @@ Reedy in :issue:`40468`.) Move the indent space setting from the Font tab to the new Windows tab. (Contributed by Mark Roseman and Terry Jan Reedy in :issue:`33962`.) +Apply syntax highlighting to `.pyi` files. (Contributed by Alex +Waygood and Terry Jan Reedy in :issue:`45447`.) + imaplib ------- From webhook-mailer at python.org Sun Feb 13 01:31:07 2022 From: webhook-mailer at python.org (miss-islington) Date: Sun, 13 Feb 2022 06:31:07 -0000 Subject: [Python-checkins] bpo-45447: Add entry to What's new 3.10 (GH-31304) Message-ID: https://github.com/python/cpython/commit/44666c3285a52c2a4632f89aeeddd7ca7d02c8df commit: 44666c3285a52c2a4632f89aeeddd7ca7d02c8df branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-12T22:31:03-08:00 summary: bpo-45447: Add entry to What's new 3.10 (GH-31304) (cherry picked from commit 2d98433549be358d1c192e30e51b8d345d618cc7) Co-authored-by: Terry Jan Reedy files: M Doc/whatsnew/3.10.rst diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 6edc09ff6cef6..5788e32da1c92 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1148,7 +1148,7 @@ Reedy in :issue:`40468`.) Move the indent space setting from the Font tab to the new Windows tab. (Contributed by Mark Roseman and Terry Jan Reedy in :issue:`33962`.) -These changes were backported to a 3.9 maintenance release. +The changes above were backported to a 3.9 maintenance release. Add a Shell sidebar. Move the primary prompt ('>>>') to the sidebar. Add secondary prompts ('...') to the sidebar. Left click and optional @@ -1170,6 +1170,11 @@ pattern-matching statements. However, this highlighting is not perfect and will be incorrect in some rare cases, including some ``_``-s in ``case`` patterns. (Contributed by Tal Einat in :issue:`44010`.) +New in 3.10 maintenance releases. + +Apply syntax highlighting to `.pyi` files. (Contributed by Alex +Waygood and Terry Jan Reedy in :issue:`45447`.) + importlib.metadata ------------------ From webhook-mailer at python.org Sun Feb 13 05:30:21 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 13 Feb 2022 10:30:21 -0000 Subject: [Python-checkins] bpo-46615: Don't crash when set operations mutate the sets (GH-31120) (GH-31312) Message-ID: https://github.com/python/cpython/commit/c31b8a97a8a7e8255231c9e12ed581c6240c0d6c commit: c31b8a97a8a7e8255231c9e12ed581c6240c0d6c branch: 3.9 author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: serhiy-storchaka date: 2022-02-13T12:29:42+02:00 summary: bpo-46615: Don't crash when set operations mutate the sets (GH-31120) (GH-31312) Ensure strong references are acquired whenever using `set_next()`. Added randomized test cases for `__eq__` methods that sometimes mutate sets when called. (cherry picked from commit 4a66615ba736f84eadf9456bfd5d32a94cccf117) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst M Lib/test/test_set.py M Objects/setobject.c diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py index ca2f4e28ae80b..ec7433c87d41c 100644 --- a/Lib/test/test_set.py +++ b/Lib/test/test_set.py @@ -1799,6 +1799,192 @@ def __eq__(self, o): s = {0} s.update(other) + +class TestOperationsMutating: + """Regression test for bpo-46615""" + + constructor1 = None + constructor2 = None + + def make_sets_of_bad_objects(self): + class Bad: + def __eq__(self, other): + if not enabled: + return False + if randrange(20) == 0: + set1.clear() + if randrange(20) == 0: + set2.clear() + return bool(randrange(2)) + def __hash__(self): + return randrange(2) + # Don't behave poorly during construction. + enabled = False + set1 = self.constructor1(Bad() for _ in range(randrange(50))) + set2 = self.constructor2(Bad() for _ in range(randrange(50))) + # Now start behaving poorly + enabled = True + return set1, set2 + + def check_set_op_does_not_crash(self, function): + for _ in range(100): + set1, set2 = self.make_sets_of_bad_objects() + try: + function(set1, set2) + except RuntimeError as e: + # Just make sure we don't crash here. + self.assertIn("changed size during iteration", str(e)) + + +class TestBinaryOpsMutating(TestOperationsMutating): + + def test_eq_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a == b) + + def test_ne_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a != b) + + def test_lt_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a < b) + + def test_le_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a <= b) + + def test_gt_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a > b) + + def test_ge_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a >= b) + + def test_and_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a & b) + + def test_or_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a | b) + + def test_sub_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a - b) + + def test_xor_with_mutation(self): + self.check_set_op_does_not_crash(lambda a, b: a ^ b) + + def test_iadd_with_mutation(self): + def f(a, b): + a &= b + self.check_set_op_does_not_crash(f) + + def test_ior_with_mutation(self): + def f(a, b): + a |= b + self.check_set_op_does_not_crash(f) + + def test_isub_with_mutation(self): + def f(a, b): + a -= b + self.check_set_op_does_not_crash(f) + + def test_ixor_with_mutation(self): + def f(a, b): + a ^= b + self.check_set_op_does_not_crash(f) + + def test_iteration_with_mutation(self): + def f1(a, b): + for x in a: + pass + for y in b: + pass + def f2(a, b): + for y in b: + pass + for x in a: + pass + def f3(a, b): + for x, y in zip(a, b): + pass + self.check_set_op_does_not_crash(f1) + self.check_set_op_does_not_crash(f2) + self.check_set_op_does_not_crash(f3) + + +class TestBinaryOpsMutating_Set_Set(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = set + constructor2 = set + +class TestBinaryOpsMutating_Subclass_Subclass(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = SetSubclass + +class TestBinaryOpsMutating_Set_Subclass(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = set + constructor2 = SetSubclass + +class TestBinaryOpsMutating_Subclass_Set(TestBinaryOpsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = set + + +class TestMethodsMutating(TestOperationsMutating): + + def test_issubset_with_mutation(self): + self.check_set_op_does_not_crash(set.issubset) + + def test_issuperset_with_mutation(self): + self.check_set_op_does_not_crash(set.issuperset) + + def test_intersection_with_mutation(self): + self.check_set_op_does_not_crash(set.intersection) + + def test_union_with_mutation(self): + self.check_set_op_does_not_crash(set.union) + + def test_difference_with_mutation(self): + self.check_set_op_does_not_crash(set.difference) + + def test_symmetric_difference_with_mutation(self): + self.check_set_op_does_not_crash(set.symmetric_difference) + + def test_isdisjoint_with_mutation(self): + self.check_set_op_does_not_crash(set.isdisjoint) + + def test_difference_update_with_mutation(self): + self.check_set_op_does_not_crash(set.difference_update) + + def test_intersection_update_with_mutation(self): + self.check_set_op_does_not_crash(set.intersection_update) + + def test_symmetric_difference_update_with_mutation(self): + self.check_set_op_does_not_crash(set.symmetric_difference_update) + + def test_update_with_mutation(self): + self.check_set_op_does_not_crash(set.update) + + +class TestMethodsMutating_Set_Set(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = set + +class TestMethodsMutating_Subclass_Subclass(TestMethodsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = SetSubclass + +class TestMethodsMutating_Set_Subclass(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = SetSubclass + +class TestMethodsMutating_Subclass_Set(TestMethodsMutating, unittest.TestCase): + constructor1 = SetSubclass + constructor2 = set + +class TestMethodsMutating_Set_Dict(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = dict.fromkeys + +class TestMethodsMutating_Set_List(TestMethodsMutating, unittest.TestCase): + constructor1 = set + constructor2 = list + + # Application tests (based on David Eppstein's graph recipes ==================================== def powerset(U): diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst new file mode 100644 index 0000000000000..6dee92a546e33 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-04-04-33-18.bpo-46615.puArY9.rst @@ -0,0 +1 @@ +When iterating over sets internally in ``setobject.c``, acquire strong references to the resulting items from the set. This prevents crashes in corner-cases of various set operations where the set gets mutated. diff --git a/Objects/setobject.c b/Objects/setobject.c index 4bd5777f967da..6d156bd4e0827 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -1207,17 +1207,21 @@ set_intersection(PySetObject *so, PyObject *other) while (set_next((PySetObject *)other, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = set_contains_entry(so, key, hash); if (rv < 0) { Py_DECREF(result); + Py_DECREF(key); return NULL; } if (rv) { if (set_add_entry(result, key, hash)) { Py_DECREF(result); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } return (PyObject *)result; } @@ -1357,11 +1361,16 @@ set_isdisjoint(PySetObject *so, PyObject *other) other = tmp; } while (set_next((PySetObject *)other, &pos, &entry)) { - rv = set_contains_entry(so, entry->key, entry->hash); - if (rv < 0) + PyObject *key = entry->key; + Py_INCREF(key); + rv = set_contains_entry(so, key, entry->hash); + Py_DECREF(key); + if (rv < 0) { return NULL; - if (rv) + } + if (rv) { Py_RETURN_FALSE; + } } Py_RETURN_TRUE; } @@ -1420,11 +1429,16 @@ set_difference_update_internal(PySetObject *so, PyObject *other) Py_INCREF(other); } - while (set_next((PySetObject *)other, &pos, &entry)) - if (set_discard_entry(so, entry->key, entry->hash) < 0) { + while (set_next((PySetObject *)other, &pos, &entry)) { + PyObject *key = entry->key; + Py_INCREF(key); + if (set_discard_entry(so, key, entry->hash) < 0) { Py_DECREF(other); + Py_DECREF(key); return -1; } + Py_DECREF(key); + } Py_DECREF(other); } else { @@ -1515,17 +1529,21 @@ set_difference(PySetObject *so, PyObject *other) while (set_next(so, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = _PyDict_Contains(other, key, hash); if (rv < 0) { Py_DECREF(result); + Py_DECREF(key); return NULL; } if (!rv) { if (set_add_entry((PySetObject *)result, key, hash)) { Py_DECREF(result); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } return result; } @@ -1534,17 +1552,21 @@ set_difference(PySetObject *so, PyObject *other) while (set_next(so, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = set_contains_entry((PySetObject *)other, key, hash); if (rv < 0) { Py_DECREF(result); + Py_DECREF(key); return NULL; } if (!rv) { if (set_add_entry((PySetObject *)result, key, hash)) { Py_DECREF(result); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } return result; } @@ -1641,17 +1663,21 @@ set_symmetric_difference_update(PySetObject *so, PyObject *other) while (set_next(otherset, &pos, &entry)) { key = entry->key; hash = entry->hash; + Py_INCREF(key); rv = set_discard_entry(so, key, hash); if (rv < 0) { Py_DECREF(otherset); + Py_DECREF(key); return NULL; } if (rv == DISCARD_NOTFOUND) { if (set_add_entry(so, key, hash)) { Py_DECREF(otherset); + Py_DECREF(key); return NULL; } } + Py_DECREF(key); } Py_DECREF(otherset); Py_RETURN_NONE; @@ -1726,11 +1752,16 @@ set_issubset(PySetObject *so, PyObject *other) Py_RETURN_FALSE; while (set_next(so, &pos, &entry)) { - rv = set_contains_entry((PySetObject *)other, entry->key, entry->hash); - if (rv < 0) + PyObject *key = entry->key; + Py_INCREF(key); + rv = set_contains_entry((PySetObject *)other, key, entry->hash); + Py_DECREF(key); + if (rv < 0) { return NULL; - if (!rv) + } + if (!rv) { Py_RETURN_FALSE; + } } Py_RETURN_TRUE; } From webhook-mailer at python.org Sun Feb 13 16:43:19 2022 From: webhook-mailer at python.org (rhettinger) Date: Sun, 13 Feb 2022 21:43:19 -0000 Subject: [Python-checkins] Improve docstring for set.discard() (GH-31315) Message-ID: https://github.com/python/cpython/commit/0ae40191793da1877a12d512f0116d99301b2c51 commit: 0ae40191793da1877a12d512f0116d99301b2c51 branch: main author: Raymond Hettinger committer: rhettinger date: 2022-02-13T15:42:59-06:00 summary: Improve docstring for set.discard() (GH-31315) files: M Objects/setobject.c diff --git a/Objects/setobject.c b/Objects/setobject.c index 0dd28402afbd2..c65b7d5d21115 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -1930,7 +1930,8 @@ set_discard(PySetObject *so, PyObject *key) PyDoc_STRVAR(discard_doc, "Remove an element from a set if it is a member.\n\ \n\ -If the element is not a member, do nothing."); +Unlike set.remove(), the discard() method does not raise\n\ +an exception when an element is missing from the set."); static PyObject * set_reduce(PySetObject *so, PyObject *Py_UNUSED(ignored)) From webhook-mailer at python.org Sun Feb 13 19:00:38 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 14 Feb 2022 00:00:38 -0000 Subject: [Python-checkins] Corrections to format precision description. (GH-31291) Message-ID: https://github.com/python/cpython/commit/1d6ce67c29aa2166ef326952cb605b908fb4f987 commit: 1d6ce67c29aa2166ef326952cb605b908fb4f987 branch: main author: John Belmonte committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-13T16:00:32-08:00 summary: Corrections to format precision description. (GH-31291) * `precision` field is a decimal integer * clarify that stated limitations are on presentation type rather than input value type. Especially misleading is "precision is not allowed for integer values", since integer value input to a format like `.1f` is fine. * regarding max field size, replace "non-number" with "string", which is the only non-numeric presentation type Automerge-Triggered-By: GH:ericvsmith files: M Doc/library/string.rst diff --git a/Doc/library/string.rst b/Doc/library/string.rst index 9bc703e70cdaa..78bd167bcf579 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -428,12 +428,13 @@ character of ``'0'`` with an *alignment* type of ``'='``. Preceding the *width* field by ``'0'`` no longer affects the default alignment for strings. -The *precision* is a decimal number indicating how many digits should be -displayed after the decimal point for a floating point value formatted with -``'f'`` and ``'F'``, or before and after the decimal point for a floating point -value formatted with ``'g'`` or ``'G'``. For non-number types the field +The *precision* is a decimal integer indicating how many digits should be +displayed after the decimal point for presentation types +``'f'`` and ``'F'``, or before and after the decimal point for presentation +types ``'g'`` or ``'G'``. For string presentation types the field indicates the maximum field size - in other words, how many characters will be -used from the field content. The *precision* is not allowed for integer values. +used from the field content. The *precision* is not allowed for integer +presentation types. Finally, the *type* determines how the data should be presented. From webhook-mailer at python.org Sun Feb 13 19:21:36 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 14 Feb 2022 00:21:36 -0000 Subject: [Python-checkins] Corrections to format precision description. (GH-31291) Message-ID: https://github.com/python/cpython/commit/6a7dd3f9e9771279c8ee71dd3fd39a42f9b0ccff commit: 6a7dd3f9e9771279c8ee71dd3fd39a42f9b0ccff branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-13T16:21:32-08:00 summary: Corrections to format precision description. (GH-31291) * `precision` field is a decimal integer * clarify that stated limitations are on presentation type rather than input value type. Especially misleading is "precision is not allowed for integer values", since integer value input to a format like `.1f` is fine. * regarding max field size, replace "non-number" with "string", which is the only non-numeric presentation type Automerge-Triggered-By: GH:ericvsmith (cherry picked from commit 1d6ce67c29aa2166ef326952cb605b908fb4f987) Co-authored-by: John Belmonte files: M Doc/library/string.rst diff --git a/Doc/library/string.rst b/Doc/library/string.rst index b27782f8d8e9b..b7108641805e6 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -428,12 +428,13 @@ character of ``'0'`` with an *alignment* type of ``'='``. Preceding the *width* field by ``'0'`` no longer affects the default alignment for strings. -The *precision* is a decimal number indicating how many digits should be -displayed after the decimal point for a floating point value formatted with -``'f'`` and ``'F'``, or before and after the decimal point for a floating point -value formatted with ``'g'`` or ``'G'``. For non-number types the field +The *precision* is a decimal integer indicating how many digits should be +displayed after the decimal point for presentation types +``'f'`` and ``'F'``, or before and after the decimal point for presentation +types ``'g'`` or ``'G'``. For string presentation types the field indicates the maximum field size - in other words, how many characters will be -used from the field content. The *precision* is not allowed for integer values. +used from the field content. The *precision* is not allowed for integer +presentation types. Finally, the *type* determines how the data should be presented. From webhook-mailer at python.org Sun Feb 13 22:05:20 2022 From: webhook-mailer at python.org (ericvsmith) Date: Mon, 14 Feb 2022 03:05:20 -0000 Subject: [Python-checkins] [3.9] Corrections to format precision description. (GH-31291) (GH-31321) Message-ID: https://github.com/python/cpython/commit/0b1417d597c9ed5f9cd981fbf62824f41c740b17 commit: 0b1417d597c9ed5f9cd981fbf62824f41c740b17 branch: 3.9 author: John Belmonte committer: ericvsmith date: 2022-02-13T22:05:11-05:00 summary: [3.9] Corrections to format precision description. (GH-31291) (GH-31321) * `precision` field is a decimal integer * clarify that stated limitations are on presentation type rather than input value type. Especially misleading is "precision is not allowed for integer values", since integer value input to a format like `.1f` is fine. * regarding max field size, replace "non-number" with "string", which is the only non-numeric presentation type Automerge-Triggered-By: GH:ericvsmith. (cherry picked from commit 1d6ce67c29aa2166ef326952cb605b908fb4f987) Co-authored-by: John Belmonte files: M Doc/library/string.rst diff --git a/Doc/library/string.rst b/Doc/library/string.rst index d1bdf4347de19..3ecae27b38577 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -424,12 +424,13 @@ When no explicit alignment is given, preceding the *width* field by a zero sign-aware zero-padding for numeric types. This is equivalent to a *fill* character of ``'0'`` with an *alignment* type of ``'='``. -The *precision* is a decimal number indicating how many digits should be -displayed after the decimal point for a floating point value formatted with -``'f'`` and ``'F'``, or before and after the decimal point for a floating point -value formatted with ``'g'`` or ``'G'``. For non-number types the field +The *precision* is a decimal integer indicating how many digits should be +displayed after the decimal point for presentation types +``'f'`` and ``'F'``, or before and after the decimal point for presentation +types ``'g'`` or ``'G'``. For string presentation types the field indicates the maximum field size - in other words, how many characters will be -used from the field content. The *precision* is not allowed for integer values. +used from the field content. The *precision* is not allowed for integer +presentation types. Finally, the *type* determines how the data should be presented. From webhook-mailer at python.org Mon Feb 14 05:02:07 2022 From: webhook-mailer at python.org (markshannon) Date: Mon, 14 Feb 2022 10:02:07 -0000 Subject: [Python-checkins] Include length in stats for UNPACK_SEQUENCE. (GH-31254) Message-ID: https://github.com/python/cpython/commit/15ee55528e1cbc47ef7e9f64186393eee40a49d9 commit: 15ee55528e1cbc47ef7e9f64186393eee40a49d9 branch: main author: Mark Shannon committer: markshannon date: 2022-02-14T10:01:31Z summary: Include length in stats for UNPACK_SEQUENCE. (GH-31254) files: M Python/ceval.c M Python/specialize.c diff --git a/Python/ceval.c b/Python/ceval.c index 5eb91502c3010..ad8b05400d564 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2740,10 +2740,10 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PREDICTED(UNPACK_SEQUENCE); PyObject *seq = POP(), *item, **items; #ifdef Py_STATS - extern int _PySpecialization_ClassifySequence(PyObject *); + extern int _PySpecialization_ClassifySequence(PyObject *, int); _py_stats.opcode_stats[UNPACK_SEQUENCE].specialization.failure++; _py_stats.opcode_stats[UNPACK_SEQUENCE].specialization. - failure_kinds[_PySpecialization_ClassifySequence(seq)]++; + failure_kinds[_PySpecialization_ClassifySequence(seq, oparg)]++; #endif if (PyTuple_CheckExact(seq) && PyTuple_GET_SIZE(seq) == oparg) { diff --git a/Python/specialize.c b/Python/specialize.c index 1259a3ca3eccc..b54a2ecd506fc 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -602,8 +602,26 @@ initial_counter_value(void) { #define SPEC_FAIL_FOR_ITER_ENUMERATE 23 /* UNPACK_SEQUENCE */ -#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE 10 -#define SPEC_FAIL_UNPACK_SEQUENCE_LIST 11 +#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_0 9 +#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_1 10 +#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_2 11 +#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_3 12 +#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_4 13 +#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_N 14 + +#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_0 15 +#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_1 16 +#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_2 17 +#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_3 18 +#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_4 19 +#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_N 20 + +#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_0 21 +#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_1 22 +#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_2 23 +#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_3 24 +#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_4 25 +#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_N 26 static int @@ -1978,15 +1996,19 @@ int } int -_PySpecialization_ClassifySequence(PyObject *seq) +_PySpecialization_ClassifySequence(PyObject *seq, int n) { + assert(n >= 0); + if (n > 4) { + n = 5; + } if (PyTuple_CheckExact(seq)) { - return SPEC_FAIL_UNPACK_SEQUENCE_TUPLE; + return SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_0 + n; } if (PyList_CheckExact(seq)) { - return SPEC_FAIL_UNPACK_SEQUENCE_LIST; + return SPEC_FAIL_UNPACK_SEQUENCE_LIST_0 + n; } - return SPEC_FAIL_OTHER; + return SPEC_FAIL_UNPACK_SEQUENCE_OTHER_0 + n; } int From webhook-mailer at python.org Mon Feb 14 10:54:01 2022 From: webhook-mailer at python.org (markshannon) Date: Mon, 14 Feb 2022 15:54:01 -0000 Subject: [Python-checkins] Add pair counts to stats output and summary. (GH-31324) Message-ID: https://github.com/python/cpython/commit/0ade875ebe0d83709ec338616d6abe439c81f29b commit: 0ade875ebe0d83709ec338616d6abe439c81f29b branch: main author: Mark Shannon committer: markshannon date: 2022-02-14T15:53:38Z summary: Add pair counts to stats output and summary. (GH-31324) files: M Python/specialize.c M Tools/scripts/summarize_stats.py diff --git a/Python/specialize.c b/Python/specialize.c index b54a2ecd506fc..0e1ffad1b9159 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -174,13 +174,13 @@ print_spec_stats(FILE *out, OpcodeStats *stats) { /* Mark some opcodes as specializable for stats, * even though we don't specialize them yet. */ - fprintf(out, " opcode[%d].specializable : 1\n", FOR_ITER); - fprintf(out, " opcode[%d].specializable : 1\n", PRECALL_FUNCTION); - fprintf(out, " opcode[%d].specializable : 1\n", PRECALL_METHOD); - fprintf(out, " opcode[%d].specializable : 1\n", UNPACK_SEQUENCE); + fprintf(out, "opcode[%d].specializable : 1\n", FOR_ITER); + fprintf(out, "opcode[%d].specializable : 1\n", PRECALL_FUNCTION); + fprintf(out, "opcode[%d].specializable : 1\n", PRECALL_METHOD); + fprintf(out, "opcode[%d].specializable : 1\n", UNPACK_SEQUENCE); for (int i = 0; i < 256; i++) { if (adaptive_opcodes[i]) { - fprintf(out, " opcode[%d].specializable : 1\n", i); + fprintf(out, "opcode[%d].specializable : 1\n", i); } PRINT_STAT(i, specialization.success); PRINT_STAT(i, specialization.failure); @@ -196,6 +196,12 @@ print_spec_stats(FILE *out, OpcodeStats *stats) PRIu64 "\n", i, j, val); } } + for(int j = 0; j < 256; j++) { + if (stats[i].pair_count[j]) { + fprintf(out, "opcode[%d].pair_count[%d] : %" PRIu64 "\n", + i, j, stats[i].pair_count[j]); + } + } } } #undef PRINT_STAT diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index f6e5b333b09fc..5805ba46afed6 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -6,6 +6,8 @@ import os.path import opcode from datetime import date +import itertools +import argparse if os.name == "nt": DEFAULT_DIR = "c:\\temp\\py_stats\\" @@ -80,7 +82,7 @@ def gather_stats(): for line in fd: key, value = line.split(":") key = key.strip() - value = int(value.strip()) + value = int(value) stats[key] += value return stats @@ -268,14 +270,42 @@ def emit_object_stats(stats): rows.append((label, value, materialize)) emit_table(("", "Count:", "Ratio:"), rows) -def main(): - stats = gather_stats() - opcode_stats = extract_opcode_stats(stats) +def get_total(opcode_stats): total = 0 - for i, opcode_stat in enumerate(opcode_stats): + for opcode_stat in opcode_stats: if "execution_count" in opcode_stat: total += opcode_stat['execution_count'] + return total + +def emit_pair_counts(opcode_stats, total): + with Section("Pair counts", summary="Pair counts for top 100 pairs"): + pair_counts = [] + for i, opcode_stat in enumerate(opcode_stats): + if i == 0: + continue + for key, value in opcode_stat.items(): + if key.startswith("pair_count"): + x, _, _ = key[11:].partition("]") + if value: + pair_counts.append((value, (i, int(x)))) + pair_counts.sort(reverse=True) + cumulative = 0 + rows = [] + for (count, pair) in itertools.islice(pair_counts, 100): + i, j = pair + cumulative += count + rows.append((opname[i] + " " + opname[j], count, f"{100*count/total:0.1f}%", + f"{100*cumulative/total:0.1f}%")) + emit_table(("Pair", "Count:", "Self:", "Cumulative:"), + rows + ) + +def main(): + stats = gather_stats() + opcode_stats = extract_opcode_stats(stats) + total = get_total(opcode_stats) emit_execution_counts(opcode_stats, total) + emit_pair_counts(opcode_stats, total) emit_specialization_stats(opcode_stats) emit_specialization_overview(opcode_stats, total) emit_call_stats(stats) From webhook-mailer at python.org Mon Feb 14 11:17:00 2022 From: webhook-mailer at python.org (sweeneyde) Date: Mon, 14 Feb 2022 16:17:00 -0000 Subject: [Python-checkins] bpo-46747: Add missing key parameters in the bisect docs (GH-31323) Message-ID: https://github.com/python/cpython/commit/96084f4256d2d523b0a4d7d900322b032326e3ed commit: 96084f4256d2d523b0a4d7d900322b032326e3ed branch: main author: Zackery Spytz committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2022-02-14T11:16:49-05:00 summary: bpo-46747: Add missing key parameters in the bisect docs (GH-31323) Added *key* parameter to `bisect.bisect()` and `bisect.insort()` in bisect module docs. files: M Doc/library/bisect.rst diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index f34ee175ba657..edcd4aeb24aa7 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -43,7 +43,7 @@ The following functions are provided: .. function:: bisect_right(a, x, lo=0, hi=len(a), *, key=None) - bisect(a, x, lo=0, hi=len(a)) + bisect(a, x, lo=0, hi=len(a), *, key=None) Similar to :func:`bisect_left`, but returns an insertion point which comes after (to the right of) any existing entries of *x* in *a*. @@ -80,7 +80,7 @@ The following functions are provided: .. function:: insort_right(a, x, lo=0, hi=len(a), *, key=None) - insort(a, x, lo=0, hi=len(a)) + insort(a, x, lo=0, hi=len(a), *, key=None) Similar to :func:`insort_left`, but inserting *x* in *a* after any existing entries of *x*. From webhook-mailer at python.org Mon Feb 14 11:51:33 2022 From: webhook-mailer at python.org (sweeneyde) Date: Mon, 14 Feb 2022 16:51:33 -0000 Subject: [Python-checkins] [3.10] bpo-46747: Add missing key parameters in the bisect docs (GH-31323) (GH-31329) Message-ID: https://github.com/python/cpython/commit/841c77d802e9ee8845fa3152700474021efe03fd commit: 841c77d802e9ee8845fa3152700474021efe03fd branch: 3.10 author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2022-02-14T11:51:16-05:00 summary: [3.10] bpo-46747: Add missing key parameters in the bisect docs (GH-31323) (GH-31329) Added *key* parameter to `bisect.bisect()` and `bisect.insort()` in bisect module docs. (cherry picked from commit 96084f4256d2d523b0a4d7d900322b032326e3ed) Co-authored-by: Zackery Spytz files: M Doc/library/bisect.rst diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index f34ee175ba657..edcd4aeb24aa7 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -43,7 +43,7 @@ The following functions are provided: .. function:: bisect_right(a, x, lo=0, hi=len(a), *, key=None) - bisect(a, x, lo=0, hi=len(a)) + bisect(a, x, lo=0, hi=len(a), *, key=None) Similar to :func:`bisect_left`, but returns an insertion point which comes after (to the right of) any existing entries of *x* in *a*. @@ -80,7 +80,7 @@ The following functions are provided: .. function:: insort_right(a, x, lo=0, hi=len(a), *, key=None) - insort(a, x, lo=0, hi=len(a)) + insort(a, x, lo=0, hi=len(a), *, key=None) Similar to :func:`insort_left`, but inserting *x* in *a* after any existing entries of *x*. From webhook-mailer at python.org Mon Feb 14 12:49:09 2022 From: webhook-mailer at python.org (markshannon) Date: Mon, 14 Feb 2022 17:49:09 -0000 Subject: [Python-checkins] Remove unnecessary basic block from comprehensions (GH-31263) Message-ID: https://github.com/python/cpython/commit/3b799d7448588d28de09ef9e3cc30c72c17db334 commit: 3b799d7448588d28de09ef9e3cc30c72c17db334 branch: main author: Brandt Bucher committer: markshannon date: 2022-02-14T17:48:45Z summary: Remove unnecessary basic block from comprehensions (GH-31263) files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index bfe451b8c1047..ac4960b5df320 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -5034,17 +5034,16 @@ compiler_sync_comprehension_generator(struct compiler *c, and then write to the element */ comprehension_ty gen; - basicblock *start, *anchor, *skip, *if_cleanup; + basicblock *start, *anchor, *if_cleanup; Py_ssize_t i, n; start = compiler_new_block(c); - skip = compiler_new_block(c); if_cleanup = compiler_new_block(c); anchor = compiler_new_block(c); - if (start == NULL || skip == NULL || if_cleanup == NULL || - anchor == NULL) + if (start == NULL || if_cleanup == NULL || anchor == NULL) { return 0; + } gen = (comprehension_ty)asdl_seq_GET(generators, gen_index); @@ -5131,8 +5130,6 @@ compiler_sync_comprehension_generator(struct compiler *c, default: return 0; } - - compiler_use_next_block(c, skip); } compiler_use_next_block(c, if_cleanup); if (start) { From webhook-mailer at python.org Mon Feb 14 12:56:18 2022 From: webhook-mailer at python.org (ambv) Date: Mon, 14 Feb 2022 17:56:18 -0000 Subject: [Python-checkins] [3.8] bpo-46474: Avoid REDoS in EntryPoint.pattern (sync with importlib_metadata 4.10.1) (GH-30803). (#30829) Message-ID: https://github.com/python/cpython/commit/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656 commit: 8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656 branch: 3.8 author: Jason R. Coombs committer: ambv date: 2022-02-14T18:56:03+01:00 summary: [3.8] bpo-46474: Avoid REDoS in EntryPoint.pattern (sync with importlib_metadata 4.10.1) (GH-30803). (#30829) (cherry picked from commit 51c3e28c8a163e58dc753765e3cc51d5a717e70d) Co-authored-by: Jason R. Coombs files: A Misc/NEWS.d/next/Library/2022-01-22-14-49-10.bpo-46474.eKQhvx.rst M Lib/importlib/metadata.py diff --git a/Lib/importlib/metadata.py b/Lib/importlib/metadata.py index 9d2285cb4d46d..c8d533c5c2b32 100644 --- a/Lib/importlib/metadata.py +++ b/Lib/importlib/metadata.py @@ -49,8 +49,8 @@ class EntryPoint( pattern = re.compile( r'(?P[\w.]+)\s*' - r'(:\s*(?P[\w.]+))?\s*' - r'(?P\[.*\])?\s*$' + r'(:\s*(?P[\w.]+)\s*)?' + r'((?P\[.*\])\s*)?$' ) """ A regular expression describing the syntax for an entry point, diff --git a/Misc/NEWS.d/next/Library/2022-01-22-14-49-10.bpo-46474.eKQhvx.rst b/Misc/NEWS.d/next/Library/2022-01-22-14-49-10.bpo-46474.eKQhvx.rst new file mode 100644 index 0000000000000..156b7de4f6787 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-22-14-49-10.bpo-46474.eKQhvx.rst @@ -0,0 +1,2 @@ +In ``importlib.metadata.EntryPoint.pattern``, avoid potential REDoS by +limiting ambiguity in consecutive whitespace. From webhook-mailer at python.org Mon Feb 14 13:45:58 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 18:45:58 -0000 Subject: [Python-checkins] Fix minor grammar error (#31325) Message-ID: https://github.com/python/cpython/commit/0d29e7a06f98e8312b699d4531d27b76add1249f commit: 0d29e7a06f98e8312b699d4531d27b76add1249f branch: main author: Scott Noyes committer: Mariatta date: 2022-02-14T10:45:48-08:00 summary: Fix minor grammar error (#31325) "an" followed by consonant should be "a" files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 154563a859e31..c156390d375e1 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1819,7 +1819,7 @@ for ``None``. This reads like plain English in code and avoids confusion with other objects that may have boolean values that evaluate to false. 2) Detecting optional arguments can be tricky when ``None`` is a valid input -value. In those situations, you can create an singleton sentinel object +value. In those situations, you can create a singleton sentinel object guaranteed to be distinct from other objects. For example, here is how to implement a method that behaves like :meth:`dict.pop`:: From webhook-mailer at python.org Mon Feb 14 13:49:36 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 18:49:36 -0000 Subject: [Python-checkins] Fix a typo in Doc/faq/programming.rst (GH-31243) Message-ID: https://github.com/python/cpython/commit/ff5a16759744ee0a43c5d1c209f47e256abc7718 commit: ff5a16759744ee0a43c5d1c209f47e256abc7718 branch: main author: Cooper Lees committer: Mariatta date: 2022-02-14T10:49:28-08:00 summary: Fix a typo in Doc/faq/programming.rst (GH-31243) is not longer used -> is no longer used files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index c156390d375e1..a1adf851bdded 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1908,7 +1908,7 @@ The *cached_property* approach only works with methods that do not take any arguments. It does not create a reference to the instance. The cached method result will be kept only as long as the instance is alive. -The advantage is that when an instance is not longer used, the cached +The advantage is that when an instance is no longer used, the cached method result will be released right away. The disadvantage is that if instances accumulate, so too will the accumulated method results. They can grow without bound. From webhook-mailer at python.org Mon Feb 14 13:50:19 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 18:50:19 -0000 Subject: [Python-checkins] Update further README references from 3.10 to 3.11 (GH-31009) Message-ID: https://github.com/python/cpython/commit/e4abd464c12b235c0003221cbc2e98be51e8087b commit: e4abd464c12b235c0003221cbc2e98be51e8087b branch: main author: Jacob Walls committer: Mariatta date: 2022-02-14T10:50:15-08:00 summary: Update further README references from 3.10 to 3.11 (GH-31009) files: M README.rst diff --git a/README.rst b/README.rst index 36611add5ccac..b34401e2beb87 100644 --- a/README.rst +++ b/README.rst @@ -133,7 +133,7 @@ What's New ---------- We have a comprehensive overview of the changes in the `What's New in Python -3.10 `_ document. For a more +3.11 `_ document. For a more detailed change log, read `Misc/NEWS `_, but a full accounting of changes can only be gleaned from the `commit history @@ -146,7 +146,7 @@ entitled "Installing multiple versions". Documentation ------------- -`Documentation for Python 3.10 `_ is online, +`Documentation for Python 3.11 `_ is online, updated daily. It can also be downloaded in many formats for faster access. The documentation @@ -205,8 +205,8 @@ intend to install multiple versions using the same prefix you must decide which version (if any) is your "primary" version. Install that version using ``make install``. Install all other versions using ``make altinstall``. -For example, if you want to install Python 2.7, 3.6, and 3.10 with 3.10 being the -primary version, you would execute ``make install`` in your 3.10 build directory +For example, if you want to install Python 2.7, 3.6, and 3.11 with 3.11 being the +primary version, you would execute ``make install`` in your 3.11 build directory and ``make altinstall`` in the others. From webhook-mailer at python.org Mon Feb 14 13:53:33 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 18:53:33 -0000 Subject: [Python-checkins] Update __main__.rst (GH-30896) Message-ID: https://github.com/python/cpython/commit/4e4e7c2684b80350e82d82e4feeda5de24d362b9 commit: 4e4e7c2684b80350e82d82e4feeda5de24d362b9 branch: main author: David Gilbertson committer: Mariatta date: 2022-02-14T10:53:16-08:00 summary: Update __main__.rst (GH-30896) The reference to `venv` appears in the paragraph below, instead of above. Fixed the documentation. files: M Doc/library/__main__.rst diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index c5f36a7fcf077..d0a65e76b8423 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -253,7 +253,7 @@ attribute will include the package's path if imported:: This won't work for ``__main__.py`` files in the root directory of a .zip file though. Hence, for consistency, minimal ``__main__.py`` like the :mod:`venv` -one mentioned above are preferred. +one mentioned below are preferred. .. seealso:: From webhook-mailer at python.org Mon Feb 14 13:58:31 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 18:58:31 -0000 Subject: [Python-checkins] bpo-38619: Update the documentation for UUID.hex (GH-29830) Message-ID: https://github.com/python/cpython/commit/5d53cf30f9cb3758849e859db5d4602cb7c521f7 commit: 5d53cf30f9cb3758849e859db5d4602cb7c521f7 branch: main author: 180909 committer: Mariatta date: 2022-02-14T10:58:22-08:00 summary: bpo-38619: Update the documentation for UUID.hex (GH-29830) Explicitly state that it is lowercase. Co-authored-by: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> files: M Doc/library/uuid.rst diff --git a/Doc/library/uuid.rst b/Doc/library/uuid.rst index 415e25bef6de7..ddfbbda279e35 100644 --- a/Doc/library/uuid.rst +++ b/Doc/library/uuid.rst @@ -118,7 +118,7 @@ which relays any information about the UUID's safety, using this enumeration: .. attribute:: UUID.hex - The UUID as a 32-character hexadecimal string. + The UUID as a 32-character lowercase hexadecimal string. .. attribute:: UUID.int From webhook-mailer at python.org Mon Feb 14 14:08:47 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 14 Feb 2022 19:08:47 -0000 Subject: [Python-checkins] Fix a typo in Doc/faq/programming.rst (GH-31243) Message-ID: https://github.com/python/cpython/commit/a606c6a350133a2bebff8bca48129f39de885218 commit: a606c6a350133a2bebff8bca48129f39de885218 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-14T11:08:39-08:00 summary: Fix a typo in Doc/faq/programming.rst (GH-31243) is not longer used -> is no longer used (cherry picked from commit ff5a16759744ee0a43c5d1c209f47e256abc7718) Co-authored-by: Cooper Lees files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 154563a859e31..a6831ff9d79b7 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1908,7 +1908,7 @@ The *cached_property* approach only works with methods that do not take any arguments. It does not create a reference to the instance. The cached method result will be kept only as long as the instance is alive. -The advantage is that when an instance is not longer used, the cached +The advantage is that when an instance is no longer used, the cached method result will be released right away. The disadvantage is that if instances accumulate, so too will the accumulated method results. They can grow without bound. From webhook-mailer at python.org Mon Feb 14 14:10:02 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 14 Feb 2022 19:10:02 -0000 Subject: [Python-checkins] Fix minor grammar error (GH-31325) Message-ID: https://github.com/python/cpython/commit/5574cacf5727764c462f9f37cc30e68a3dacbaa5 commit: 5574cacf5727764c462f9f37cc30e68a3dacbaa5 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-14T11:09:57-08:00 summary: Fix minor grammar error (GH-31325) "an" followed by consonant should be "a" (cherry picked from commit 0d29e7a06f98e8312b699d4531d27b76add1249f) Co-authored-by: Scott Noyes files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index a6831ff9d79b7..a1adf851bdded 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1819,7 +1819,7 @@ for ``None``. This reads like plain English in code and avoids confusion with other objects that may have boolean values that evaluate to false. 2) Detecting optional arguments can be tricky when ``None`` is a valid input -value. In those situations, you can create an singleton sentinel object +value. In those situations, you can create a singleton sentinel object guaranteed to be distinct from other objects. For example, here is how to implement a method that behaves like :meth:`dict.pop`:: From webhook-mailer at python.org Mon Feb 14 14:10:43 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 14 Feb 2022 19:10:43 -0000 Subject: [Python-checkins] Fix minor grammar error (GH-31325) Message-ID: https://github.com/python/cpython/commit/0bd2d58f36af4e6482abe57509c418681a898956 commit: 0bd2d58f36af4e6482abe57509c418681a898956 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-14T11:10:38-08:00 summary: Fix minor grammar error (GH-31325) "an" followed by consonant should be "a" (cherry picked from commit 0d29e7a06f98e8312b699d4531d27b76add1249f) Co-authored-by: Scott Noyes files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index d838e6141f8d4..3cf041e1a4230 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1820,7 +1820,7 @@ for ``None``. This reads like plain English in code and avoids confusion with other objects that may have boolean values that evaluate to false. 2) Detecting optional arguments can be tricky when ``None`` is a valid input -value. In those situations, you can create an singleton sentinel object +value. In those situations, you can create a singleton sentinel object guaranteed to be distinct from other objects. For example, here is how to implement a method that behaves like :meth:`dict.pop`:: From webhook-mailer at python.org Mon Feb 14 14:20:57 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 14 Feb 2022 19:20:57 -0000 Subject: [Python-checkins] bpo-38619: Update the documentation for UUID.hex (GH-29830) Message-ID: https://github.com/python/cpython/commit/828253227efe16f0e759df36bbb2ca49083223c8 commit: 828253227efe16f0e759df36bbb2ca49083223c8 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-14T11:20:48-08:00 summary: bpo-38619: Update the documentation for UUID.hex (GH-29830) Explicitly state that it is lowercase. Co-authored-by: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> (cherry picked from commit 5d53cf30f9cb3758849e859db5d4602cb7c521f7) Co-authored-by: 180909 files: M Doc/library/uuid.rst diff --git a/Doc/library/uuid.rst b/Doc/library/uuid.rst index 415e25bef6de7..ddfbbda279e35 100644 --- a/Doc/library/uuid.rst +++ b/Doc/library/uuid.rst @@ -118,7 +118,7 @@ which relays any information about the UUID's safety, using this enumeration: .. attribute:: UUID.hex - The UUID as a 32-character hexadecimal string. + The UUID as a 32-character lowercase hexadecimal string. .. attribute:: UUID.int From webhook-mailer at python.org Mon Feb 14 14:25:35 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 14 Feb 2022 19:25:35 -0000 Subject: [Python-checkins] bpo-38619: Update the documentation for UUID.hex (GH-29830) Message-ID: https://github.com/python/cpython/commit/1953f03174c18f315aca883babc3819828f868d8 commit: 1953f03174c18f315aca883babc3819828f868d8 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-14T11:25:15-08:00 summary: bpo-38619: Update the documentation for UUID.hex (GH-29830) Explicitly state that it is lowercase. Co-authored-by: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> (cherry picked from commit 5d53cf30f9cb3758849e859db5d4602cb7c521f7) Co-authored-by: 180909 files: M Doc/library/uuid.rst diff --git a/Doc/library/uuid.rst b/Doc/library/uuid.rst index 415e25bef6de7..ddfbbda279e35 100644 --- a/Doc/library/uuid.rst +++ b/Doc/library/uuid.rst @@ -118,7 +118,7 @@ which relays any information about the UUID's safety, using this enumeration: .. attribute:: UUID.hex - The UUID as a 32-character hexadecimal string. + The UUID as a 32-character lowercase hexadecimal string. .. attribute:: UUID.int From webhook-mailer at python.org Mon Feb 14 14:46:56 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 19:46:56 -0000 Subject: [Python-checkins] doc: use colon for all color's rangs (GH-28998) Message-ID: https://github.com/python/cpython/commit/6b9f27dd0b5da53b1fd648494879d857572bec80 commit: 6b9f27dd0b5da53b1fd648494879d857572bec80 branch: main author: Rafael Fontenelle committer: Mariatta date: 2022-02-14T11:46:40-08:00 summary: doc: use colon for all color's rangs (GH-28998) files: M Doc/library/tkinter.rst diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index a48bc13c8b436..65395d824cab7 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -756,8 +756,8 @@ callback color Colors can be given as the names of X colors in the rgb.txt file, or as strings - representing RGB values in 4 bit: ``"#RGB"``, 8 bit: ``"#RRGGBB"``, 12 bit" - ``"#RRRGGGBBB"``, or 16 bit ``"#RRRRGGGGBBBB"`` ranges, where R,G,B here + representing RGB values in 4 bit: ``"#RGB"``, 8 bit: ``"#RRGGBB"``, 12 bit: + ``"#RRRGGGBBB"``, or 16 bit: ``"#RRRRGGGGBBBB"`` ranges, where R,G,B here represent any legal hex digit. See page 160 of Ousterhout's book for details. cursor From webhook-mailer at python.org Mon Feb 14 15:02:04 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 20:02:04 -0000 Subject: [Python-checkins] Fix missing import in dataclass code snippet (GH-29214) Message-ID: https://github.com/python/cpython/commit/ad4e8d2b871c4f5bce27520627bbb2e0e544bc24 commit: ad4e8d2b871c4f5bce27520627bbb2e0e544bc24 branch: main author: Leo committer: Mariatta date: 2022-02-14T12:01:53-08:00 summary: Fix missing import in dataclass code snippet (GH-29214) field was not imported. It has been added now. files: M Doc/whatsnew/3.10.rst diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 806170ede1596..905305be3176c 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1019,7 +1019,7 @@ You can specify keyword-only on a per-field basis: .. code-block:: python - from dataclasses import dataclass + from dataclasses import dataclass, field @dataclass class Birthday: From webhook-mailer at python.org Mon Feb 14 15:10:22 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 20:10:22 -0000 Subject: [Python-checkins] doc: use colon for all color's rangs (GH-28998) (#31336) Message-ID: https://github.com/python/cpython/commit/c5a987bf4d6cde9f9120aa2597ff49f765a684ce commit: c5a987bf4d6cde9f9120aa2597ff49f765a684ce branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Mariatta date: 2022-02-14T12:10:05-08:00 summary: doc: use colon for all color's rangs (GH-28998) (#31336) (cherry picked from commit 6b9f27dd0b5da53b1fd648494879d857572bec80) Co-authored-by: Rafael Fontenelle Co-authored-by: Rafael Fontenelle files: M Doc/library/tkinter.rst diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index a48bc13c8b436..65395d824cab7 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -756,8 +756,8 @@ callback color Colors can be given as the names of X colors in the rgb.txt file, or as strings - representing RGB values in 4 bit: ``"#RGB"``, 8 bit: ``"#RRGGBB"``, 12 bit" - ``"#RRRGGGBBB"``, or 16 bit ``"#RRRRGGGGBBBB"`` ranges, where R,G,B here + representing RGB values in 4 bit: ``"#RGB"``, 8 bit: ``"#RRGGBB"``, 12 bit: + ``"#RRRGGGBBB"``, or 16 bit: ``"#RRRRGGGGBBBB"`` ranges, where R,G,B here represent any legal hex digit. See page 160 of Ousterhout's book for details. cursor From webhook-mailer at python.org Mon Feb 14 15:10:44 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 20:10:44 -0000 Subject: [Python-checkins] Update __main__.rst (GH-30896) Message-ID: https://github.com/python/cpython/commit/a414cb35c2cbfc9516484fdef382c7487a54f563 commit: a414cb35c2cbfc9516484fdef382c7487a54f563 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Mariatta date: 2022-02-14T12:10:34-08:00 summary: Update __main__.rst (GH-30896) The reference to `venv` appears in the paragraph below, instead of above. Fixed the documentation. (cherry picked from commit 4e4e7c2684b80350e82d82e4feeda5de24d362b9) Co-authored-by: David Gilbertson Co-authored-by: David Gilbertson files: M Doc/library/__main__.rst diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index c5f36a7fcf077..d0a65e76b8423 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -253,7 +253,7 @@ attribute will include the package's path if imported:: This won't work for ``__main__.py`` files in the root directory of a .zip file though. Hence, for consistency, minimal ``__main__.py`` like the :mod:`venv` -one mentioned above are preferred. +one mentioned below are preferred. .. seealso:: From webhook-mailer at python.org Mon Feb 14 15:11:22 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 20:11:22 -0000 Subject: [Python-checkins] Fix the signature of multiprocessing.set_executable (GH-31276) Message-ID: https://github.com/python/cpython/commit/543242ad8ccbf098c23a192a0f2b0767d9ee198a commit: 543242ad8ccbf098c23a192a0f2b0767d9ee198a branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Mariatta date: 2022-02-14T12:11:02-08:00 summary: Fix the signature of multiprocessing.set_executable (GH-31276) Automerge-Triggered-By: GH:merwok (cherry picked from commit 4f9386661d51b78348395e78710f3bfbee9fd1de) Co-authored-by: G?ry Ogam Co-authored-by: G?ry Ogam files: M Doc/library/multiprocessing.rst diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index e0954b285b37b..096e6492cd42e 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -1047,9 +1047,9 @@ Miscellaneous .. versionadded:: 3.4 -.. function:: set_executable() +.. function:: set_executable(executable) - Sets the path of the Python interpreter to use when starting a child process. + Set the path of the Python interpreter to use when starting a child process. (By default :data:`sys.executable` is used). Embedders will probably need to do some thing like :: From webhook-mailer at python.org Mon Feb 14 15:12:26 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 20:12:26 -0000 Subject: [Python-checkins] bpo-46436: Fix command-line option -d/--directory in module http.server (GH-30701) Message-ID: https://github.com/python/cpython/commit/502ad3930ee8fcf76026edfc06a33621363cebea commit: 502ad3930ee8fcf76026edfc06a33621363cebea branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Mariatta date: 2022-02-14T12:12:15-08:00 summary: bpo-46436: Fix command-line option -d/--directory in module http.server (GH-30701) Fix command-line option -d/--directory in http.server main function that was ignored when combined with --cgi. Automerge-Triggered-By: GH:merwok (cherry picked from commit 2d080347d74078a55c47715d232d1ab8dc8cd603) Co-authored-by: G?ry Ogam Co-authored-by: G?ry Ogam files: A Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst M Lib/http/server.py diff --git a/Lib/http/server.py b/Lib/http/server.py index d7cce20432ead..2d2300c2aeab3 100644 --- a/Lib/http/server.py +++ b/Lib/http/server.py @@ -103,8 +103,6 @@ import sys import time import urllib.parse -import contextlib -from functools import partial from http import HTTPStatus @@ -1240,7 +1238,6 @@ def test(HandlerClass=BaseHTTPRequestHandler, """ ServerClass.address_family, addr = _get_best_family(bind, port) - HandlerClass.protocol_version = protocol with ServerClass(addr, HandlerClass) as httpd: host, port = httpd.socket.getsockname()[:2] @@ -1257,29 +1254,29 @@ def test(HandlerClass=BaseHTTPRequestHandler, if __name__ == '__main__': import argparse + import contextlib parser = argparse.ArgumentParser() parser.add_argument('--cgi', action='store_true', - help='Run as CGI Server') + help='run as CGI server') parser.add_argument('--bind', '-b', metavar='ADDRESS', - help='Specify alternate bind address ' - '[default: all interfaces]') + help='specify alternate bind address ' + '(default: all interfaces)') parser.add_argument('--directory', '-d', default=os.getcwd(), - help='Specify alternative directory ' - '[default:current directory]') - parser.add_argument('port', action='store', - default=8000, type=int, + help='specify alternate directory ' + '(default: current directory)') + parser.add_argument('port', action='store', default=8000, type=int, nargs='?', - help='Specify alternate port [default: 8000]') + help='specify alternate port (default: 8000)') args = parser.parse_args() if args.cgi: handler_class = CGIHTTPRequestHandler else: - handler_class = partial(SimpleHTTPRequestHandler, - directory=args.directory) + handler_class = SimpleHTTPRequestHandler # ensure dual-stack is not disabled; ref #38907 class DualStackServer(ThreadingHTTPServer): + def server_bind(self): # suppress exception when protocol is IPv4 with contextlib.suppress(Exception): @@ -1287,6 +1284,10 @@ def server_bind(self): socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0) return super().server_bind() + def finish_request(self, request, client_address): + self.RequestHandlerClass(request, client_address, self, + directory=args.directory) + test( HandlerClass=handler_class, ServerClass=DualStackServer, diff --git a/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst b/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst new file mode 100644 index 0000000000000..ccfd949506443 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst @@ -0,0 +1,3 @@ +Fix command-line option ``-d``/``--directory`` in module :mod:`http.server` +which is ignored when combined with command-line option ``--cgi``. Patch by +G?ry Ogam. From webhook-mailer at python.org Mon Feb 14 15:12:41 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 20:12:41 -0000 Subject: [Python-checkins] bpo-46436: Fix command-line option -d/--directory in module http.server (GH-30701) Message-ID: https://github.com/python/cpython/commit/b27195332e91e932501f16cf9877761b218a9c99 commit: b27195332e91e932501f16cf9877761b218a9c99 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Mariatta date: 2022-02-14T12:12:30-08:00 summary: bpo-46436: Fix command-line option -d/--directory in module http.server (GH-30701) Fix command-line option -d/--directory in http.server main function that was ignored when combined with --cgi. Automerge-Triggered-By: GH:merwok (cherry picked from commit 2d080347d74078a55c47715d232d1ab8dc8cd603) Co-authored-by: G?ry Ogam Co-authored-by: G?ry Ogam files: A Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst M Lib/http/server.py diff --git a/Lib/http/server.py b/Lib/http/server.py index e985dfd564008..58abadf73770e 100644 --- a/Lib/http/server.py +++ b/Lib/http/server.py @@ -103,8 +103,6 @@ import sys import time import urllib.parse -import contextlib -from functools import partial from http import HTTPStatus @@ -1239,7 +1237,6 @@ def test(HandlerClass=BaseHTTPRequestHandler, """ ServerClass.address_family, addr = _get_best_family(bind, port) - HandlerClass.protocol_version = protocol with ServerClass(addr, HandlerClass) as httpd: host, port = httpd.socket.getsockname()[:2] @@ -1256,29 +1253,29 @@ def test(HandlerClass=BaseHTTPRequestHandler, if __name__ == '__main__': import argparse + import contextlib parser = argparse.ArgumentParser() parser.add_argument('--cgi', action='store_true', - help='Run as CGI Server') + help='run as CGI server') parser.add_argument('--bind', '-b', metavar='ADDRESS', - help='Specify alternate bind address ' - '[default: all interfaces]') + help='specify alternate bind address ' + '(default: all interfaces)') parser.add_argument('--directory', '-d', default=os.getcwd(), - help='Specify alternative directory ' - '[default:current directory]') - parser.add_argument('port', action='store', - default=8000, type=int, + help='specify alternate directory ' + '(default: current directory)') + parser.add_argument('port', action='store', default=8000, type=int, nargs='?', - help='Specify alternate port [default: 8000]') + help='specify alternate port (default: 8000)') args = parser.parse_args() if args.cgi: handler_class = CGIHTTPRequestHandler else: - handler_class = partial(SimpleHTTPRequestHandler, - directory=args.directory) + handler_class = SimpleHTTPRequestHandler # ensure dual-stack is not disabled; ref #38907 class DualStackServer(ThreadingHTTPServer): + def server_bind(self): # suppress exception when protocol is IPv4 with contextlib.suppress(Exception): @@ -1286,6 +1283,10 @@ def server_bind(self): socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0) return super().server_bind() + def finish_request(self, request, client_address): + self.RequestHandlerClass(request, client_address, self, + directory=args.directory) + test( HandlerClass=handler_class, ServerClass=DualStackServer, diff --git a/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst b/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst new file mode 100644 index 0000000000000..ccfd949506443 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-23-19-37-00.bpo-46436.Biz1p9.rst @@ -0,0 +1,3 @@ +Fix command-line option ``-d``/``--directory`` in module :mod:`http.server` +which is ignored when combined with command-line option ``--cgi``. Patch by +G?ry Ogam. From webhook-mailer at python.org Mon Feb 14 15:13:46 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 14 Feb 2022 20:13:46 -0000 Subject: [Python-checkins] doc: use colon for all color's rangs (GH-28998) Message-ID: https://github.com/python/cpython/commit/5c303f32094230baad8295541f948f8d43ba2ce2 commit: 5c303f32094230baad8295541f948f8d43ba2ce2 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-14T12:13:33-08:00 summary: doc: use colon for all color's rangs (GH-28998) (cherry picked from commit 6b9f27dd0b5da53b1fd648494879d857572bec80) Co-authored-by: Rafael Fontenelle files: M Doc/library/tkinter.rst diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index e1b2a87b0ff2e..999852a58cf6a 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -644,8 +644,8 @@ callback color Colors can be given as the names of X colors in the rgb.txt file, or as strings - representing RGB values in 4 bit: ``"#RGB"``, 8 bit: ``"#RRGGBB"``, 12 bit" - ``"#RRRGGGBBB"``, or 16 bit ``"#RRRRGGGGBBBB"`` ranges, where R,G,B here + representing RGB values in 4 bit: ``"#RGB"``, 8 bit: ``"#RRGGBB"``, 12 bit: + ``"#RRRGGGBBB"``, or 16 bit: ``"#RRRRGGGGBBBB"`` ranges, where R,G,B here represent any legal hex digit. See page 160 of Ousterhout's book for details. cursor From webhook-mailer at python.org Mon Feb 14 16:40:36 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 21:40:36 -0000 Subject: [Python-checkins] Update the copyright symbols in README.rst (GH-25514) Message-ID: https://github.com/python/cpython/commit/965be0e940649c851807b9936bf56fb668600f1f commit: 965be0e940649c851807b9936bf56fb668600f1f branch: main author: MishManners?? <36594527+mishmanners at users.noreply.github.com> committer: Mariatta date: 2022-02-14T13:40:19-08:00 summary: Update the copyright symbols in README.rst (GH-25514) Updated a couple copyright symbols from (c) to ? The rest of the doc are already using ? Co-authored-by: Mariatta Wijaya files: M README.rst diff --git a/README.rst b/README.rst index b34401e2beb87..e7bdd0f64efb5 100644 --- a/README.rst +++ b/README.rst @@ -14,7 +14,7 @@ This is Python version 3.11.0 alpha 5 :target: https://discuss.python.org/ -Copyright (c) 2001-2022 Python Software Foundation. All rights reserved. +Copyright ? 2001-2021 Python Software Foundation. All rights reserved. See the end of this file for further copyright and license information. @@ -243,14 +243,15 @@ See :pep:`664` for Python 3.11 release details. Copyright and License Information --------------------------------- -Copyright (c) 2001-2022 Python Software Foundation. All rights reserved. -Copyright (c) 2000 BeOpen.com. All rights reserved. +Copyright ? 2001-2021 Python Software Foundation. All rights reserved. -Copyright (c) 1995-2001 Corporation for National Research Initiatives. All +Copyright ? 2000 BeOpen.com. All rights reserved. + +Copyright ? 1995-2001 Corporation for National Research Initiatives. All rights reserved. -Copyright (c) 1991-1995 Stichting Mathematisch Centrum. All rights reserved. +Copyright ? 1991-1995 Stichting Mathematisch Centrum. All rights reserved. See the `LICENSE `_ for information on the history of this software, terms & conditions for usage, and a From webhook-mailer at python.org Mon Feb 14 16:49:19 2022 From: webhook-mailer at python.org (Mariatta) Date: Mon, 14 Feb 2022 21:49:19 -0000 Subject: [Python-checkins] Update the OMG link (GH-30383) Message-ID: https://github.com/python/cpython/commit/04215a427232e55efdc1adcda50a15197f518ca9 commit: 04215a427232e55efdc1adcda50a15197f518ca9 branch: main author: David Gilbertson committer: Mariatta date: 2022-02-14T13:49:11-08:00 summary: Update the OMG link (GH-30383) files: M Doc/library/xml.dom.rst diff --git a/Doc/library/xml.dom.rst b/Doc/library/xml.dom.rst index 98454e18ff1f8..b387240a3716c 100644 --- a/Doc/library/xml.dom.rst +++ b/Doc/library/xml.dom.rst @@ -73,7 +73,7 @@ implementations are free to support the strict mapping from IDL). See section `Document Object Model (DOM) Level 1 Specification `_ The W3C recommendation for the DOM supported by :mod:`xml.dom.minidom`. - `Python Language Mapping Specification `_ + `Python Language Mapping Specification `_ This specifies the mapping from OMG IDL to Python. From webhook-mailer at python.org Mon Feb 14 17:15:17 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 14 Feb 2022 22:15:17 -0000 Subject: [Python-checkins] Update the OMG link (GH-30383) Message-ID: https://github.com/python/cpython/commit/1933238983134dd405ada7a758f8ac0e6410057b commit: 1933238983134dd405ada7a758f8ac0e6410057b branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-14T14:15:08-08:00 summary: Update the OMG link (GH-30383) (cherry picked from commit 04215a427232e55efdc1adcda50a15197f518ca9) Co-authored-by: David Gilbertson files: M Doc/library/xml.dom.rst diff --git a/Doc/library/xml.dom.rst b/Doc/library/xml.dom.rst index 98454e18ff1f8..b387240a3716c 100644 --- a/Doc/library/xml.dom.rst +++ b/Doc/library/xml.dom.rst @@ -73,7 +73,7 @@ implementations are free to support the strict mapping from IDL). See section `Document Object Model (DOM) Level 1 Specification `_ The W3C recommendation for the DOM supported by :mod:`xml.dom.minidom`. - `Python Language Mapping Specification `_ + `Python Language Mapping Specification `_ This specifies the mapping from OMG IDL to Python. From webhook-mailer at python.org Mon Feb 14 18:07:14 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Mon, 14 Feb 2022 23:07:14 -0000 Subject: [Python-checkins] Regen the global objects using PYTHON_FOR_REGEN. (gh-31344) Message-ID: https://github.com/python/cpython/commit/1d9036425e811f2c6c231fbc54159f9e82f91470 commit: 1d9036425e811f2c6c231fbc54159f9e82f91470 branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-14T16:07:09-07:00 summary: Regen the global objects using PYTHON_FOR_REGEN. (gh-31344) https://bugs.python.org/issue46541 files: M Makefile.pre.in M Tools/scripts/generate_global_objects.py diff --git a/Makefile.pre.in b/Makefile.pre.in index c279cf38bd926..9b08929dd67c2 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1176,8 +1176,8 @@ regen-importlib: regen-frozen # Global objects .PHONY: regen-global-objects -regen-global-objects: $(srcdir)/Tools/scripts/generate_global_objects.py $(FREEZE_MODULE_DEPS) - $(PYTHON_FOR_FREEZE) $(srcdir)/Tools/scripts/generate_global_objects.py +regen-global-objects: $(srcdir)/Tools/scripts/generate_global_objects.py + $(PYTHON_FOR_REGEN) $(srcdir)/Tools/scripts/generate_global_objects.py ############################################################################ # ABI diff --git a/Tools/scripts/generate_global_objects.py b/Tools/scripts/generate_global_objects.py index b184e74898940..73068894d974b 100644 --- a/Tools/scripts/generate_global_objects.py +++ b/Tools/scripts/generate_global_objects.py @@ -5,7 +5,7 @@ import sys -assert os.path.isabs(__file__), __file__ +__file__ = os.path.abspath(__file__) ROOT = os.path.dirname(os.path.dirname(os.path.dirname(__file__))) INTERNAL = os.path.join(ROOT, 'Include', 'internal') From webhook-mailer at python.org Mon Feb 14 19:02:30 2022 From: webhook-mailer at python.org (corona10) Date: Tue, 15 Feb 2022 00:02:30 -0000 Subject: [Python-checkins] bpo-46728: fix docstring of combinations_with_replacement for consistency (GH-31293) Message-ID: https://github.com/python/cpython/commit/278fdd3e3a2492665b2c2888fd2f428f7f59a3f5 commit: 278fdd3e3a2492665b2c2888fd2f428f7f59a3f5 branch: main author: DongGeon Lee committer: corona10 date: 2022-02-15T09:02:21+09:00 summary: bpo-46728: fix docstring of combinations_with_replacement for consistency (GH-31293) files: M Modules/clinic/itertoolsmodule.c.h M Modules/itertoolsmodule.c diff --git a/Modules/clinic/itertoolsmodule.c.h b/Modules/clinic/itertoolsmodule.c.h index 7f5abe648af88..6a6998082484c 100644 --- a/Modules/clinic/itertoolsmodule.c.h +++ b/Modules/clinic/itertoolsmodule.c.h @@ -414,7 +414,7 @@ PyDoc_STRVAR(itertools_combinations_with_replacement__doc__, "\n" "Return successive r-length combinations of elements in the iterable allowing individual elements to have successive repeats.\n" "\n" -"combinations_with_replacement(\'ABC\', 2) --> AA AB AC BB BC CC\""); +"combinations_with_replacement(\'ABC\', 2) --> (\'A\',\'A\'), (\'A\',\'B\'), (\'A\',\'C\'), (\'B\',\'B\'), (\'B\',\'C\'), (\'C\',\'C\')"); static PyObject * itertools_combinations_with_replacement_impl(PyTypeObject *type, @@ -667,4 +667,4 @@ itertools_count(PyTypeObject *type, PyObject *args, PyObject *kwargs) exit: return return_value; } -/*[clinic end generated code: output=5364de2e143609b9 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=5cb14d4a3fe8aeec input=a9049054013a1b77]*/ diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index 0ec65d57381d8..4a7a95730395e 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -2924,14 +2924,14 @@ itertools.combinations_with_replacement.__new__ r: Py_ssize_t Return successive r-length combinations of elements in the iterable allowing individual elements to have successive repeats. -combinations_with_replacement('ABC', 2) --> AA AB AC BB BC CC" +combinations_with_replacement('ABC', 2) --> ('A','A'), ('A','B'), ('A','C'), ('B','B'), ('B','C'), ('C','C') [clinic start generated code]*/ static PyObject * itertools_combinations_with_replacement_impl(PyTypeObject *type, PyObject *iterable, Py_ssize_t r) -/*[clinic end generated code: output=48b26856d4e659ca input=dc2a8c7ba785fad7]*/ +/*[clinic end generated code: output=48b26856d4e659ca input=1dc58e82a0878fdc]*/ { cwrobject *co; Py_ssize_t n; From webhook-mailer at python.org Mon Feb 14 19:37:00 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Tue, 15 Feb 2022 00:37:00 -0000 Subject: [Python-checkins] bpo-46541: Discover the global strings. (gh-31346) Message-ID: https://github.com/python/cpython/commit/12360aa159c42c7798fd14225d271e6fd84db7eb commit: 12360aa159c42c7798fd14225d271e6fd84db7eb branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-14T17:36:51-07:00 summary: bpo-46541: Discover the global strings. (gh-31346) Instead of manually enumerating the global strings in generate_global_objects.py, we extrapolate the list from usage of _Py_ID() and _Py_STR() in the source files. This is partly inspired by gh-31261. https://bugs.python.org/issue46541 files: M Include/internal/pycore_global_strings.h M Include/internal/pycore_runtime_init.h M Objects/typeobject.c M Objects/weakrefobject.c M Python/_warnings.c M Python/ast_opt.c M Python/compile.c M Python/pythonrun.c M Tools/scripts/generate_global_objects.py diff --git a/Include/internal/pycore_global_strings.h b/Include/internal/pycore_global_strings.h index 17241b3a3dd16..aa597bc8281a5 100644 --- a/Include/internal/pycore_global_strings.h +++ b/Include/internal/pycore_global_strings.h @@ -28,13 +28,6 @@ extern "C" { /* The following is auto-generated by Tools/scripts/generate_global_objects.py. */ struct _Py_global_strings { struct { - STRUCT_FOR_STR(empty, "") - STRUCT_FOR_STR(dot, ".") - STRUCT_FOR_STR(comma_sep, ", ") - STRUCT_FOR_STR(percent, "%") - STRUCT_FOR_STR(dbl_percent, "%%") - - // "anonymous" labels STRUCT_FOR_STR(anon_dictcomp, "") STRUCT_FOR_STR(anon_genexpr, "") STRUCT_FOR_STR(anon_lambda, "") @@ -42,7 +35,12 @@ struct _Py_global_strings { STRUCT_FOR_STR(anon_module, "") STRUCT_FOR_STR(anon_setcomp, "") STRUCT_FOR_STR(anon_string, "") + STRUCT_FOR_STR(comma_sep, ", ") + STRUCT_FOR_STR(dbl_percent, "%%") + STRUCT_FOR_STR(dot, ".") STRUCT_FOR_STR(dot_locals, ".") + STRUCT_FOR_STR(empty, "") + STRUCT_FOR_STR(percent, "%") } literals; struct { @@ -330,6 +328,7 @@ struct _Py_global_strings { #define _Py_STR(NAME) \ (_Py_SINGLETON(strings.literals._ ## NAME._ascii.ob_base)) +#define _Py_DECLARE_STR(name, str) #ifdef __cplusplus } diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h index 045ae5d2835b1..04c1e671235ea 100644 --- a/Include/internal/pycore_runtime_init.h +++ b/Include/internal/pycore_runtime_init.h @@ -644,12 +644,6 @@ extern "C" { \ .strings = { \ .literals = { \ - INIT_STR(empty, ""), \ - INIT_STR(dot, "."), \ - INIT_STR(comma_sep, ", "), \ - INIT_STR(percent, "%"), \ - INIT_STR(dbl_percent, "%%"), \ - \ INIT_STR(anon_dictcomp, ""), \ INIT_STR(anon_genexpr, ""), \ INIT_STR(anon_lambda, ""), \ @@ -657,7 +651,12 @@ extern "C" { INIT_STR(anon_module, ""), \ INIT_STR(anon_setcomp, ""), \ INIT_STR(anon_string, ""), \ + INIT_STR(comma_sep, ", "), \ + INIT_STR(dbl_percent, "%%"), \ + INIT_STR(dot, "."), \ INIT_STR(dot_locals, "."), \ + INIT_STR(empty, ""), \ + INIT_STR(percent, "%"), \ }, \ .identifiers = { \ INIT_ID(Py_Repr), \ diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 3f8f36a9c4648..8c4901119de7d 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -4546,6 +4546,7 @@ object_new(PyTypeObject *type, PyObject *args, PyObject *kwds) Py_DECREF(sorted_methods); return NULL; } + _Py_DECLARE_STR(comma_sep, ", "); joined = PyUnicode_Join(&_Py_STR(comma_sep), sorted_methods); method_count = PyObject_Length(sorted_methods); Py_DECREF(sorted_methods); diff --git a/Objects/weakrefobject.c b/Objects/weakrefobject.c index 71dfa640ebf57..1712533a39d80 100644 --- a/Objects/weakrefobject.c +++ b/Objects/weakrefobject.c @@ -458,12 +458,12 @@ proxy_checkref(PyWeakReference *proxy) return res; \ } -#define WRAP_METHOD(method, special) \ +#define WRAP_METHOD(method, SPECIAL) \ static PyObject * \ method(PyObject *proxy, PyObject *Py_UNUSED(ignored)) { \ UNWRAP(proxy); \ Py_INCREF(proxy); \ - PyObject* res = PyObject_CallMethodNoArgs(proxy, &_Py_ID(special)); \ + PyObject* res = PyObject_CallMethodNoArgs(proxy, &_Py_ID(SPECIAL)); \ Py_DECREF(proxy); \ return res; \ } diff --git a/Python/_warnings.c b/Python/_warnings.c index a47e5fef6865f..03e6ffcee0ac2 100644 --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -186,8 +186,8 @@ check_matched(PyInterpreterState *interp, PyObject *obj, PyObject *arg) return rc; } -#define GET_WARNINGS_ATTR(interp, attr, try_import) \ - get_warnings_attr(interp, &_Py_ID(attr), try_import) +#define GET_WARNINGS_ATTR(interp, ATTR, try_import) \ + get_warnings_attr(interp, &_Py_ID(ATTR), try_import) /* Returns a new reference. diff --git a/Python/ast_opt.c b/Python/ast_opt.c index 2911370649790..77ed29d0cdddd 100644 --- a/Python/ast_opt.c +++ b/Python/ast_opt.c @@ -268,6 +268,8 @@ parse_literal(PyObject *fmt, Py_ssize_t *ppos, PyArena *arena) PyObject *str = PyUnicode_Substring(fmt, start, pos); /* str = str.replace('%%', '%') */ if (str && has_percents) { + _Py_DECLARE_STR(percent, "%"); + _Py_DECLARE_STR(dbl_percent, "%%"); Py_SETREF(str, PyUnicode_Replace(str, &_Py_STR(dbl_percent), &_Py_STR(percent), -1)); } diff --git a/Python/compile.c b/Python/compile.c index ac4960b5df320..1cf20d3a36ac1 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -667,6 +667,7 @@ compiler_set_qualname(struct compiler *c) || parent->u_scope_type == COMPILER_SCOPE_ASYNC_FUNCTION || parent->u_scope_type == COMPILER_SCOPE_LAMBDA) { + _Py_DECLARE_STR(dot_locals, "."); base = PyUnicode_Concat(parent->u_qualname, &_Py_STR(dot_locals)); if (base == NULL) @@ -2022,6 +2023,7 @@ compiler_mod(struct compiler *c, mod_ty mod) { PyCodeObject *co; int addNone = 1; + _Py_DECLARE_STR(anon_module, ""); if (!compiler_enter_scope(c, &_Py_STR(anon_module), COMPILER_SCOPE_MODULE, mod, 1)) { return NULL; @@ -2876,6 +2878,7 @@ compiler_lambda(struct compiler *c, expr_ty e) return 0; } + _Py_DECLARE_STR(anon_lambda, ""); if (!compiler_enter_scope(c, &_Py_STR(anon_lambda), COMPILER_SCOPE_LAMBDA, (void *)e, e->lineno)) { return 0; @@ -5347,6 +5350,7 @@ static int compiler_genexp(struct compiler *c, expr_ty e) { assert(e->kind == GeneratorExp_kind); + _Py_DECLARE_STR(anon_genexpr, ""); return compiler_comprehension(c, e, COMP_GENEXP, &_Py_STR(anon_genexpr), e->v.GeneratorExp.generators, e->v.GeneratorExp.elt, NULL); @@ -5356,6 +5360,7 @@ static int compiler_listcomp(struct compiler *c, expr_ty e) { assert(e->kind == ListComp_kind); + _Py_DECLARE_STR(anon_listcomp, ""); return compiler_comprehension(c, e, COMP_LISTCOMP, &_Py_STR(anon_listcomp), e->v.ListComp.generators, e->v.ListComp.elt, NULL); @@ -5365,6 +5370,7 @@ static int compiler_setcomp(struct compiler *c, expr_ty e) { assert(e->kind == SetComp_kind); + _Py_DECLARE_STR(anon_setcomp, ""); return compiler_comprehension(c, e, COMP_SETCOMP, &_Py_STR(anon_setcomp), e->v.SetComp.generators, e->v.SetComp.elt, NULL); @@ -5375,6 +5381,7 @@ static int compiler_dictcomp(struct compiler *c, expr_ty e) { assert(e->kind == DictComp_kind); + _Py_DECLARE_STR(anon_dictcomp, ""); return compiler_comprehension(c, e, COMP_DICTCOMP, &_Py_STR(anon_dictcomp), e->v.DictComp.generators, e->v.DictComp.key, e->v.DictComp.value); diff --git a/Python/pythonrun.c b/Python/pythonrun.c index b34a22391822b..38ca952838a1f 100644 --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -515,6 +515,7 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename, goto finally; if (v == Py_None) { Py_DECREF(v); + _Py_DECLARE_STR(anon_string, ""); *filename = &_Py_STR(anon_string); Py_INCREF(*filename); } @@ -1562,6 +1563,7 @@ PyRun_StringFlags(const char *str, int start, PyObject *globals, if (arena == NULL) return NULL; + _Py_DECLARE_STR(anon_string, ""); mod = _PyParser_ASTFromString( str, &_Py_STR(anon_string), start, flags, arena); diff --git a/Tools/scripts/generate_global_objects.py b/Tools/scripts/generate_global_objects.py index 73068894d974b..e989f3c086f34 100644 --- a/Tools/scripts/generate_global_objects.py +++ b/Tools/scripts/generate_global_objects.py @@ -13,298 +13,112 @@ STRING_LITERALS = { 'empty': '', 'dot': '.', - 'comma_sep': ', ', - 'percent': '%', - 'dbl_percent': '%%', - - '"anonymous" labels': None, - 'anon_dictcomp': '', - 'anon_genexpr': '', - 'anon_lambda': '', - 'anon_listcomp': '', - 'anon_module': '', - 'anon_setcomp': '', - 'anon_string': '', - 'dot_locals': '.', +} +IGNORED = { + 'ACTION', # Python/_warnings.c + 'ATTR', # Python/_warnings.c and Objects/funcobject.c + 'DUNDER', # Objects/typeobject.c + 'RDUNDER', # Objects/typeobject.c + 'SPECIAL', # Objects/weakrefobject.c } IDENTIFIERS = [ - 'Py_Repr', - 'TextIOWrapper', + # from ADD() Python/_warnings.c + 'default', + 'ignore', + + # from GET_WARNINGS_ATTR() in Python/_warnings.c 'WarningMessage', - '_', - '__IOBase_closed', - '__abc_tpflags__', + '_showwarnmsg', + '_warn_unawaited_coroutine', + 'defaultaction', + 'filters', + 'onceregistry', + + # from WRAP_METHOD() in Objects/weakrefobject.c + '__bytes__', + '__reversed__', + + # from COPY_ATTR() in Objects/funcobject.c + '__module__', + '__name__', + '__qualname__', + '__doc__', + '__annotations__', + + # from SLOT* in Objects/typeobject.c '__abs__', - '__abstractmethods__', '__add__', - '__aenter__', - '__aexit__', - '__aiter__', - '__all__', '__and__', - '__anext__', - '__annotations__', - '__args__', - '__await__', - '__bases__', - '__bool__', - '__build_class__', - '__builtins__', - '__bytes__', - '__call__', - '__cantrace__', - '__class__', - '__class_getitem__', - '__classcell__', - '__complex__', - '__contains__', - '__copy__', - '__del__', - '__delattr__', - '__delete__', - '__delitem__', - '__dict__', - '__dir__', '__divmod__', - '__doc__', - '__enter__', - '__eq__', - '__exit__', - '__file__', '__float__', '__floordiv__', - '__format__', - '__fspath__', - '__ge__', - '__get__', - '__getattr__', - '__getattribute__', - '__getinitargs__', '__getitem__', - '__getnewargs__', - '__getnewargs_ex__', - '__getstate__', - '__gt__', - '__hash__', '__iadd__', '__iand__', '__ifloordiv__', '__ilshift__', '__imatmul__', '__imod__', - '__import__', '__imul__', - '__index__', - '__init__', - '__init_subclass__', - '__instancecheck__', '__int__', '__invert__', '__ior__', - '__ipow__', '__irshift__', - '__isabstractmethod__', '__isub__', - '__iter__', '__itruediv__', '__ixor__', - '__le__', - '__len__', - '__length_hint__', - '__loader__', '__lshift__', - '__lt__', - '__ltrace__', - '__main__', '__matmul__', - '__missing__', '__mod__', - '__module__', - '__mro_entries__', '__mul__', - '__name__', - '__ne__', '__neg__', - '__new__', - '__newobj__', - '__newobj_ex__', - '__next__', - '__note__', '__or__', - '__origin__', - '__package__', - '__parameters__', - '__path__', '__pos__', '__pow__', - '__prepare__', - '__qualname__', '__radd__', '__rand__', '__rdivmod__', - '__reduce__', - '__reduce_ex__', - '__repr__', - '__reversed__', '__rfloordiv__', '__rlshift__', '__rmatmul__', '__rmod__', '__rmul__', '__ror__', - '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', - '__set__', - '__set_name__', - '__setattr__', - '__setitem__', - '__setstate__', - '__sizeof__', - '__slotnames__', - '__slots__', - '__spec__', '__str__', '__sub__', - '__subclasscheck__', - '__subclasshook__', '__truediv__', - '__trunc__', - '__warningregistry__', - '__weakref__', '__xor__', - '_abc_impl', - '_blksize', - '_dealloc_warn', - '_finalizing', - '_find_and_load', - '_fix_up_module', - '_get_sourcefile', - '_handle_fromlist', - '_initializing', - '_is_text_encoding', - '_lock_unlock_module', - '_showwarnmsg', - '_shutdown', - '_slotnames', - '_strptime_time', - '_uninitialized_submodules', - '_warn_unawaited_coroutine', - '_xoptions', - 'add', - 'append', - 'big', - 'buffer', - 'builtins', - 'clear', - 'close', - 'code', - 'copy', - 'copyreg', - 'decode', - 'default', - 'defaultaction', - 'difference_update', - 'dispatch_table', - 'displayhook', - 'enable', - 'encoding', - 'end_lineno', - 'end_offset', - 'errors', - 'excepthook', - 'extend', - 'filename', - 'fileno', - 'fillvalue', - 'filters', - 'find_class', - 'flush', - 'get', - 'get_source', - 'getattr', - 'ignore', - 'importlib', - 'intersection', - 'isatty', - 'items', - 'iter', - 'keys', - 'last_traceback', - 'last_type', - 'last_value', - 'latin1', - 'lineno', - 'little', - 'match', - 'metaclass', - 'mode', - 'modules', - 'mro', - 'msg', - 'n_fields', - 'n_sequence_fields', - 'n_unnamed_fields', - 'name', - 'obj', - 'offset', - 'onceregistry', - 'open', - 'parent', - 'partial', - 'path', - 'peek', - 'persistent_id', - 'persistent_load', - 'print_file_and_line', - 'ps1', - 'ps2', - 'raw', - 'read', - 'read1', - 'readable', - 'readall', - 'readinto', - 'readinto1', - 'readline', - 'reducer_override', - 'reload', - 'replace', - 'reset', - 'return', - 'reversed', - 'seek', - 'seekable', - 'send', - 'setstate', - 'sort', - 'stderr', - 'stdin', - 'stdout', - 'strict', - 'symmetric_difference_update', - 'tell', - 'text', - 'threading', - 'throw', - 'unraisablehook', - 'values', - 'version', - 'warnings', - 'warnoptions', - 'writable', - 'write', - 'zipimporter', ] ####################################### # helpers +def iter_global_strings(): + id_regex = re.compile(r'\b_Py_ID\((\w+)\)') + str_regex = re.compile(r'\b_Py_DECLARE_STR\((\w+), "(.*?)"\)') + for dirname, _, files in os.walk(ROOT): + if os.path.relpath(dirname, ROOT).startswith('Include'): + continue + for name in files: + if not name.endswith(('.c', '.h')): + continue + filename = os.path.join(dirname, name) + with open(os.path.join(filename), encoding='utf-8') as infile: + for lno, line in enumerate(infile, 1): + for m in id_regex.finditer(line): + identifier, = m.groups() + yield identifier, None, filename, lno, line + for m in str_regex.finditer(line): + varname, string = m.groups() + yield varname, string, filename, lno, line + def iter_to_marker(lines, marker): for line in lines: if line.rstrip() == marker: @@ -354,7 +168,7 @@ def block(self, prefix, suffix="", *, continuation=None): END = '/* End auto-generated code */' -def generate_global_strings(): +def generate_global_strings(identifiers, strings): filename = os.path.join(INTERNAL, 'pycore_global_strings.h') # Read the non-generated part of the file. @@ -371,22 +185,18 @@ def generate_global_strings(): printer.write(START) with printer.block('struct _Py_global_strings', ';'): with printer.block('struct', ' literals;'): - for name, literal in STRING_LITERALS.items(): - if literal is None: - outfile.write('\n') - printer.write(f'// {name}') - else: - printer.write(f'STRUCT_FOR_STR({name}, "{literal}")') + for name, literal in sorted(strings.items()): + printer.write(f'STRUCT_FOR_STR({name}, "{literal}")') outfile.write('\n') with printer.block('struct', ' identifiers;'): - for name in sorted(IDENTIFIERS): + for name in sorted(identifiers): assert name.isidentifier(), name printer.write(f'STRUCT_FOR_ID({name})') printer.write(END) printer.write(after) -def generate_runtime_init(): +def generate_runtime_init(identifiers, strings): # First get some info from the declarations. nsmallposints = None nsmallnegints = None @@ -432,13 +242,10 @@ def generate_runtime_init(): # Global strings. with printer.block('.strings =', ','): with printer.block('.literals =', ','): - for name, literal in STRING_LITERALS.items(): - if literal is None: - printer.write('') - else: - printer.write(f'INIT_STR({name}, "{literal}"),') + for name, literal in sorted(strings.items()): + printer.write(f'INIT_STR({name}, "{literal}"),') with printer.block('.identifiers =', ','): - for name in sorted(IDENTIFIERS): + for name in sorted(identifiers): assert name.isidentifier(), name printer.write(f'INIT_ID({name}),') printer.write(END) @@ -507,9 +314,9 @@ def err(msg): ) ''', re.VERBOSE) -def check_orphan_strings(): +def check_orphan_strings(identifiers): literals = set(n for n, s in STRING_LITERALS.items() if s) - identifiers = set(IDENTIFIERS) + identifiers = set(identifiers) files = glob.iglob(os.path.join(ROOT, '**', '*.[ch]'), recursive=True) for i, filename in enumerate(files, start=1): print('.', end='') @@ -586,11 +393,23 @@ def check_orphan_strings(): # the script def main(*, check=False) -> None: - generate_global_strings() - generate_runtime_init() + identifiers = set(IDENTIFIERS) + strings = dict(STRING_LITERALS) + for name, string, filename, lno, _ in iter_global_strings(): + if string is None: + if name not in IGNORED: + identifiers.add(name) + else: + if name not in strings: + strings[name] = string + elif string != strings[name]: + raise ValueError(f'string mismatch for {name!r} ({string!r} != {strings[name]!r}') + + generate_global_strings(identifiers, strings) + generate_runtime_init(identifiers, strings) if check: - check_orphan_strings() + check_orphan_strings(identifiers) if __name__ == '__main__': From webhook-mailer at python.org Tue Feb 15 04:35:29 2022 From: webhook-mailer at python.org (markshannon) Date: Tue, 15 Feb 2022 09:35:29 -0000 Subject: [Python-checkins] bpo-46724: Use `JUMP_ABSOLUTE` for all backward jumps. (GH-31326) Message-ID: https://github.com/python/cpython/commit/3be1a443ca8e7d4ba85f95b78df5c4122cae9ede commit: 3be1a443ca8e7d4ba85f95b78df5c4122cae9ede branch: main author: Mark Shannon committer: markshannon date: 2022-02-15T09:35:16Z summary: bpo-46724: Use `JUMP_ABSOLUTE` for all backward jumps. (GH-31326) * Make sure all backward jumps use JUMP_ABSOLUTE. * Add news. * Fix up news item. * Make test use consistent style. files: A Misc/NEWS.d/next/Core and Builtins/2022-02-14-14-44-06.bpo-46724.jym_K6.rst M Lib/test/test_compile.py M Python/compile.c diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py index 096501513c14b..7ebe837fce1b5 100644 --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -1008,6 +1008,16 @@ def if_else_break(): elif instr.opname in HANDLED_JUMPS: self.assertNotEqual(instr.arg, (line + 1)*INSTR_SIZE) + def test_no_wraparound_jump(self): + # See https://bugs.python.org/issue46724 + + def while_not_chained(a, b, c): + while not (a < b < c): + pass + + for instr in dis.Bytecode(while_not_chained): + self.assertNotEqual(instr.opname, "EXTENDED_ARG") + @requires_debug_ranges() class TestSourcePositions(unittest.TestCase): # Ensure that compiled code snippets have correct line and column numbers diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-14-14-44-06.bpo-46724.jym_K6.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-14-14-44-06.bpo-46724.jym_K6.rst new file mode 100644 index 0000000000000..7324182677abf --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-14-14-44-06.bpo-46724.jym_K6.rst @@ -0,0 +1,2 @@ +Make sure that all backwards jumps use the ``JUMP_ABSOLUTE`` instruction, rather +than ``JUMP_FORWARD`` with an argument of ``(2**32)+offset``. diff --git a/Python/compile.c b/Python/compile.c index 1cf20d3a36ac1..786ef4e8ad3e2 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -7534,6 +7534,11 @@ normalize_jumps(struct assembler *a) last->i_opcode = JUMP_FORWARD; } } + if (last->i_opcode == JUMP_FORWARD) { + if (last->i_target->b_visited == 1) { + last->i_opcode = JUMP_ABSOLUTE; + } + } } } From webhook-mailer at python.org Tue Feb 15 05:24:13 2022 From: webhook-mailer at python.org (corona10) Date: Tue, 15 Feb 2022 10:24:13 -0000 Subject: [Python-checkins] [3.10] bpo-46728: fix docstring of combinations_with_replacement for consistency (GH-31293) (GH-31350) Message-ID: https://github.com/python/cpython/commit/ac55cea3cbee4d40cf7320bd691f5327cff30f13 commit: ac55cea3cbee4d40cf7320bd691f5327cff30f13 branch: 3.10 author: DongGeon Lee committer: corona10 date: 2022-02-15T19:24:04+09:00 summary: [3.10] bpo-46728: fix docstring of combinations_with_replacement for consistency (GH-31293) (GH-31350) files: M Modules/clinic/itertoolsmodule.c.h M Modules/itertoolsmodule.c diff --git a/Modules/clinic/itertoolsmodule.c.h b/Modules/clinic/itertoolsmodule.c.h index 82729eeb56bce..d3f54221dd4cf 100644 --- a/Modules/clinic/itertoolsmodule.c.h +++ b/Modules/clinic/itertoolsmodule.c.h @@ -406,7 +406,7 @@ PyDoc_STRVAR(itertools_combinations_with_replacement__doc__, "\n" "Return successive r-length combinations of elements in the iterable allowing individual elements to have successive repeats.\n" "\n" -"combinations_with_replacement(\'ABC\', 2) --> AA AB AC BB BC CC\""); +"combinations_with_replacement(\'ABC\', 2) --> (\'A\',\'A\'), (\'A\',\'B\'), (\'A\',\'C\'), (\'B\',\'B\'), (\'B\',\'C\'), (\'C\',\'C\')"); static PyObject * itertools_combinations_with_replacement_impl(PyTypeObject *type, @@ -658,4 +658,4 @@ itertools_count(PyTypeObject *type, PyObject *args, PyObject *kwargs) exit: return return_value; } -/*[clinic end generated code: output=889c4afc3b13574f input=a9049054013a1b77]*/ +/*[clinic end generated code: output=c91f57481a2461d3 input=a9049054013a1b77]*/ diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index 643f47b045046..f8e2c45aecadf 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -2926,14 +2926,14 @@ itertools.combinations_with_replacement.__new__ r: Py_ssize_t Return successive r-length combinations of elements in the iterable allowing individual elements to have successive repeats. -combinations_with_replacement('ABC', 2) --> AA AB AC BB BC CC" +combinations_with_replacement('ABC', 2) --> ('A','A'), ('A','B'), ('A','C'), ('B','B'), ('B','C'), ('C','C') [clinic start generated code]*/ static PyObject * itertools_combinations_with_replacement_impl(PyTypeObject *type, PyObject *iterable, Py_ssize_t r) -/*[clinic end generated code: output=48b26856d4e659ca input=dc2a8c7ba785fad7]*/ +/*[clinic end generated code: output=48b26856d4e659ca input=1dc58e82a0878fdc]*/ { cwrobject *co; Py_ssize_t n; From webhook-mailer at python.org Tue Feb 15 07:22:13 2022 From: webhook-mailer at python.org (corona10) Date: Tue, 15 Feb 2022 12:22:13 -0000 Subject: [Python-checkins] [3.9] bpo-46728: fix docstring of combinations_with_replacement for consistency (GH-31293). (GH-31356) Message-ID: https://github.com/python/cpython/commit/ad47db34be23c5cd75ed5d5c220d4b56a9b5683c commit: ad47db34be23c5cd75ed5d5c220d4b56a9b5683c branch: 3.9 author: DongGeon Lee committer: corona10 date: 2022-02-15T21:22:01+09:00 summary: [3.9] bpo-46728: fix docstring of combinations_with_replacement for consistency (GH-31293). (GH-31356) files: M Modules/clinic/itertoolsmodule.c.h M Modules/itertoolsmodule.c diff --git a/Modules/clinic/itertoolsmodule.c.h b/Modules/clinic/itertoolsmodule.c.h index 20594b0fed4c3..251b67b17ea44 100644 --- a/Modules/clinic/itertoolsmodule.c.h +++ b/Modules/clinic/itertoolsmodule.c.h @@ -385,7 +385,7 @@ PyDoc_STRVAR(itertools_combinations_with_replacement__doc__, "\n" "Return successive r-length combinations of elements in the iterable allowing individual elements to have successive repeats.\n" "\n" -"combinations_with_replacement(\'ABC\', 2) --> AA AB AC BB BC CC\""); +"combinations_with_replacement(\'ABC\', 2) --> (\'A\',\'A\'), (\'A\',\'B\'), (\'A\',\'C\'), (\'B\',\'B\'), (\'B\',\'C\'), (\'C\',\'C\')"); static PyObject * itertools_combinations_with_replacement_impl(PyTypeObject *type, @@ -642,4 +642,4 @@ itertools_count(PyTypeObject *type, PyObject *args, PyObject *kwargs) exit: return return_value; } -/*[clinic end generated code: output=392c9706e79f6710 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=0c2799c88400b63f input=a9049054013a1b77]*/ diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index bc5382af7e98e..fae5560ab7145 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -2793,14 +2793,14 @@ itertools.combinations_with_replacement.__new__ r: Py_ssize_t Return successive r-length combinations of elements in the iterable allowing individual elements to have successive repeats. -combinations_with_replacement('ABC', 2) --> AA AB AC BB BC CC" +combinations_with_replacement('ABC', 2) --> ('A','A'), ('A','B'), ('A','C'), ('B','B'), ('B','C'), ('C','C') [clinic start generated code]*/ static PyObject * itertools_combinations_with_replacement_impl(PyTypeObject *type, PyObject *iterable, Py_ssize_t r) -/*[clinic end generated code: output=48b26856d4e659ca input=dc2a8c7ba785fad7]*/ +/*[clinic end generated code: output=48b26856d4e659ca input=1dc58e82a0878fdc]*/ { cwrobject *co; Py_ssize_t n; From webhook-mailer at python.org Tue Feb 15 08:04:10 2022 From: webhook-mailer at python.org (asvetlov) Date: Tue, 15 Feb 2022 13:04:10 -0000 Subject: [Python-checkins] bpo-44011: New asyncio ssl implementation (#31275) Message-ID: https://github.com/python/cpython/commit/13c10bfb777483c7b02877aab029345a056b809c commit: 13c10bfb777483c7b02877aab029345a056b809c branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: asvetlov date: 2022-02-15T15:04:00+02:00 summary: bpo-44011: New asyncio ssl implementation (#31275) * bpo-44011: New asyncio ssl implementation Co-Authored-By: Andrew Svetlov * fix warning * fix typo Co-authored-by: Andrew Svetlov files: A Lib/test/test_asyncio/test_ssl.py A Misc/NEWS.d/next/Library/2021-05-02-23-44-21.bpo-44011.hd8iUO.rst M Lib/asyncio/base_events.py M Lib/asyncio/constants.py M Lib/asyncio/events.py M Lib/asyncio/proactor_events.py M Lib/asyncio/selector_events.py M Lib/asyncio/sslproto.py M Lib/asyncio/unix_events.py M Lib/test/test_asyncio/test_base_events.py M Lib/test/test_asyncio/test_selector_events.py M Lib/test/test_asyncio/test_sslproto.py diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index 56ea7ba44e2ea..703c8a4ce2408 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -269,7 +269,7 @@ async def restore(self): class Server(events.AbstractServer): def __init__(self, loop, sockets, protocol_factory, ssl_context, backlog, - ssl_handshake_timeout): + ssl_handshake_timeout, ssl_shutdown_timeout=None): self._loop = loop self._sockets = sockets self._active_count = 0 @@ -278,6 +278,7 @@ def __init__(self, loop, sockets, protocol_factory, ssl_context, backlog, self._backlog = backlog self._ssl_context = ssl_context self._ssl_handshake_timeout = ssl_handshake_timeout + self._ssl_shutdown_timeout = ssl_shutdown_timeout self._serving = False self._serving_forever_fut = None @@ -309,7 +310,8 @@ def _start_serving(self): sock.listen(self._backlog) self._loop._start_serving( self._protocol_factory, sock, self._ssl_context, - self, self._backlog, self._ssl_handshake_timeout) + self, self._backlog, self._ssl_handshake_timeout, + self._ssl_shutdown_timeout) def get_loop(self): return self._loop @@ -463,6 +465,7 @@ def _make_ssl_transport( *, server_side=False, server_hostname=None, extra=None, server=None, ssl_handshake_timeout=None, + ssl_shutdown_timeout=None, call_connection_made=True): """Create SSL transport.""" raise NotImplementedError @@ -965,6 +968,7 @@ async def create_connection( proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None, + ssl_shutdown_timeout=None, happy_eyeballs_delay=None, interleave=None): """Connect to a TCP server. @@ -1000,6 +1004,10 @@ async def create_connection( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if ssl_shutdown_timeout is not None and not ssl: + raise ValueError( + 'ssl_shutdown_timeout is only meaningful with ssl') + if happy_eyeballs_delay is not None and interleave is None: # If using happy eyeballs, default to interleave addresses by family interleave = 1 @@ -1075,7 +1083,8 @@ async def create_connection( transport, protocol = await self._create_connection_transport( sock, protocol_factory, ssl, server_hostname, - ssl_handshake_timeout=ssl_handshake_timeout) + ssl_handshake_timeout=ssl_handshake_timeout, + ssl_shutdown_timeout=ssl_shutdown_timeout) if self._debug: # Get the socket from the transport because SSL transport closes # the old socket and creates a new SSL socket @@ -1087,7 +1096,8 @@ async def create_connection( async def _create_connection_transport( self, sock, protocol_factory, ssl, server_hostname, server_side=False, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): sock.setblocking(False) @@ -1098,7 +1108,8 @@ async def _create_connection_transport( transport = self._make_ssl_transport( sock, protocol, sslcontext, waiter, server_side=server_side, server_hostname=server_hostname, - ssl_handshake_timeout=ssl_handshake_timeout) + ssl_handshake_timeout=ssl_handshake_timeout, + ssl_shutdown_timeout=ssl_shutdown_timeout) else: transport = self._make_socket_transport(sock, protocol, waiter) @@ -1189,7 +1200,8 @@ async def _sendfile_fallback(self, transp, file, offset, count): async def start_tls(self, transport, protocol, sslcontext, *, server_side=False, server_hostname=None, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): """Upgrade transport to TLS. Return a new transport that *protocol* should start using @@ -1212,6 +1224,7 @@ async def start_tls(self, transport, protocol, sslcontext, *, self, protocol, sslcontext, waiter, server_side, server_hostname, ssl_handshake_timeout=ssl_handshake_timeout, + ssl_shutdown_timeout=ssl_shutdown_timeout, call_connection_made=False) # Pause early so that "ssl_protocol.data_received()" doesn't @@ -1397,6 +1410,7 @@ async def create_server( reuse_address=None, reuse_port=None, ssl_handshake_timeout=None, + ssl_shutdown_timeout=None, start_serving=True): """Create a TCP server. @@ -1420,6 +1434,10 @@ async def create_server( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if ssl_shutdown_timeout is not None and ssl is None: + raise ValueError( + 'ssl_shutdown_timeout is only meaningful with ssl') + if host is not None or port is not None: if sock is not None: raise ValueError( @@ -1492,7 +1510,8 @@ async def create_server( sock.setblocking(False) server = Server(self, sockets, protocol_factory, - ssl, backlog, ssl_handshake_timeout) + ssl, backlog, ssl_handshake_timeout, + ssl_shutdown_timeout) if start_serving: server._start_serving() # Skip one loop iteration so that all 'loop.add_reader' @@ -1506,7 +1525,8 @@ async def create_server( async def connect_accepted_socket( self, protocol_factory, sock, *, ssl=None, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): if sock.type != socket.SOCK_STREAM: raise ValueError(f'A Stream Socket was expected, got {sock!r}') @@ -1514,9 +1534,14 @@ async def connect_accepted_socket( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if ssl_shutdown_timeout is not None and not ssl: + raise ValueError( + 'ssl_shutdown_timeout is only meaningful with ssl') + transport, protocol = await self._create_connection_transport( sock, protocol_factory, ssl, '', server_side=True, - ssl_handshake_timeout=ssl_handshake_timeout) + ssl_handshake_timeout=ssl_handshake_timeout, + ssl_shutdown_timeout=ssl_shutdown_timeout) if self._debug: # Get the socket from the transport because SSL transport closes # the old socket and creates a new SSL socket diff --git a/Lib/asyncio/constants.py b/Lib/asyncio/constants.py index 33feed60e55b0..f171ead28fecd 100644 --- a/Lib/asyncio/constants.py +++ b/Lib/asyncio/constants.py @@ -15,10 +15,17 @@ # The default timeout matches that of Nginx. SSL_HANDSHAKE_TIMEOUT = 60.0 +# Number of seconds to wait for SSL shutdown to complete +# The default timeout mimics lingering_time +SSL_SHUTDOWN_TIMEOUT = 30.0 + # Used in sendfile fallback code. We use fallback for platforms # that don't support sendfile, or for TLS connections. SENDFILE_FALLBACK_READBUFFER_SIZE = 1024 * 256 +FLOW_CONTROL_HIGH_WATER_SSL_READ = 256 # KiB +FLOW_CONTROL_HIGH_WATER_SSL_WRITE = 512 # KiB + # The enum should be here to break circular dependencies between # base_events and sslproto class _SendfileMode(enum.Enum): diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py index e3c55b22aace2..1d305e3ddff1c 100644 --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -303,6 +303,7 @@ async def create_connection( flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None, + ssl_shutdown_timeout=None, happy_eyeballs_delay=None, interleave=None): raise NotImplementedError @@ -312,6 +313,7 @@ async def create_server( flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, ssl_handshake_timeout=None, + ssl_shutdown_timeout=None, start_serving=True): """A coroutine which creates a TCP server bound to host and port. @@ -352,6 +354,10 @@ async def create_server( will wait for completion of the SSL handshake before aborting the connection. Default is 60s. + ssl_shutdown_timeout is the time in seconds that an SSL server + will wait for completion of the SSL shutdown procedure + before aborting the connection. Default is 30s. + start_serving set to True (default) causes the created server to start accepting connections immediately. When set to False, the user should await Server.start_serving() or Server.serve_forever() @@ -370,7 +376,8 @@ async def sendfile(self, transport, file, offset=0, count=None, async def start_tls(self, transport, protocol, sslcontext, *, server_side=False, server_hostname=None, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): """Upgrade a transport to TLS. Return a new transport that *protocol* should start using @@ -382,13 +389,15 @@ async def create_unix_connection( self, protocol_factory, path=None, *, ssl=None, sock=None, server_hostname=None, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): raise NotImplementedError async def create_unix_server( self, protocol_factory, path=None, *, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, + ssl_shutdown_timeout=None, start_serving=True): """A coroutine which creates a UNIX Domain Socket server. @@ -410,6 +419,9 @@ async def create_unix_server( ssl_handshake_timeout is the time in seconds that an SSL server will wait for the SSL handshake to complete (defaults to 60s). + ssl_shutdown_timeout is the time in seconds that an SSL server + will wait for the SSL shutdown to finish (defaults to 30s). + start_serving set to True (default) causes the created server to start accepting connections immediately. When set to False, the user should await Server.start_serving() or Server.serve_forever() @@ -420,7 +432,8 @@ async def create_unix_server( async def connect_accepted_socket( self, protocol_factory, sock, *, ssl=None, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): """Handle an accepted connection. This is used by servers that accept connections outside of diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py index 1d9e2fe2ca728..ae59f30db1c3c 100644 --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -642,11 +642,13 @@ def _make_ssl_transport( self, rawsock, protocol, sslcontext, waiter=None, *, server_side=False, server_hostname=None, extra=None, server=None, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): ssl_protocol = sslproto.SSLProtocol( self, protocol, sslcontext, waiter, server_side, server_hostname, - ssl_handshake_timeout=ssl_handshake_timeout) + ssl_handshake_timeout=ssl_handshake_timeout, + ssl_shutdown_timeout=ssl_shutdown_timeout) _ProactorSocketTransport(self, rawsock, ssl_protocol, extra=extra, server=server) return ssl_protocol._app_transport @@ -812,7 +814,8 @@ def _write_to_self(self): def _start_serving(self, protocol_factory, sock, sslcontext=None, server=None, backlog=100, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): def loop(f=None): try: @@ -826,7 +829,8 @@ def loop(f=None): self._make_ssl_transport( conn, protocol, sslcontext, server_side=True, extra={'peername': addr}, server=server, - ssl_handshake_timeout=ssl_handshake_timeout) + ssl_handshake_timeout=ssl_handshake_timeout, + ssl_shutdown_timeout=ssl_shutdown_timeout) else: self._make_socket_transport( conn, protocol, diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index 59cb6b1babec5..63ab15f30fb5d 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -70,11 +70,15 @@ def _make_ssl_transport( self, rawsock, protocol, sslcontext, waiter=None, *, server_side=False, server_hostname=None, extra=None, server=None, - ssl_handshake_timeout=constants.SSL_HANDSHAKE_TIMEOUT): + ssl_handshake_timeout=constants.SSL_HANDSHAKE_TIMEOUT, + ssl_shutdown_timeout=constants.SSL_SHUTDOWN_TIMEOUT, + ): ssl_protocol = sslproto.SSLProtocol( - self, protocol, sslcontext, waiter, - server_side, server_hostname, - ssl_handshake_timeout=ssl_handshake_timeout) + self, protocol, sslcontext, waiter, + server_side, server_hostname, + ssl_handshake_timeout=ssl_handshake_timeout, + ssl_shutdown_timeout=ssl_shutdown_timeout + ) _SelectorSocketTransport(self, rawsock, ssl_protocol, extra=extra, server=server) return ssl_protocol._app_transport @@ -146,15 +150,17 @@ def _write_to_self(self): def _start_serving(self, protocol_factory, sock, sslcontext=None, server=None, backlog=100, - ssl_handshake_timeout=constants.SSL_HANDSHAKE_TIMEOUT): + ssl_handshake_timeout=constants.SSL_HANDSHAKE_TIMEOUT, + ssl_shutdown_timeout=constants.SSL_SHUTDOWN_TIMEOUT): self._add_reader(sock.fileno(), self._accept_connection, protocol_factory, sock, sslcontext, server, backlog, - ssl_handshake_timeout) + ssl_handshake_timeout, ssl_shutdown_timeout) def _accept_connection( self, protocol_factory, sock, sslcontext=None, server=None, backlog=100, - ssl_handshake_timeout=constants.SSL_HANDSHAKE_TIMEOUT): + ssl_handshake_timeout=constants.SSL_HANDSHAKE_TIMEOUT, + ssl_shutdown_timeout=constants.SSL_SHUTDOWN_TIMEOUT): # This method is only called once for each event loop tick where the # listening socket has triggered an EVENT_READ. There may be multiple # connections waiting for an .accept() so it is called in a loop. @@ -185,20 +191,22 @@ def _accept_connection( self.call_later(constants.ACCEPT_RETRY_DELAY, self._start_serving, protocol_factory, sock, sslcontext, server, - backlog, ssl_handshake_timeout) + backlog, ssl_handshake_timeout, + ssl_shutdown_timeout) else: raise # The event loop will catch, log and ignore it. else: extra = {'peername': addr} accept = self._accept_connection2( protocol_factory, conn, extra, sslcontext, server, - ssl_handshake_timeout) + ssl_handshake_timeout, ssl_shutdown_timeout) self.create_task(accept) async def _accept_connection2( self, protocol_factory, conn, extra, sslcontext=None, server=None, - ssl_handshake_timeout=constants.SSL_HANDSHAKE_TIMEOUT): + ssl_handshake_timeout=constants.SSL_HANDSHAKE_TIMEOUT, + ssl_shutdown_timeout=constants.SSL_SHUTDOWN_TIMEOUT): protocol = None transport = None try: @@ -208,7 +216,8 @@ async def _accept_connection2( transport = self._make_ssl_transport( conn, protocol, sslcontext, waiter=waiter, server_side=True, extra=extra, server=server, - ssl_handshake_timeout=ssl_handshake_timeout) + ssl_handshake_timeout=ssl_handshake_timeout, + ssl_shutdown_timeout=ssl_shutdown_timeout) else: transport = self._make_socket_transport( conn, protocol, waiter=waiter, extra=extra, diff --git a/Lib/asyncio/sslproto.py b/Lib/asyncio/sslproto.py index 00fc16c014c9b..de7c3332e4dca 100644 --- a/Lib/asyncio/sslproto.py +++ b/Lib/asyncio/sslproto.py @@ -1,4 +1,5 @@ import collections +import enum import warnings try: import ssl @@ -6,10 +7,38 @@ ssl = None from . import constants +from . import exceptions from . import protocols from . import transports from .log import logger +if ssl is not None: + SSLAgainErrors = (ssl.SSLWantReadError, ssl.SSLSyscallError) + + +class SSLProtocolState(enum.Enum): + UNWRAPPED = "UNWRAPPED" + DO_HANDSHAKE = "DO_HANDSHAKE" + WRAPPED = "WRAPPED" + FLUSHING = "FLUSHING" + SHUTDOWN = "SHUTDOWN" + + +class AppProtocolState(enum.Enum): + # This tracks the state of app protocol (https://git.io/fj59P): + # + # INIT -cm-> CON_MADE [-dr*->] [-er-> EOF?] -cl-> CON_LOST + # + # * cm: connection_made() + # * dr: data_received() + # * er: eof_received() + # * cl: connection_lost() + + STATE_INIT = "STATE_INIT" + STATE_CON_MADE = "STATE_CON_MADE" + STATE_EOF = "STATE_EOF" + STATE_CON_LOST = "STATE_CON_LOST" + def _create_transport_context(server_side, server_hostname): if server_side: @@ -25,269 +54,35 @@ def _create_transport_context(server_side, server_hostname): return sslcontext -# States of an _SSLPipe. -_UNWRAPPED = "UNWRAPPED" -_DO_HANDSHAKE = "DO_HANDSHAKE" -_WRAPPED = "WRAPPED" -_SHUTDOWN = "SHUTDOWN" - - -class _SSLPipe(object): - """An SSL "Pipe". - - An SSL pipe allows you to communicate with an SSL/TLS protocol instance - through memory buffers. It can be used to implement a security layer for an - existing connection where you don't have access to the connection's file - descriptor, or for some reason you don't want to use it. - - An SSL pipe can be in "wrapped" and "unwrapped" mode. In unwrapped mode, - data is passed through untransformed. In wrapped mode, application level - data is encrypted to SSL record level data and vice versa. The SSL record - level is the lowest level in the SSL protocol suite and is what travels - as-is over the wire. - - An SslPipe initially is in "unwrapped" mode. To start SSL, call - do_handshake(). To shutdown SSL again, call unwrap(). - """ - - max_size = 256 * 1024 # Buffer size passed to read() - - def __init__(self, context, server_side, server_hostname=None): - """ - The *context* argument specifies the ssl.SSLContext to use. - - The *server_side* argument indicates whether this is a server side or - client side transport. - - The optional *server_hostname* argument can be used to specify the - hostname you are connecting to. You may only specify this parameter if - the _ssl module supports Server Name Indication (SNI). - """ - self._context = context - self._server_side = server_side - self._server_hostname = server_hostname - self._state = _UNWRAPPED - self._incoming = ssl.MemoryBIO() - self._outgoing = ssl.MemoryBIO() - self._sslobj = None - self._need_ssldata = False - self._handshake_cb = None - self._shutdown_cb = None - - @property - def context(self): - """The SSL context passed to the constructor.""" - return self._context - - @property - def ssl_object(self): - """The internal ssl.SSLObject instance. - - Return None if the pipe is not wrapped. - """ - return self._sslobj - - @property - def need_ssldata(self): - """Whether more record level data is needed to complete a handshake - that is currently in progress.""" - return self._need_ssldata - - @property - def wrapped(self): - """ - Whether a security layer is currently in effect. - - Return False during handshake. - """ - return self._state == _WRAPPED - - def do_handshake(self, callback=None): - """Start the SSL handshake. - - Return a list of ssldata. A ssldata element is a list of buffers - - The optional *callback* argument can be used to install a callback that - will be called when the handshake is complete. The callback will be - called with None if successful, else an exception instance. - """ - if self._state != _UNWRAPPED: - raise RuntimeError('handshake in progress or completed') - self._sslobj = self._context.wrap_bio( - self._incoming, self._outgoing, - server_side=self._server_side, - server_hostname=self._server_hostname) - self._state = _DO_HANDSHAKE - self._handshake_cb = callback - ssldata, appdata = self.feed_ssldata(b'', only_handshake=True) - assert len(appdata) == 0 - return ssldata - - def shutdown(self, callback=None): - """Start the SSL shutdown sequence. - - Return a list of ssldata. A ssldata element is a list of buffers - - The optional *callback* argument can be used to install a callback that - will be called when the shutdown is complete. The callback will be - called without arguments. - """ - if self._state == _UNWRAPPED: - raise RuntimeError('no security layer present') - if self._state == _SHUTDOWN: - raise RuntimeError('shutdown in progress') - assert self._state in (_WRAPPED, _DO_HANDSHAKE) - self._state = _SHUTDOWN - self._shutdown_cb = callback - ssldata, appdata = self.feed_ssldata(b'') - assert appdata == [] or appdata == [b''] - return ssldata - - def feed_eof(self): - """Send a potentially "ragged" EOF. - - This method will raise an SSL_ERROR_EOF exception if the EOF is - unexpected. - """ - self._incoming.write_eof() - ssldata, appdata = self.feed_ssldata(b'') - assert appdata == [] or appdata == [b''] - - def feed_ssldata(self, data, only_handshake=False): - """Feed SSL record level data into the pipe. - - The data must be a bytes instance. It is OK to send an empty bytes - instance. This can be used to get ssldata for a handshake initiated by - this endpoint. - - Return a (ssldata, appdata) tuple. The ssldata element is a list of - buffers containing SSL data that needs to be sent to the remote SSL. - - The appdata element is a list of buffers containing plaintext data that - needs to be forwarded to the application. The appdata list may contain - an empty buffer indicating an SSL "close_notify" alert. This alert must - be acknowledged by calling shutdown(). - """ - if self._state == _UNWRAPPED: - # If unwrapped, pass plaintext data straight through. - if data: - appdata = [data] - else: - appdata = [] - return ([], appdata) +def add_flowcontrol_defaults(high, low, kb): + if high is None: + if low is None: + hi = kb * 1024 + else: + lo = low + hi = 4 * lo + else: + hi = high + if low is None: + lo = hi // 4 + else: + lo = low - self._need_ssldata = False - if data: - self._incoming.write(data) + if not hi >= lo >= 0: + raise ValueError('high (%r) must be >= low (%r) must be >= 0' % + (hi, lo)) - ssldata = [] - appdata = [] - try: - if self._state == _DO_HANDSHAKE: - # Call do_handshake() until it doesn't raise anymore. - self._sslobj.do_handshake() - self._state = _WRAPPED - if self._handshake_cb: - self._handshake_cb(None) - if only_handshake: - return (ssldata, appdata) - # Handshake done: execute the wrapped block - - if self._state == _WRAPPED: - # Main state: read data from SSL until close_notify - while True: - chunk = self._sslobj.read(self.max_size) - appdata.append(chunk) - if not chunk: # close_notify - break - - elif self._state == _SHUTDOWN: - # Call shutdown() until it doesn't raise anymore. - self._sslobj.unwrap() - self._sslobj = None - self._state = _UNWRAPPED - if self._shutdown_cb: - self._shutdown_cb() - - elif self._state == _UNWRAPPED: - # Drain possible plaintext data after close_notify. - appdata.append(self._incoming.read()) - except (ssl.SSLError, ssl.CertificateError) as exc: - exc_errno = getattr(exc, 'errno', None) - if exc_errno not in ( - ssl.SSL_ERROR_WANT_READ, ssl.SSL_ERROR_WANT_WRITE, - ssl.SSL_ERROR_SYSCALL): - if self._state == _DO_HANDSHAKE and self._handshake_cb: - self._handshake_cb(exc) - raise - self._need_ssldata = (exc_errno == ssl.SSL_ERROR_WANT_READ) - - # Check for record level data that needs to be sent back. - # Happens for the initial handshake and renegotiations. - if self._outgoing.pending: - ssldata.append(self._outgoing.read()) - return (ssldata, appdata) - - def feed_appdata(self, data, offset=0): - """Feed plaintext data into the pipe. - - Return an (ssldata, offset) tuple. The ssldata element is a list of - buffers containing record level data that needs to be sent to the - remote SSL instance. The offset is the number of plaintext bytes that - were processed, which may be less than the length of data. - - NOTE: In case of short writes, this call MUST be retried with the SAME - buffer passed into the *data* argument (i.e. the id() must be the - same). This is an OpenSSL requirement. A further particularity is that - a short write will always have offset == 0, because the _ssl module - does not enable partial writes. And even though the offset is zero, - there will still be encrypted data in ssldata. - """ - assert 0 <= offset <= len(data) - if self._state == _UNWRAPPED: - # pass through data in unwrapped mode - if offset < len(data): - ssldata = [data[offset:]] - else: - ssldata = [] - return (ssldata, len(data)) - - ssldata = [] - view = memoryview(data) - while True: - self._need_ssldata = False - try: - if offset < len(view): - offset += self._sslobj.write(view[offset:]) - except ssl.SSLError as exc: - # It is not allowed to call write() after unwrap() until the - # close_notify is acknowledged. We return the condition to the - # caller as a short write. - exc_errno = getattr(exc, 'errno', None) - if exc.reason == 'PROTOCOL_IS_SHUTDOWN': - exc_errno = exc.errno = ssl.SSL_ERROR_WANT_READ - if exc_errno not in (ssl.SSL_ERROR_WANT_READ, - ssl.SSL_ERROR_WANT_WRITE, - ssl.SSL_ERROR_SYSCALL): - raise - self._need_ssldata = (exc_errno == ssl.SSL_ERROR_WANT_READ) - - # See if there's any record level data back for us. - if self._outgoing.pending: - ssldata.append(self._outgoing.read()) - if offset == len(view) or self._need_ssldata: - break - return (ssldata, offset) + return hi, lo class _SSLProtocolTransport(transports._FlowControlMixin, transports.Transport): + _start_tls_compatible = True _sendfile_compatible = constants._SendfileMode.FALLBACK def __init__(self, loop, ssl_protocol): self._loop = loop - # SSLProtocol instance self._ssl_protocol = ssl_protocol self._closed = False @@ -315,16 +110,15 @@ def close(self): self._closed = True self._ssl_protocol._start_shutdown() - def __del__(self, _warn=warnings.warn): + def __del__(self, _warnings=warnings): if not self._closed: - _warn(f"unclosed transport {self!r}", ResourceWarning, source=self) - self.close() + self._closed = True + _warnings.warn( + "unclosed transport ", ResourceWarning) def is_reading(self): - tr = self._ssl_protocol._transport - if tr is None: - raise RuntimeError('SSL transport has not been initialized yet') - return tr.is_reading() + return not self._ssl_protocol._app_reading_paused def pause_reading(self): """Pause the receiving end. @@ -332,7 +126,7 @@ def pause_reading(self): No data will be passed to the protocol's data_received() method until resume_reading() is called. """ - self._ssl_protocol._transport.pause_reading() + self._ssl_protocol._pause_reading() def resume_reading(self): """Resume the receiving end. @@ -340,7 +134,7 @@ def resume_reading(self): Data received will once again be passed to the protocol's data_received() method. """ - self._ssl_protocol._transport.resume_reading() + self._ssl_protocol._resume_reading() def set_write_buffer_limits(self, high=None, low=None): """Set the high- and low-water limits for write flow control. @@ -361,11 +155,46 @@ def set_write_buffer_limits(self, high=None, low=None): reduces opportunities for doing I/O and computation concurrently. """ - self._ssl_protocol._transport.set_write_buffer_limits(high, low) + self._ssl_protocol._set_write_buffer_limits(high, low) + self._ssl_protocol._control_app_writing() + + def get_write_buffer_limits(self): + return (self._ssl_protocol._outgoing_low_water, + self._ssl_protocol._outgoing_high_water) def get_write_buffer_size(self): - """Return the current size of the write buffer.""" - return self._ssl_protocol._transport.get_write_buffer_size() + """Return the current size of the write buffers.""" + return self._ssl_protocol._get_write_buffer_size() + + def set_read_buffer_limits(self, high=None, low=None): + """Set the high- and low-water limits for read flow control. + + These two values control when to call the upstream transport's + pause_reading() and resume_reading() methods. If specified, + the low-water limit must be less than or equal to the + high-water limit. Neither value can be negative. + + The defaults are implementation-specific. If only the + high-water limit is given, the low-water limit defaults to an + implementation-specific value less than or equal to the + high-water limit. Setting high to zero forces low to zero as + well, and causes pause_reading() to be called whenever the + buffer becomes non-empty. Setting low to zero causes + resume_reading() to be called only once the buffer is empty. + Use of zero for either limit is generally sub-optimal as it + reduces opportunities for doing I/O and computation + concurrently. + """ + self._ssl_protocol._set_read_buffer_limits(high, low) + self._ssl_protocol._control_ssl_reading() + + def get_read_buffer_limits(self): + return (self._ssl_protocol._incoming_low_water, + self._ssl_protocol._incoming_high_water) + + def get_read_buffer_size(self): + """Return the current size of the read buffer.""" + return self._ssl_protocol._get_read_buffer_size() def get_write_buffer_limits(self): """Get the high and low watermarks for write flow control. @@ -376,7 +205,7 @@ def get_write_buffer_limits(self): @property def _protocol_paused(self): # Required for sendfile fallback pause_writing/resume_writing logic - return self._ssl_protocol._transport._protocol_paused + return self._ssl_protocol._app_writing_paused def write(self, data): """Write some data bytes to the transport. @@ -389,7 +218,22 @@ def write(self, data): f"got {type(data).__name__}") if not data: return - self._ssl_protocol._write_appdata(data) + self._ssl_protocol._write_appdata((data,)) + + def writelines(self, list_of_data): + """Write a list (or any iterable) of data bytes to the transport. + + The default implementation concatenates the arguments and + calls write() on the result. + """ + self._ssl_protocol._write_appdata(list_of_data) + + def write_eof(self): + """Close the write end after flushing buffered data. + + This raises :exc:`NotImplementedError` right now. + """ + raise NotImplementedError def can_write_eof(self): """Return True if this transport supports write_eof(), False if not.""" @@ -402,23 +246,36 @@ def abort(self): The protocol's connection_lost() method will (eventually) be called with None as its argument. """ + self._closed = True self._ssl_protocol._abort() + + def _force_close(self, exc): self._closed = True + self._ssl_protocol._abort(exc) + def _test__append_write_backlog(self, data): + # for test only + self._ssl_protocol._write_backlog.append(data) + self._ssl_protocol._write_buffer_size += len(data) -class SSLProtocol(protocols.Protocol): - """SSL protocol. - Implementation of SSL on top of a socket using incoming and outgoing - buffers which are ssl.MemoryBIO objects. - """ +class SSLProtocol(protocols.BufferedProtocol): + max_size = 256 * 1024 # Buffer size passed to read() + + _handshake_start_time = None + _handshake_timeout_handle = None + _shutdown_timeout_handle = None def __init__(self, loop, app_protocol, sslcontext, waiter, server_side=False, server_hostname=None, call_connection_made=True, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): if ssl is None: - raise RuntimeError('stdlib ssl module not available') + raise RuntimeError("stdlib ssl module not available") + + self._ssl_buffer = bytearray(self.max_size) + self._ssl_buffer_view = memoryview(self._ssl_buffer) if ssl_handshake_timeout is None: ssl_handshake_timeout = constants.SSL_HANDSHAKE_TIMEOUT @@ -426,6 +283,12 @@ def __init__(self, loop, app_protocol, sslcontext, waiter, raise ValueError( f"ssl_handshake_timeout should be a positive number, " f"got {ssl_handshake_timeout}") + if ssl_shutdown_timeout is None: + ssl_shutdown_timeout = constants.SSL_SHUTDOWN_TIMEOUT + elif ssl_shutdown_timeout <= 0: + raise ValueError( + f"ssl_shutdown_timeout should be a positive number, " + f"got {ssl_shutdown_timeout}") if not sslcontext: sslcontext = _create_transport_context( @@ -448,21 +311,54 @@ def __init__(self, loop, app_protocol, sslcontext, waiter, self._waiter = waiter self._loop = loop self._set_app_protocol(app_protocol) - self._app_transport = _SSLProtocolTransport(self._loop, self) - # _SSLPipe instance (None until the connection is made) - self._sslpipe = None - self._session_established = False - self._in_handshake = False - self._in_shutdown = False + self._app_transport = None + self._app_transport_created = False # transport, ex: SelectorSocketTransport self._transport = None - self._call_connection_made = call_connection_made self._ssl_handshake_timeout = ssl_handshake_timeout + self._ssl_shutdown_timeout = ssl_shutdown_timeout + # SSL and state machine + self._incoming = ssl.MemoryBIO() + self._outgoing = ssl.MemoryBIO() + self._state = SSLProtocolState.UNWRAPPED + self._conn_lost = 0 # Set when connection_lost called + if call_connection_made: + self._app_state = AppProtocolState.STATE_INIT + else: + self._app_state = AppProtocolState.STATE_CON_MADE + self._sslobj = self._sslcontext.wrap_bio( + self._incoming, self._outgoing, + server_side=self._server_side, + server_hostname=self._server_hostname) + + # Flow Control + + self._ssl_writing_paused = False + + self._app_reading_paused = False + + self._ssl_reading_paused = False + self._incoming_high_water = 0 + self._incoming_low_water = 0 + self._set_read_buffer_limits() + self._eof_received = False + + self._app_writing_paused = False + self._outgoing_high_water = 0 + self._outgoing_low_water = 0 + self._set_write_buffer_limits() + self._get_app_transport() def _set_app_protocol(self, app_protocol): self._app_protocol = app_protocol - self._app_protocol_is_buffer = \ - isinstance(app_protocol, protocols.BufferedProtocol) + # Make fast hasattr check first + if (hasattr(app_protocol, 'get_buffer') and + isinstance(app_protocol, protocols.BufferedProtocol)): + self._app_protocol_get_buffer = app_protocol.get_buffer + self._app_protocol_buffer_updated = app_protocol.buffer_updated + self._app_protocol_is_buffer = True + else: + self._app_protocol_is_buffer = False def _wakeup_waiter(self, exc=None): if self._waiter is None: @@ -474,15 +370,20 @@ def _wakeup_waiter(self, exc=None): self._waiter.set_result(None) self._waiter = None + def _get_app_transport(self): + if self._app_transport is None: + if self._app_transport_created: + raise RuntimeError('Creating _SSLProtocolTransport twice') + self._app_transport = _SSLProtocolTransport(self._loop, self) + self._app_transport_created = True + return self._app_transport + def connection_made(self, transport): """Called when the low-level connection is made. Start the SSL handshake. """ self._transport = transport - self._sslpipe = _SSLPipe(self._sslcontext, - self._server_side, - self._server_hostname) self._start_handshake() def connection_lost(self, exc): @@ -492,72 +393,58 @@ def connection_lost(self, exc): meaning a regular EOF is received or the connection was aborted or closed). """ - if self._session_established: - self._session_established = False - self._loop.call_soon(self._app_protocol.connection_lost, exc) - else: - # Most likely an exception occurred while in SSL handshake. - # Just mark the app transport as closed so that its __del__ - # doesn't complain. - if self._app_transport is not None: - self._app_transport._closed = True + self._write_backlog.clear() + self._outgoing.read() + self._conn_lost += 1 + + # Just mark the app transport as closed so that its __dealloc__ + # doesn't complain. + if self._app_transport is not None: + self._app_transport._closed = True + + if self._state != SSLProtocolState.DO_HANDSHAKE: + if ( + self._app_state == AppProtocolState.STATE_CON_MADE or + self._app_state == AppProtocolState.STATE_EOF + ): + self._app_state = AppProtocolState.STATE_CON_LOST + self._loop.call_soon(self._app_protocol.connection_lost, exc) + self._set_state(SSLProtocolState.UNWRAPPED) self._transport = None self._app_transport = None - if getattr(self, '_handshake_timeout_handle', None): - self._handshake_timeout_handle.cancel() - self._wakeup_waiter(exc) self._app_protocol = None - self._sslpipe = None + self._wakeup_waiter(exc) - def pause_writing(self): - """Called when the low-level transport's buffer goes over - the high-water mark. - """ - self._app_protocol.pause_writing() + if self._shutdown_timeout_handle: + self._shutdown_timeout_handle.cancel() + self._shutdown_timeout_handle = None + if self._handshake_timeout_handle: + self._handshake_timeout_handle.cancel() + self._handshake_timeout_handle = None - def resume_writing(self): - """Called when the low-level transport's buffer drains below - the low-water mark. - """ - self._app_protocol.resume_writing() + def get_buffer(self, n): + want = n + if want <= 0 or want > self.max_size: + want = self.max_size + if len(self._ssl_buffer) < want: + self._ssl_buffer = bytearray(want) + self._ssl_buffer_view = memoryview(self._ssl_buffer) + return self._ssl_buffer_view - def data_received(self, data): - """Called when some SSL data is received. + def buffer_updated(self, nbytes): + self._incoming.write(self._ssl_buffer_view[:nbytes]) - The argument is a bytes object. - """ - if self._sslpipe is None: - # transport closing, sslpipe is destroyed - return + if self._state == SSLProtocolState.DO_HANDSHAKE: + self._do_handshake() - try: - ssldata, appdata = self._sslpipe.feed_ssldata(data) - except (SystemExit, KeyboardInterrupt): - raise - except BaseException as e: - self._fatal_error(e, 'SSL error in data received') - return + elif self._state == SSLProtocolState.WRAPPED: + self._do_read() - for chunk in ssldata: - self._transport.write(chunk) + elif self._state == SSLProtocolState.FLUSHING: + self._do_flush() - for chunk in appdata: - if chunk: - try: - if self._app_protocol_is_buffer: - protocols._feed_data_to_buffered_proto( - self._app_protocol, chunk) - else: - self._app_protocol.data_received(chunk) - except (SystemExit, KeyboardInterrupt): - raise - except BaseException as ex: - self._fatal_error( - ex, 'application protocol failed to receive SSL data') - return - else: - self._start_shutdown() - break + elif self._state == SSLProtocolState.SHUTDOWN: + self._do_shutdown() def eof_received(self): """Called when the other end of the low-level stream @@ -567,19 +454,32 @@ def eof_received(self): will close itself. If it returns a true value, closing the transport is up to the protocol. """ + self._eof_received = True try: if self._loop.get_debug(): logger.debug("%r received EOF", self) - self._wakeup_waiter(ConnectionResetError) + if self._state == SSLProtocolState.DO_HANDSHAKE: + self._on_handshake_complete(ConnectionResetError) - if not self._in_handshake: - keep_open = self._app_protocol.eof_received() - if keep_open: - logger.warning('returning true from eof_received() ' - 'has no effect when using ssl') - finally: + elif self._state == SSLProtocolState.WRAPPED: + self._set_state(SSLProtocolState.FLUSHING) + if self._app_reading_paused: + return True + else: + self._do_flush() + + elif self._state == SSLProtocolState.FLUSHING: + self._do_write() + self._set_state(SSLProtocolState.SHUTDOWN) + self._do_shutdown() + + elif self._state == SSLProtocolState.SHUTDOWN: + self._do_shutdown() + + except Exception: self._transport.close() + raise def _get_extra_info(self, name, default=None): if name in self._extra: @@ -589,19 +489,45 @@ def _get_extra_info(self, name, default=None): else: return default - def _start_shutdown(self): - if self._in_shutdown: - return - if self._in_handshake: - self._abort() + def _set_state(self, new_state): + allowed = False + + if new_state == SSLProtocolState.UNWRAPPED: + allowed = True + + elif ( + self._state == SSLProtocolState.UNWRAPPED and + new_state == SSLProtocolState.DO_HANDSHAKE + ): + allowed = True + + elif ( + self._state == SSLProtocolState.DO_HANDSHAKE and + new_state == SSLProtocolState.WRAPPED + ): + allowed = True + + elif ( + self._state == SSLProtocolState.WRAPPED and + new_state == SSLProtocolState.FLUSHING + ): + allowed = True + + elif ( + self._state == SSLProtocolState.FLUSHING and + new_state == SSLProtocolState.SHUTDOWN + ): + allowed = True + + if allowed: + self._state = new_state + else: - self._in_shutdown = True - self._write_appdata(b'') + raise RuntimeError( + 'cannot switch state from {} to {}'.format( + self._state, new_state)) - def _write_appdata(self, data): - self._write_backlog.append((data, 0)) - self._write_buffer_size += len(data) - self._process_write_backlog() + # Handshake flow def _start_handshake(self): if self._loop.get_debug(): @@ -609,17 +535,18 @@ def _start_handshake(self): self._handshake_start_time = self._loop.time() else: self._handshake_start_time = None - self._in_handshake = True - # (b'', 1) is a special value in _process_write_backlog() to do - # the SSL handshake - self._write_backlog.append((b'', 1)) + + self._set_state(SSLProtocolState.DO_HANDSHAKE) + + # start handshake timeout count down self._handshake_timeout_handle = \ self._loop.call_later(self._ssl_handshake_timeout, - self._check_handshake_timeout) - self._process_write_backlog() + lambda: self._check_handshake_timeout()) + + self._do_handshake() def _check_handshake_timeout(self): - if self._in_handshake is True: + if self._state == SSLProtocolState.DO_HANDSHAKE: msg = ( f"SSL handshake is taking longer than " f"{self._ssl_handshake_timeout} seconds: " @@ -627,24 +554,37 @@ def _check_handshake_timeout(self): ) self._fatal_error(ConnectionAbortedError(msg)) + def _do_handshake(self): + try: + self._sslobj.do_handshake() + except SSLAgainErrors: + self._process_outgoing() + except ssl.SSLError as exc: + self._on_handshake_complete(exc) + else: + self._on_handshake_complete(None) + def _on_handshake_complete(self, handshake_exc): - self._in_handshake = False - self._handshake_timeout_handle.cancel() + if self._handshake_timeout_handle is not None: + self._handshake_timeout_handle.cancel() + self._handshake_timeout_handle = None - sslobj = self._sslpipe.ssl_object + sslobj = self._sslobj try: - if handshake_exc is not None: + if handshake_exc is None: + self._set_state(SSLProtocolState.WRAPPED) + else: raise handshake_exc peercert = sslobj.getpeercert() - except (SystemExit, KeyboardInterrupt): - raise - except BaseException as exc: + except Exception as exc: + self._set_state(SSLProtocolState.UNWRAPPED) if isinstance(exc, ssl.CertificateError): msg = 'SSL handshake failed on verifying the certificate' else: msg = 'SSL handshake failed' self._fatal_error(exc, msg) + self._wakeup_waiter(exc) return if self._loop.get_debug(): @@ -655,85 +595,330 @@ def _on_handshake_complete(self, handshake_exc): self._extra.update(peercert=peercert, cipher=sslobj.cipher(), compression=sslobj.compression(), - ssl_object=sslobj, - ) - if self._call_connection_made: - self._app_protocol.connection_made(self._app_transport) + ssl_object=sslobj) + if self._app_state == AppProtocolState.STATE_INIT: + self._app_state = AppProtocolState.STATE_CON_MADE + self._app_protocol.connection_made(self._get_app_transport()) self._wakeup_waiter() - self._session_established = True - # In case transport.write() was already called. Don't call - # immediately _process_write_backlog(), but schedule it: - # _on_handshake_complete() can be called indirectly from - # _process_write_backlog(), and _process_write_backlog() is not - # reentrant. - self._loop.call_soon(self._process_write_backlog) - - def _process_write_backlog(self): - # Try to make progress on the write backlog. - if self._transport is None or self._sslpipe is None: + self._do_read() + + # Shutdown flow + + def _start_shutdown(self): + if ( + self._state in ( + SSLProtocolState.FLUSHING, + SSLProtocolState.SHUTDOWN, + SSLProtocolState.UNWRAPPED + ) + ): + return + if self._app_transport is not None: + self._app_transport._closed = True + if self._state == SSLProtocolState.DO_HANDSHAKE: + self._abort() + else: + self._set_state(SSLProtocolState.FLUSHING) + self._shutdown_timeout_handle = self._loop.call_later( + self._ssl_shutdown_timeout, + lambda: self._check_shutdown_timeout() + ) + self._do_flush() + + def _check_shutdown_timeout(self): + if ( + self._state in ( + SSLProtocolState.FLUSHING, + SSLProtocolState.SHUTDOWN + ) + ): + self._transport._force_close( + exceptions.TimeoutError('SSL shutdown timed out')) + + def _do_flush(self): + self._do_read() + self._set_state(SSLProtocolState.SHUTDOWN) + self._do_shutdown() + + def _do_shutdown(self): + try: + if not self._eof_received: + self._sslobj.unwrap() + except SSLAgainErrors: + self._process_outgoing() + except ssl.SSLError as exc: + self._on_shutdown_complete(exc) + else: + self._process_outgoing() + self._call_eof_received() + self._on_shutdown_complete(None) + + def _on_shutdown_complete(self, shutdown_exc): + if self._shutdown_timeout_handle is not None: + self._shutdown_timeout_handle.cancel() + self._shutdown_timeout_handle = None + + if shutdown_exc: + self._fatal_error(shutdown_exc) + else: + self._loop.call_soon(self._transport.close) + + def _abort(self): + self._set_state(SSLProtocolState.UNWRAPPED) + if self._transport is not None: + self._transport.abort() + + # Outgoing flow + + def _write_appdata(self, list_of_data): + if ( + self._state in ( + SSLProtocolState.FLUSHING, + SSLProtocolState.SHUTDOWN, + SSLProtocolState.UNWRAPPED + ) + ): + if self._conn_lost >= constants.LOG_THRESHOLD_FOR_CONNLOST_WRITES: + logger.warning('SSL connection is closed') + self._conn_lost += 1 return + for data in list_of_data: + self._write_backlog.append(data) + self._write_buffer_size += len(data) + try: - for i in range(len(self._write_backlog)): - data, offset = self._write_backlog[0] - if data: - ssldata, offset = self._sslpipe.feed_appdata(data, offset) - elif offset: - ssldata = self._sslpipe.do_handshake( - self._on_handshake_complete) - offset = 1 + if self._state == SSLProtocolState.WRAPPED: + self._do_write() + + except Exception as ex: + self._fatal_error(ex, 'Fatal error on SSL protocol') + + def _do_write(self): + try: + while self._write_backlog: + data = self._write_backlog[0] + count = self._sslobj.write(data) + data_len = len(data) + if count < data_len: + self._write_backlog[0] = data[count:] + self._write_buffer_size -= count else: - ssldata = self._sslpipe.shutdown(self._finalize) - offset = 1 - - for chunk in ssldata: - self._transport.write(chunk) - - if offset < len(data): - self._write_backlog[0] = (data, offset) - # A short write means that a write is blocked on a read - # We need to enable reading if it is paused! - assert self._sslpipe.need_ssldata - if self._transport._paused: - self._transport.resume_reading() - break + del self._write_backlog[0] + self._write_buffer_size -= data_len + except SSLAgainErrors: + pass + self._process_outgoing() + + def _process_outgoing(self): + if not self._ssl_writing_paused: + data = self._outgoing.read() + if len(data): + self._transport.write(data) + self._control_app_writing() + + # Incoming flow + + def _do_read(self): + if ( + self._state not in ( + SSLProtocolState.WRAPPED, + SSLProtocolState.FLUSHING, + ) + ): + return + try: + if not self._app_reading_paused: + if self._app_protocol_is_buffer: + self._do_read__buffered() + else: + self._do_read__copied() + if self._write_backlog: + self._do_write() + else: + self._process_outgoing() + self._control_ssl_reading() + except Exception as ex: + self._fatal_error(ex, 'Fatal error on SSL protocol') + + def _do_read__buffered(self): + offset = 0 + count = 1 + + buf = self._app_protocol_get_buffer(self._get_read_buffer_size()) + wants = len(buf) - # An entire chunk from the backlog was processed. We can - # delete it and reduce the outstanding buffer size. - del self._write_backlog[0] - self._write_buffer_size -= len(data) - except (SystemExit, KeyboardInterrupt): + try: + count = self._sslobj.read(wants, buf) + + if count > 0: + offset = count + while offset < wants: + count = self._sslobj.read(wants - offset, buf[offset:]) + if count > 0: + offset += count + else: + break + else: + self._loop.call_soon(lambda: self._do_read()) + except SSLAgainErrors: + pass + if offset > 0: + self._app_protocol_buffer_updated(offset) + if not count: + # close_notify + self._call_eof_received() + self._start_shutdown() + + def _do_read__copied(self): + chunk = b'1' + zero = True + one = False + + try: + while True: + chunk = self._sslobj.read(self.max_size) + if not chunk: + break + if zero: + zero = False + one = True + first = chunk + elif one: + one = False + data = [first, chunk] + else: + data.append(chunk) + except SSLAgainErrors: + pass + if one: + self._app_protocol.data_received(first) + elif not zero: + self._app_protocol.data_received(b''.join(data)) + if not chunk: + # close_notify + self._call_eof_received() + self._start_shutdown() + + def _call_eof_received(self): + try: + if self._app_state == AppProtocolState.STATE_CON_MADE: + self._app_state = AppProtocolState.STATE_EOF + keep_open = self._app_protocol.eof_received() + if keep_open: + logger.warning('returning true from eof_received() ' + 'has no effect when using ssl') + except (KeyboardInterrupt, SystemExit): raise - except BaseException as exc: - if self._in_handshake: - # Exceptions will be re-raised in _on_handshake_complete. - self._on_handshake_complete(exc) - else: - self._fatal_error(exc, 'Fatal error on SSL transport') + except BaseException as ex: + self._fatal_error(ex, 'Error calling eof_received()') + + # Flow control for writes from APP socket + + def _control_app_writing(self): + size = self._get_write_buffer_size() + if size >= self._outgoing_high_water and not self._app_writing_paused: + self._app_writing_paused = True + try: + self._app_protocol.pause_writing() + except (KeyboardInterrupt, SystemExit): + raise + except BaseException as exc: + self._loop.call_exception_handler({ + 'message': 'protocol.pause_writing() failed', + 'exception': exc, + 'transport': self._app_transport, + 'protocol': self, + }) + elif size <= self._outgoing_low_water and self._app_writing_paused: + self._app_writing_paused = False + try: + self._app_protocol.resume_writing() + except (KeyboardInterrupt, SystemExit): + raise + except BaseException as exc: + self._loop.call_exception_handler({ + 'message': 'protocol.resume_writing() failed', + 'exception': exc, + 'transport': self._app_transport, + 'protocol': self, + }) + + def _get_write_buffer_size(self): + return self._outgoing.pending + self._write_buffer_size + + def _set_write_buffer_limits(self, high=None, low=None): + high, low = add_flowcontrol_defaults( + high, low, constants.FLOW_CONTROL_HIGH_WATER_SSL_WRITE) + self._outgoing_high_water = high + self._outgoing_low_water = low + + # Flow control for reads to APP socket + + def _pause_reading(self): + self._app_reading_paused = True + + def _resume_reading(self): + if self._app_reading_paused: + self._app_reading_paused = False + + def resume(): + if self._state == SSLProtocolState.WRAPPED: + self._do_read() + elif self._state == SSLProtocolState.FLUSHING: + self._do_flush() + elif self._state == SSLProtocolState.SHUTDOWN: + self._do_shutdown() + self._loop.call_soon(resume) + + # Flow control for reads from SSL socket + + def _control_ssl_reading(self): + size = self._get_read_buffer_size() + if size >= self._incoming_high_water and not self._ssl_reading_paused: + self._ssl_reading_paused = True + self._transport.pause_reading() + elif size <= self._incoming_low_water and self._ssl_reading_paused: + self._ssl_reading_paused = False + self._transport.resume_reading() + + def _set_read_buffer_limits(self, high=None, low=None): + high, low = add_flowcontrol_defaults( + high, low, constants.FLOW_CONTROL_HIGH_WATER_SSL_READ) + self._incoming_high_water = high + self._incoming_low_water = low + + def _get_read_buffer_size(self): + return self._incoming.pending + + # Flow control for writes to SSL socket + + def pause_writing(self): + """Called when the low-level transport's buffer goes over + the high-water mark. + """ + assert not self._ssl_writing_paused + self._ssl_writing_paused = True + + def resume_writing(self): + """Called when the low-level transport's buffer drains below + the low-water mark. + """ + assert self._ssl_writing_paused + self._ssl_writing_paused = False + self._process_outgoing() def _fatal_error(self, exc, message='Fatal error on transport'): + if self._transport: + self._transport._force_close(exc) + if isinstance(exc, OSError): if self._loop.get_debug(): logger.debug("%r: %s", self, message, exc_info=True) - else: + elif not isinstance(exc, exceptions.CancelledError): self._loop.call_exception_handler({ 'message': message, 'exception': exc, 'transport': self._transport, 'protocol': self, }) - if self._transport: - self._transport._force_close(exc) - - def _finalize(self): - self._sslpipe = None - - if self._transport is not None: - self._transport.close() - - def _abort(self): - try: - if self._transport is not None: - self._transport.abort() - finally: - self._finalize() diff --git a/Lib/asyncio/unix_events.py b/Lib/asyncio/unix_events.py index c88b818de62a6..cf7683fee6462 100644 --- a/Lib/asyncio/unix_events.py +++ b/Lib/asyncio/unix_events.py @@ -229,7 +229,8 @@ async def create_unix_connection( self, protocol_factory, path=None, *, ssl=None, sock=None, server_hostname=None, - ssl_handshake_timeout=None): + ssl_handshake_timeout=None, + ssl_shutdown_timeout=None): assert server_hostname is None or isinstance(server_hostname, str) if ssl: if server_hostname is None: @@ -241,6 +242,9 @@ async def create_unix_connection( if ssl_handshake_timeout is not None: raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if ssl_shutdown_timeout is not None: + raise ValueError( + 'ssl_shutdown_timeout is only meaningful with ssl') if path is not None: if sock is not None: @@ -267,13 +271,15 @@ async def create_unix_connection( transport, protocol = await self._create_connection_transport( sock, protocol_factory, ssl, server_hostname, - ssl_handshake_timeout=ssl_handshake_timeout) + ssl_handshake_timeout=ssl_handshake_timeout, + ssl_shutdown_timeout=ssl_shutdown_timeout) return transport, protocol async def create_unix_server( self, protocol_factory, path=None, *, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, + ssl_shutdown_timeout=None, start_serving=True): if isinstance(ssl, bool): raise TypeError('ssl argument must be an SSLContext or None') @@ -282,6 +288,10 @@ async def create_unix_server( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if ssl_shutdown_timeout is not None and not ssl: + raise ValueError( + 'ssl_shutdown_timeout is only meaningful with ssl') + if path is not None: if sock is not None: raise ValueError( @@ -328,7 +338,8 @@ async def create_unix_server( sock.setblocking(False) server = base_events.Server(self, [sock], protocol_factory, - ssl, backlog, ssl_handshake_timeout) + ssl, backlog, ssl_handshake_timeout, + ssl_shutdown_timeout) if start_serving: server._start_serving() # Skip one loop iteration so that all 'loop.add_reader' diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py index c64e1623a0395..c6671bd0ad3d8 100644 --- a/Lib/test/test_asyncio/test_base_events.py +++ b/Lib/test/test_asyncio/test_base_events.py @@ -1451,44 +1451,51 @@ def mock_make_ssl_transport(sock, protocol, sslcontext, waiter, self.loop._make_ssl_transport.side_effect = mock_make_ssl_transport ANY = mock.ANY handshake_timeout = object() + shutdown_timeout = object() # First try the default server_hostname. self.loop._make_ssl_transport.reset_mock() coro = self.loop.create_connection( MyProto, 'python.org', 80, ssl=True, - ssl_handshake_timeout=handshake_timeout) + ssl_handshake_timeout=handshake_timeout, + ssl_shutdown_timeout=shutdown_timeout) transport, _ = self.loop.run_until_complete(coro) transport.close() self.loop._make_ssl_transport.assert_called_with( ANY, ANY, ANY, ANY, server_side=False, server_hostname='python.org', - ssl_handshake_timeout=handshake_timeout) + ssl_handshake_timeout=handshake_timeout, + ssl_shutdown_timeout=shutdown_timeout) # Next try an explicit server_hostname. self.loop._make_ssl_transport.reset_mock() coro = self.loop.create_connection( MyProto, 'python.org', 80, ssl=True, server_hostname='perl.com', - ssl_handshake_timeout=handshake_timeout) + ssl_handshake_timeout=handshake_timeout, + ssl_shutdown_timeout=shutdown_timeout) transport, _ = self.loop.run_until_complete(coro) transport.close() self.loop._make_ssl_transport.assert_called_with( ANY, ANY, ANY, ANY, server_side=False, server_hostname='perl.com', - ssl_handshake_timeout=handshake_timeout) + ssl_handshake_timeout=handshake_timeout, + ssl_shutdown_timeout=shutdown_timeout) # Finally try an explicit empty server_hostname. self.loop._make_ssl_transport.reset_mock() coro = self.loop.create_connection( MyProto, 'python.org', 80, ssl=True, server_hostname='', - ssl_handshake_timeout=handshake_timeout) + ssl_handshake_timeout=handshake_timeout, + ssl_shutdown_timeout=shutdown_timeout) transport, _ = self.loop.run_until_complete(coro) transport.close() self.loop._make_ssl_transport.assert_called_with( ANY, ANY, ANY, ANY, server_side=False, server_hostname='', - ssl_handshake_timeout=handshake_timeout) + ssl_handshake_timeout=handshake_timeout, + ssl_shutdown_timeout=shutdown_timeout) def test_create_connection_no_ssl_server_hostname_errors(self): # When not using ssl, server_hostname must be None. @@ -1869,7 +1876,7 @@ def test_accept_connection_exception(self, m_log): constants.ACCEPT_RETRY_DELAY, # self.loop._start_serving mock.ANY, - MyProto, sock, None, None, mock.ANY, mock.ANY) + MyProto, sock, None, None, mock.ANY, mock.ANY, mock.ANY) def test_call_coroutine(self): async def simple_coroutine(): diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py index b684fab2771f2..9c46018c9414b 100644 --- a/Lib/test/test_asyncio/test_selector_events.py +++ b/Lib/test/test_asyncio/test_selector_events.py @@ -71,44 +71,6 @@ def test_make_socket_transport(self): close_transport(transport) - @unittest.skipIf(ssl is None, 'No ssl module') - def test_make_ssl_transport(self): - m = mock.Mock() - self.loop._add_reader = mock.Mock() - self.loop._add_reader._is_coroutine = False - self.loop._add_writer = mock.Mock() - self.loop._remove_reader = mock.Mock() - self.loop._remove_writer = mock.Mock() - waiter = self.loop.create_future() - with test_utils.disable_logger(): - transport = self.loop._make_ssl_transport( - m, asyncio.Protocol(), m, waiter) - - with self.assertRaisesRegex(RuntimeError, - r'SSL transport.*not.*initialized'): - transport.is_reading() - - # execute the handshake while the logger is disabled - # to ignore SSL handshake failure - test_utils.run_briefly(self.loop) - - self.assertTrue(transport.is_reading()) - transport.pause_reading() - transport.pause_reading() - self.assertFalse(transport.is_reading()) - transport.resume_reading() - transport.resume_reading() - self.assertTrue(transport.is_reading()) - - # Sanity check - class_name = transport.__class__.__name__ - self.assertIn("ssl", class_name.lower()) - self.assertIn("transport", class_name.lower()) - - transport.close() - # execute pending callbacks to close the socket transport - test_utils.run_briefly(self.loop) - @mock.patch('asyncio.selector_events.ssl', None) @mock.patch('asyncio.sslproto.ssl', None) def test_make_ssl_transport_without_ssl_error(self): diff --git a/Lib/test/test_asyncio/test_ssl.py b/Lib/test/test_asyncio/test_ssl.py new file mode 100644 index 0000000000000..8d1bb0360934d --- /dev/null +++ b/Lib/test/test_asyncio/test_ssl.py @@ -0,0 +1,1721 @@ +import asyncio +import asyncio.sslproto +import contextlib +import gc +import logging +import select +import socket +import tempfile +import threading +import time +import weakref +import unittest + +try: + import ssl +except ImportError: + ssl = None + +from test import support +from test.test_asyncio import utils as test_utils + + +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + +class MyBaseProto(asyncio.Protocol): + connected = None + done = None + + def __init__(self, loop=None): + self.transport = None + self.state = 'INITIAL' + self.nbytes = 0 + if loop is not None: + self.connected = asyncio.Future(loop=loop) + self.done = asyncio.Future(loop=loop) + + def connection_made(self, transport): + self.transport = transport + assert self.state == 'INITIAL', self.state + self.state = 'CONNECTED' + if self.connected: + self.connected.set_result(None) + + def data_received(self, data): + assert self.state == 'CONNECTED', self.state + self.nbytes += len(data) + + def eof_received(self): + assert self.state == 'CONNECTED', self.state + self.state = 'EOF' + + def connection_lost(self, exc): + assert self.state in ('CONNECTED', 'EOF'), self.state + self.state = 'CLOSED' + if self.done: + self.done.set_result(None) + + + at unittest.skipIf(ssl is None, 'No ssl module') +class TestSSL(test_utils.TestCase): + + PAYLOAD_SIZE = 1024 * 100 + TIMEOUT = 60 + + def setUp(self): + super().setUp() + self.loop = asyncio.new_event_loop() + self.set_event_loop(self.loop) + self.addCleanup(self.loop.close) + + def tearDown(self): + # just in case if we have transport close callbacks + if not self.loop.is_closed(): + test_utils.run_briefly(self.loop) + + self.doCleanups() + support.gc_collect() + super().tearDown() + + def tcp_server(self, server_prog, *, + family=socket.AF_INET, + addr=None, + timeout=5, + backlog=1, + max_clients=10): + + if addr is None: + if family == getattr(socket, "AF_UNIX", None): + with tempfile.NamedTemporaryFile() as tmp: + addr = tmp.name + else: + addr = ('127.0.0.1', 0) + + sock = socket.socket(family, socket.SOCK_STREAM) + + if timeout is None: + raise RuntimeError('timeout is required') + if timeout <= 0: + raise RuntimeError('only blocking sockets are supported') + sock.settimeout(timeout) + + try: + sock.bind(addr) + sock.listen(backlog) + except OSError as ex: + sock.close() + raise ex + + return TestThreadedServer( + self, sock, server_prog, timeout, max_clients) + + def tcp_client(self, client_prog, + family=socket.AF_INET, + timeout=10): + + sock = socket.socket(family, socket.SOCK_STREAM) + + if timeout is None: + raise RuntimeError('timeout is required') + if timeout <= 0: + raise RuntimeError('only blocking sockets are supported') + sock.settimeout(timeout) + + return TestThreadedClient( + self, sock, client_prog, timeout) + + def unix_server(self, *args, **kwargs): + return self.tcp_server(*args, family=socket.AF_UNIX, **kwargs) + + def unix_client(self, *args, **kwargs): + return self.tcp_client(*args, family=socket.AF_UNIX, **kwargs) + + def _create_server_ssl_context(self, certfile, keyfile=None): + sslcontext = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + sslcontext.options |= ssl.OP_NO_SSLv2 + sslcontext.load_cert_chain(certfile, keyfile) + return sslcontext + + def _create_client_ssl_context(self, *, disable_verify=True): + sslcontext = ssl.create_default_context() + sslcontext.check_hostname = False + if disable_verify: + sslcontext.verify_mode = ssl.CERT_NONE + return sslcontext + + @contextlib.contextmanager + def _silence_eof_received_warning(self): + # TODO This warning has to be fixed in asyncio. + logger = logging.getLogger('asyncio') + filter = logging.Filter('has no effect when using ssl') + logger.addFilter(filter) + try: + yield + finally: + logger.removeFilter(filter) + + def _abort_socket_test(self, ex): + try: + self.loop.stop() + finally: + self.fail(ex) + + def new_loop(self): + return asyncio.new_event_loop() + + def new_policy(self): + return asyncio.DefaultEventLoopPolicy() + + async def wait_closed(self, obj): + if not isinstance(obj, asyncio.StreamWriter): + return + try: + await obj.wait_closed() + except (BrokenPipeError, ConnectionError): + pass + + def test_create_server_ssl_1(self): + CNT = 0 # number of clients that were successful + TOTAL_CNT = 25 # total number of clients that test will create + TIMEOUT = 60.0 # timeout for this test + + A_DATA = b'A' * 1024 * 1024 + B_DATA = b'B' * 1024 * 1024 + + sslctx = self._create_server_ssl_context( + test_utils.ONLYCERT, test_utils.ONLYKEY + ) + client_sslctx = self._create_client_ssl_context() + + clients = [] + + async def handle_client(reader, writer): + nonlocal CNT + + data = await reader.readexactly(len(A_DATA)) + self.assertEqual(data, A_DATA) + writer.write(b'OK') + + data = await reader.readexactly(len(B_DATA)) + self.assertEqual(data, B_DATA) + writer.writelines([b'SP', bytearray(b'A'), memoryview(b'M')]) + + await writer.drain() + writer.close() + + CNT += 1 + + async def test_client(addr): + fut = asyncio.Future() + + def prog(sock): + try: + sock.starttls(client_sslctx) + sock.connect(addr) + sock.send(A_DATA) + + data = sock.recv_all(2) + self.assertEqual(data, b'OK') + + sock.send(B_DATA) + data = sock.recv_all(4) + self.assertEqual(data, b'SPAM') + + sock.close() + + except Exception as ex: + self.loop.call_soon_threadsafe(fut.set_exception, ex) + else: + self.loop.call_soon_threadsafe(fut.set_result, None) + + client = self.tcp_client(prog) + client.start() + clients.append(client) + + await fut + + async def start_server(): + extras = {} + extras = dict(ssl_handshake_timeout=40.0) + + srv = await asyncio.start_server( + handle_client, + '127.0.0.1', 0, + family=socket.AF_INET, + ssl=sslctx, + **extras) + + try: + srv_socks = srv.sockets + self.assertTrue(srv_socks) + + addr = srv_socks[0].getsockname() + + tasks = [] + for _ in range(TOTAL_CNT): + tasks.append(test_client(addr)) + + await asyncio.wait_for(asyncio.gather(*tasks), TIMEOUT) + + finally: + self.loop.call_soon(srv.close) + await srv.wait_closed() + + with self._silence_eof_received_warning(): + self.loop.run_until_complete(start_server()) + + self.assertEqual(CNT, TOTAL_CNT) + + for client in clients: + client.stop() + + def test_create_connection_ssl_1(self): + self.loop.set_exception_handler(None) + + CNT = 0 + TOTAL_CNT = 25 + + A_DATA = b'A' * 1024 * 1024 + B_DATA = b'B' * 1024 * 1024 + + sslctx = self._create_server_ssl_context( + test_utils.ONLYCERT, + test_utils.ONLYKEY + ) + client_sslctx = self._create_client_ssl_context() + + def server(sock): + sock.starttls( + sslctx, + server_side=True) + + data = sock.recv_all(len(A_DATA)) + self.assertEqual(data, A_DATA) + sock.send(b'OK') + + data = sock.recv_all(len(B_DATA)) + self.assertEqual(data, B_DATA) + sock.send(b'SPAM') + + sock.close() + + async def client(addr): + extras = {} + extras = dict(ssl_handshake_timeout=40.0) + + reader, writer = await asyncio.open_connection( + *addr, + ssl=client_sslctx, + server_hostname='', + **extras) + + writer.write(A_DATA) + self.assertEqual(await reader.readexactly(2), b'OK') + + writer.write(B_DATA) + self.assertEqual(await reader.readexactly(4), b'SPAM') + + nonlocal CNT + CNT += 1 + + writer.close() + await self.wait_closed(writer) + + async def client_sock(addr): + sock = socket.socket() + sock.connect(addr) + reader, writer = await asyncio.open_connection( + sock=sock, + ssl=client_sslctx, + server_hostname='') + + writer.write(A_DATA) + self.assertEqual(await reader.readexactly(2), b'OK') + + writer.write(B_DATA) + self.assertEqual(await reader.readexactly(4), b'SPAM') + + nonlocal CNT + CNT += 1 + + writer.close() + await self.wait_closed(writer) + sock.close() + + def run(coro): + nonlocal CNT + CNT = 0 + + async def _gather(*tasks): + # trampoline + return await asyncio.gather(*tasks) + + with self.tcp_server(server, + max_clients=TOTAL_CNT, + backlog=TOTAL_CNT) as srv: + tasks = [] + for _ in range(TOTAL_CNT): + tasks.append(coro(srv.addr)) + + self.loop.run_until_complete(_gather(*tasks)) + + self.assertEqual(CNT, TOTAL_CNT) + + with self._silence_eof_received_warning(): + run(client) + + with self._silence_eof_received_warning(): + run(client_sock) + + def test_create_connection_ssl_slow_handshake(self): + client_sslctx = self._create_client_ssl_context() + + # silence error logger + self.loop.set_exception_handler(lambda *args: None) + + def server(sock): + try: + sock.recv_all(1024 * 1024) + except ConnectionAbortedError: + pass + finally: + sock.close() + + async def client(addr): + reader, writer = await asyncio.open_connection( + *addr, + ssl=client_sslctx, + server_hostname='', + ssl_handshake_timeout=1.0) + writer.close() + await self.wait_closed(writer) + + with self.tcp_server(server, + max_clients=1, + backlog=1) as srv: + + with self.assertRaisesRegex( + ConnectionAbortedError, + r'SSL handshake.*is taking longer'): + + self.loop.run_until_complete(client(srv.addr)) + + def test_create_connection_ssl_failed_certificate(self): + # silence error logger + self.loop.set_exception_handler(lambda *args: None) + + sslctx = self._create_server_ssl_context( + test_utils.ONLYCERT, + test_utils.ONLYKEY + ) + client_sslctx = self._create_client_ssl_context(disable_verify=False) + + def server(sock): + try: + sock.starttls( + sslctx, + server_side=True) + sock.connect() + except (ssl.SSLError, OSError): + pass + finally: + sock.close() + + async def client(addr): + reader, writer = await asyncio.open_connection( + *addr, + ssl=client_sslctx, + server_hostname='', + ssl_handshake_timeout=1.0) + writer.close() + await self.wait_closed(writer) + + with self.tcp_server(server, + max_clients=1, + backlog=1) as srv: + + with self.assertRaises(ssl.SSLCertVerificationError): + self.loop.run_until_complete(client(srv.addr)) + + def test_ssl_handshake_timeout(self): + # bpo-29970: Check that a connection is aborted if handshake is not + # completed in timeout period, instead of remaining open indefinitely + client_sslctx = test_utils.simple_client_sslcontext() + + # silence error logger + messages = [] + self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx)) + + server_side_aborted = False + + def server(sock): + nonlocal server_side_aborted + try: + sock.recv_all(1024 * 1024) + except ConnectionAbortedError: + server_side_aborted = True + finally: + sock.close() + + async def client(addr): + await asyncio.wait_for( + self.loop.create_connection( + asyncio.Protocol, + *addr, + ssl=client_sslctx, + server_hostname='', + ssl_handshake_timeout=10.0), + 0.5) + + with self.tcp_server(server, + max_clients=1, + backlog=1) as srv: + + with self.assertRaises(asyncio.TimeoutError): + self.loop.run_until_complete(client(srv.addr)) + + self.assertTrue(server_side_aborted) + + # Python issue #23197: cancelling a handshake must not raise an + # exception or log an error, even if the handshake failed + self.assertEqual(messages, []) + + def test_ssl_handshake_connection_lost(self): + # #246: make sure that no connection_lost() is called before + # connection_made() is called first + + client_sslctx = test_utils.simple_client_sslcontext() + + # silence error logger + self.loop.set_exception_handler(lambda loop, ctx: None) + + connection_made_called = False + connection_lost_called = False + + def server(sock): + sock.recv(1024) + # break the connection during handshake + sock.close() + + class ClientProto(asyncio.Protocol): + def connection_made(self, transport): + nonlocal connection_made_called + connection_made_called = True + + def connection_lost(self, exc): + nonlocal connection_lost_called + connection_lost_called = True + + async def client(addr): + await self.loop.create_connection( + ClientProto, + *addr, + ssl=client_sslctx, + server_hostname=''), + + with self.tcp_server(server, + max_clients=1, + backlog=1) as srv: + + with self.assertRaises(ConnectionResetError): + self.loop.run_until_complete(client(srv.addr)) + + if connection_lost_called: + if connection_made_called: + self.fail("unexpected call to connection_lost()") + else: + self.fail("unexpected call to connection_lost() without" + "calling connection_made()") + elif connection_made_called: + self.fail("unexpected call to connection_made()") + + def test_ssl_connect_accepted_socket(self): + proto = ssl.PROTOCOL_TLS_SERVER + server_context = ssl.SSLContext(proto) + server_context.load_cert_chain(test_utils.ONLYCERT, test_utils.ONLYKEY) + if hasattr(server_context, 'check_hostname'): + server_context.check_hostname = False + server_context.verify_mode = ssl.CERT_NONE + + client_context = ssl.SSLContext(proto) + if hasattr(server_context, 'check_hostname'): + client_context.check_hostname = False + client_context.verify_mode = ssl.CERT_NONE + + def test_connect_accepted_socket(self, server_ssl=None, client_ssl=None): + loop = self.loop + + class MyProto(MyBaseProto): + + def connection_lost(self, exc): + super().connection_lost(exc) + loop.call_soon(loop.stop) + + def data_received(self, data): + super().data_received(data) + self.transport.write(expected_response) + + lsock = socket.socket(socket.AF_INET) + lsock.bind(('127.0.0.1', 0)) + lsock.listen(1) + addr = lsock.getsockname() + + message = b'test data' + response = None + expected_response = b'roger' + + def client(): + nonlocal response + try: + csock = socket.socket(socket.AF_INET) + if client_ssl is not None: + csock = client_ssl.wrap_socket(csock) + csock.connect(addr) + csock.sendall(message) + response = csock.recv(99) + csock.close() + except Exception as exc: + print( + "Failure in client thread in test_connect_accepted_socket", + exc) + + thread = threading.Thread(target=client, daemon=True) + thread.start() + + conn, _ = lsock.accept() + proto = MyProto(loop=loop) + proto.loop = loop + + extras = {} + if server_ssl: + extras = dict(ssl_handshake_timeout=10.0) + + f = loop.create_task( + loop.connect_accepted_socket( + (lambda: proto), conn, ssl=server_ssl, + **extras)) + loop.run_forever() + conn.close() + lsock.close() + + thread.join(1) + self.assertFalse(thread.is_alive()) + self.assertEqual(proto.state, 'CLOSED') + self.assertEqual(proto.nbytes, len(message)) + self.assertEqual(response, expected_response) + tr, _ = f.result() + + if server_ssl: + self.assertIn('SSL', tr.__class__.__name__) + + tr.close() + # let it close + self.loop.run_until_complete(asyncio.sleep(0.1)) + + def test_start_tls_client_corrupted_ssl(self): + self.loop.set_exception_handler(lambda loop, ctx: None) + + sslctx = test_utils.simple_server_sslcontext() + client_sslctx = test_utils.simple_client_sslcontext() + + def server(sock): + orig_sock = sock.dup() + try: + sock.starttls( + sslctx, + server_side=True) + sock.sendall(b'A\n') + sock.recv_all(1) + orig_sock.send(b'please corrupt the SSL connection') + except ssl.SSLError: + pass + finally: + sock.close() + orig_sock.close() + + async def client(addr): + reader, writer = await asyncio.open_connection( + *addr, + ssl=client_sslctx, + server_hostname='') + + self.assertEqual(await reader.readline(), b'A\n') + writer.write(b'B') + with self.assertRaises(ssl.SSLError): + await reader.readline() + writer.close() + try: + await self.wait_closed(writer) + except ssl.SSLError: + pass + return 'OK' + + with self.tcp_server(server, + max_clients=1, + backlog=1) as srv: + + res = self.loop.run_until_complete(client(srv.addr)) + + self.assertEqual(res, 'OK') + + def test_start_tls_client_reg_proto_1(self): + HELLO_MSG = b'1' * self.PAYLOAD_SIZE + + server_context = test_utils.simple_server_sslcontext() + client_context = test_utils.simple_client_sslcontext() + + def serve(sock): + sock.settimeout(self.TIMEOUT) + + data = sock.recv_all(len(HELLO_MSG)) + self.assertEqual(len(data), len(HELLO_MSG)) + + sock.starttls(server_context, server_side=True) + + sock.sendall(b'O') + data = sock.recv_all(len(HELLO_MSG)) + self.assertEqual(len(data), len(HELLO_MSG)) + + sock.unwrap() + sock.close() + + class ClientProto(asyncio.Protocol): + def __init__(self, on_data, on_eof): + self.on_data = on_data + self.on_eof = on_eof + self.con_made_cnt = 0 + + def connection_made(proto, tr): + proto.con_made_cnt += 1 + # Ensure connection_made gets called only once. + self.assertEqual(proto.con_made_cnt, 1) + + def data_received(self, data): + self.on_data.set_result(data) + + def eof_received(self): + self.on_eof.set_result(True) + + async def client(addr): + await asyncio.sleep(0.5) + + on_data = self.loop.create_future() + on_eof = self.loop.create_future() + + tr, proto = await self.loop.create_connection( + lambda: ClientProto(on_data, on_eof), *addr) + + tr.write(HELLO_MSG) + new_tr = await self.loop.start_tls(tr, proto, client_context) + + self.assertEqual(await on_data, b'O') + new_tr.write(HELLO_MSG) + await on_eof + + new_tr.close() + + with self.tcp_server(serve, timeout=self.TIMEOUT) as srv: + self.loop.run_until_complete( + asyncio.wait_for(client(srv.addr), timeout=10)) + + def test_create_connection_memory_leak(self): + HELLO_MSG = b'1' * self.PAYLOAD_SIZE + + server_context = self._create_server_ssl_context( + test_utils.ONLYCERT, test_utils.ONLYKEY) + client_context = self._create_client_ssl_context() + + def serve(sock): + sock.settimeout(self.TIMEOUT) + + sock.starttls(server_context, server_side=True) + + sock.sendall(b'O') + data = sock.recv_all(len(HELLO_MSG)) + self.assertEqual(len(data), len(HELLO_MSG)) + + sock.unwrap() + sock.close() + + class ClientProto(asyncio.Protocol): + def __init__(self, on_data, on_eof): + self.on_data = on_data + self.on_eof = on_eof + self.con_made_cnt = 0 + + def connection_made(proto, tr): + # XXX: We assume user stores the transport in protocol + proto.tr = tr + proto.con_made_cnt += 1 + # Ensure connection_made gets called only once. + self.assertEqual(proto.con_made_cnt, 1) + + def data_received(self, data): + self.on_data.set_result(data) + + def eof_received(self): + self.on_eof.set_result(True) + + async def client(addr): + await asyncio.sleep(0.5) + + on_data = self.loop.create_future() + on_eof = self.loop.create_future() + + tr, proto = await self.loop.create_connection( + lambda: ClientProto(on_data, on_eof), *addr, + ssl=client_context) + + self.assertEqual(await on_data, b'O') + tr.write(HELLO_MSG) + await on_eof + + tr.close() + + with self.tcp_server(serve, timeout=self.TIMEOUT) as srv: + self.loop.run_until_complete( + asyncio.wait_for(client(srv.addr), timeout=10)) + + # No garbage is left for SSL client from loop.create_connection, even + # if user stores the SSLTransport in corresponding protocol instance + client_context = weakref.ref(client_context) + self.assertIsNone(client_context()) + + def test_start_tls_client_buf_proto_1(self): + HELLO_MSG = b'1' * self.PAYLOAD_SIZE + + server_context = test_utils.simple_server_sslcontext() + client_context = test_utils.simple_client_sslcontext() + + client_con_made_calls = 0 + + def serve(sock): + sock.settimeout(self.TIMEOUT) + + data = sock.recv_all(len(HELLO_MSG)) + self.assertEqual(len(data), len(HELLO_MSG)) + + sock.starttls(server_context, server_side=True) + + sock.sendall(b'O') + data = sock.recv_all(len(HELLO_MSG)) + self.assertEqual(len(data), len(HELLO_MSG)) + + sock.sendall(b'2') + data = sock.recv_all(len(HELLO_MSG)) + self.assertEqual(len(data), len(HELLO_MSG)) + + sock.unwrap() + sock.close() + + class ClientProtoFirst(asyncio.BufferedProtocol): + def __init__(self, on_data): + self.on_data = on_data + self.buf = bytearray(1) + + def connection_made(self, tr): + nonlocal client_con_made_calls + client_con_made_calls += 1 + + def get_buffer(self, sizehint): + return self.buf + + def buffer_updated(self, nsize): + assert nsize == 1 + self.on_data.set_result(bytes(self.buf[:nsize])) + + def eof_received(self): + pass + + class ClientProtoSecond(asyncio.Protocol): + def __init__(self, on_data, on_eof): + self.on_data = on_data + self.on_eof = on_eof + self.con_made_cnt = 0 + + def connection_made(self, tr): + nonlocal client_con_made_calls + client_con_made_calls += 1 + + def data_received(self, data): + self.on_data.set_result(data) + + def eof_received(self): + self.on_eof.set_result(True) + + async def client(addr): + await asyncio.sleep(0.5) + + on_data1 = self.loop.create_future() + on_data2 = self.loop.create_future() + on_eof = self.loop.create_future() + + tr, proto = await self.loop.create_connection( + lambda: ClientProtoFirst(on_data1), *addr) + + tr.write(HELLO_MSG) + new_tr = await self.loop.start_tls(tr, proto, client_context) + + self.assertEqual(await on_data1, b'O') + new_tr.write(HELLO_MSG) + + new_tr.set_protocol(ClientProtoSecond(on_data2, on_eof)) + self.assertEqual(await on_data2, b'2') + new_tr.write(HELLO_MSG) + await on_eof + + new_tr.close() + + # connection_made() should be called only once -- when + # we establish connection for the first time. Start TLS + # doesn't call connection_made() on application protocols. + self.assertEqual(client_con_made_calls, 1) + + with self.tcp_server(serve, timeout=self.TIMEOUT) as srv: + self.loop.run_until_complete( + asyncio.wait_for(client(srv.addr), + timeout=self.TIMEOUT)) + + def test_start_tls_slow_client_cancel(self): + HELLO_MSG = b'1' * self.PAYLOAD_SIZE + + client_context = test_utils.simple_client_sslcontext() + server_waits_on_handshake = self.loop.create_future() + + def serve(sock): + sock.settimeout(self.TIMEOUT) + + data = sock.recv_all(len(HELLO_MSG)) + self.assertEqual(len(data), len(HELLO_MSG)) + + try: + self.loop.call_soon_threadsafe( + server_waits_on_handshake.set_result, None) + data = sock.recv_all(1024 * 1024) + except ConnectionAbortedError: + pass + finally: + sock.close() + + class ClientProto(asyncio.Protocol): + def __init__(self, on_data, on_eof): + self.on_data = on_data + self.on_eof = on_eof + self.con_made_cnt = 0 + + def connection_made(proto, tr): + proto.con_made_cnt += 1 + # Ensure connection_made gets called only once. + self.assertEqual(proto.con_made_cnt, 1) + + def data_received(self, data): + self.on_data.set_result(data) + + def eof_received(self): + self.on_eof.set_result(True) + + async def client(addr): + await asyncio.sleep(0.5) + + on_data = self.loop.create_future() + on_eof = self.loop.create_future() + + tr, proto = await self.loop.create_connection( + lambda: ClientProto(on_data, on_eof), *addr) + + tr.write(HELLO_MSG) + + await server_waits_on_handshake + + with self.assertRaises(asyncio.TimeoutError): + await asyncio.wait_for( + self.loop.start_tls(tr, proto, client_context), + 0.5) + + with self.tcp_server(serve, timeout=self.TIMEOUT) as srv: + self.loop.run_until_complete( + asyncio.wait_for(client(srv.addr), timeout=10)) + + def test_start_tls_server_1(self): + HELLO_MSG = b'1' * self.PAYLOAD_SIZE + + server_context = test_utils.simple_server_sslcontext() + client_context = test_utils.simple_client_sslcontext() + + def client(sock, addr): + sock.settimeout(self.TIMEOUT) + + sock.connect(addr) + data = sock.recv_all(len(HELLO_MSG)) + self.assertEqual(len(data), len(HELLO_MSG)) + + sock.starttls(client_context) + sock.sendall(HELLO_MSG) + + sock.unwrap() + sock.close() + + class ServerProto(asyncio.Protocol): + def __init__(self, on_con, on_eof, on_con_lost): + self.on_con = on_con + self.on_eof = on_eof + self.on_con_lost = on_con_lost + self.data = b'' + + def connection_made(self, tr): + self.on_con.set_result(tr) + + def data_received(self, data): + self.data += data + + def eof_received(self): + self.on_eof.set_result(1) + + def connection_lost(self, exc): + if exc is None: + self.on_con_lost.set_result(None) + else: + self.on_con_lost.set_exception(exc) + + async def main(proto, on_con, on_eof, on_con_lost): + tr = await on_con + tr.write(HELLO_MSG) + + self.assertEqual(proto.data, b'') + + new_tr = await self.loop.start_tls( + tr, proto, server_context, + server_side=True, + ssl_handshake_timeout=self.TIMEOUT) + + await on_eof + await on_con_lost + self.assertEqual(proto.data, HELLO_MSG) + new_tr.close() + + async def run_main(): + on_con = self.loop.create_future() + on_eof = self.loop.create_future() + on_con_lost = self.loop.create_future() + proto = ServerProto(on_con, on_eof, on_con_lost) + + server = await self.loop.create_server( + lambda: proto, '127.0.0.1', 0) + addr = server.sockets[0].getsockname() + + with self.tcp_client(lambda sock: client(sock, addr), + timeout=self.TIMEOUT): + await asyncio.wait_for( + main(proto, on_con, on_eof, on_con_lost), + timeout=self.TIMEOUT) + + server.close() + await server.wait_closed() + + self.loop.run_until_complete(run_main()) + + def test_create_server_ssl_over_ssl(self): + CNT = 0 # number of clients that were successful + TOTAL_CNT = 25 # total number of clients that test will create + TIMEOUT = 10.0 # timeout for this test + + A_DATA = b'A' * 1024 * 1024 + B_DATA = b'B' * 1024 * 1024 + + sslctx_1 = self._create_server_ssl_context( + test_utils.ONLYCERT, test_utils.ONLYKEY) + client_sslctx_1 = self._create_client_ssl_context() + sslctx_2 = self._create_server_ssl_context( + test_utils.ONLYCERT, test_utils.ONLYKEY) + client_sslctx_2 = self._create_client_ssl_context() + + clients = [] + + async def handle_client(reader, writer): + nonlocal CNT + + data = await reader.readexactly(len(A_DATA)) + self.assertEqual(data, A_DATA) + writer.write(b'OK') + + data = await reader.readexactly(len(B_DATA)) + self.assertEqual(data, B_DATA) + writer.writelines([b'SP', bytearray(b'A'), memoryview(b'M')]) + + await writer.drain() + writer.close() + + CNT += 1 + + class ServerProtocol(asyncio.StreamReaderProtocol): + def connection_made(self, transport): + super_ = super() + transport.pause_reading() + fut = self._loop.create_task(self._loop.start_tls( + transport, self, sslctx_2, server_side=True)) + + def cb(_): + try: + tr = fut.result() + except Exception as ex: + super_.connection_lost(ex) + else: + super_.connection_made(tr) + fut.add_done_callback(cb) + + def server_protocol_factory(): + reader = asyncio.StreamReader() + protocol = ServerProtocol(reader, handle_client) + return protocol + + async def test_client(addr): + fut = asyncio.Future() + + def prog(sock): + try: + sock.connect(addr) + sock.starttls(client_sslctx_1) + + # because wrap_socket() doesn't work correctly on + # SSLSocket, we have to do the 2nd level SSL manually + incoming = ssl.MemoryBIO() + outgoing = ssl.MemoryBIO() + sslobj = client_sslctx_2.wrap_bio(incoming, outgoing) + + def do(func, *args): + while True: + try: + rv = func(*args) + break + except ssl.SSLWantReadError: + if outgoing.pending: + sock.send(outgoing.read()) + incoming.write(sock.recv(65536)) + if outgoing.pending: + sock.send(outgoing.read()) + return rv + + do(sslobj.do_handshake) + + do(sslobj.write, A_DATA) + data = do(sslobj.read, 2) + self.assertEqual(data, b'OK') + + do(sslobj.write, B_DATA) + data = b'' + while True: + chunk = do(sslobj.read, 4) + if not chunk: + break + data += chunk + self.assertEqual(data, b'SPAM') + + do(sslobj.unwrap) + sock.close() + + except Exception as ex: + self.loop.call_soon_threadsafe(fut.set_exception, ex) + sock.close() + else: + self.loop.call_soon_threadsafe(fut.set_result, None) + + client = self.tcp_client(prog) + client.start() + clients.append(client) + + await fut + + async def start_server(): + extras = {} + + srv = await self.loop.create_server( + server_protocol_factory, + '127.0.0.1', 0, + family=socket.AF_INET, + ssl=sslctx_1, + **extras) + + try: + srv_socks = srv.sockets + self.assertTrue(srv_socks) + + addr = srv_socks[0].getsockname() + + tasks = [] + for _ in range(TOTAL_CNT): + tasks.append(test_client(addr)) + + await asyncio.wait_for(asyncio.gather(*tasks), TIMEOUT) + + finally: + self.loop.call_soon(srv.close) + await srv.wait_closed() + + with self._silence_eof_received_warning(): + self.loop.run_until_complete(start_server()) + + self.assertEqual(CNT, TOTAL_CNT) + + for client in clients: + client.stop() + + def test_shutdown_cleanly(self): + CNT = 0 + TOTAL_CNT = 25 + + A_DATA = b'A' * 1024 * 1024 + + sslctx = self._create_server_ssl_context( + test_utils.ONLYCERT, test_utils.ONLYKEY) + client_sslctx = self._create_client_ssl_context() + + def server(sock): + sock.starttls( + sslctx, + server_side=True) + + data = sock.recv_all(len(A_DATA)) + self.assertEqual(data, A_DATA) + sock.send(b'OK') + + sock.unwrap() + + sock.close() + + async def client(addr): + extras = {} + extras = dict(ssl_handshake_timeout=10.0) + + reader, writer = await asyncio.open_connection( + *addr, + ssl=client_sslctx, + server_hostname='', + **extras) + + writer.write(A_DATA) + self.assertEqual(await reader.readexactly(2), b'OK') + + self.assertEqual(await reader.read(), b'') + + nonlocal CNT + CNT += 1 + + writer.close() + await self.wait_closed(writer) + + def run(coro): + nonlocal CNT + CNT = 0 + + async def _gather(*tasks): + return await asyncio.gather(*tasks) + + with self.tcp_server(server, + max_clients=TOTAL_CNT, + backlog=TOTAL_CNT) as srv: + tasks = [] + for _ in range(TOTAL_CNT): + tasks.append(coro(srv.addr)) + + self.loop.run_until_complete( + _gather(*tasks)) + + self.assertEqual(CNT, TOTAL_CNT) + + with self._silence_eof_received_warning(): + run(client) + + def test_flush_before_shutdown(self): + CHUNK = 1024 * 128 + SIZE = 32 + + sslctx = self._create_server_ssl_context( + test_utils.ONLYCERT, test_utils.ONLYKEY) + client_sslctx = self._create_client_ssl_context() + + future = None + + def server(sock): + sock.starttls(sslctx, server_side=True) + self.assertEqual(sock.recv_all(4), b'ping') + sock.send(b'pong') + time.sleep(0.5) # hopefully stuck the TCP buffer + data = sock.recv_all(CHUNK * SIZE) + self.assertEqual(len(data), CHUNK * SIZE) + sock.close() + + def run(meth): + def wrapper(sock): + try: + meth(sock) + except Exception as ex: + self.loop.call_soon_threadsafe(future.set_exception, ex) + else: + self.loop.call_soon_threadsafe(future.set_result, None) + return wrapper + + async def client(addr): + nonlocal future + future = self.loop.create_future() + reader, writer = await asyncio.open_connection( + *addr, + ssl=client_sslctx, + server_hostname='') + sslprotocol = writer.transport._ssl_protocol + writer.write(b'ping') + data = await reader.readexactly(4) + self.assertEqual(data, b'pong') + + sslprotocol.pause_writing() + for _ in range(SIZE): + writer.write(b'x' * CHUNK) + + writer.close() + sslprotocol.resume_writing() + + await self.wait_closed(writer) + try: + data = await reader.read() + self.assertEqual(data, b'') + except ConnectionResetError: + pass + await future + + with self.tcp_server(run(server)) as srv: + self.loop.run_until_complete(client(srv.addr)) + + def test_remote_shutdown_receives_trailing_data(self): + CHUNK = 1024 * 128 + SIZE = 32 + + sslctx = self._create_server_ssl_context( + test_utils.ONLYCERT, + test_utils.ONLYKEY + ) + client_sslctx = self._create_client_ssl_context() + future = None + + def server(sock): + incoming = ssl.MemoryBIO() + outgoing = ssl.MemoryBIO() + sslobj = sslctx.wrap_bio(incoming, outgoing, server_side=True) + + while True: + try: + sslobj.do_handshake() + except ssl.SSLWantReadError: + if outgoing.pending: + sock.send(outgoing.read()) + incoming.write(sock.recv(16384)) + else: + if outgoing.pending: + sock.send(outgoing.read()) + break + + while True: + try: + data = sslobj.read(4) + except ssl.SSLWantReadError: + incoming.write(sock.recv(16384)) + else: + break + + self.assertEqual(data, b'ping') + sslobj.write(b'pong') + sock.send(outgoing.read()) + + time.sleep(0.2) # wait for the peer to fill its backlog + + # send close_notify but don't wait for response + with self.assertRaises(ssl.SSLWantReadError): + sslobj.unwrap() + sock.send(outgoing.read()) + + # should receive all data + data_len = 0 + while True: + try: + chunk = len(sslobj.read(16384)) + data_len += chunk + except ssl.SSLWantReadError: + incoming.write(sock.recv(16384)) + except ssl.SSLZeroReturnError: + break + + self.assertEqual(data_len, CHUNK * SIZE) + + # verify that close_notify is received + sslobj.unwrap() + + sock.close() + + def eof_server(sock): + sock.starttls(sslctx, server_side=True) + self.assertEqual(sock.recv_all(4), b'ping') + sock.send(b'pong') + + time.sleep(0.2) # wait for the peer to fill its backlog + + # send EOF + sock.shutdown(socket.SHUT_WR) + + # should receive all data + data = sock.recv_all(CHUNK * SIZE) + self.assertEqual(len(data), CHUNK * SIZE) + + sock.close() + + async def client(addr): + nonlocal future + future = self.loop.create_future() + + reader, writer = await asyncio.open_connection( + *addr, + ssl=client_sslctx, + server_hostname='') + writer.write(b'ping') + data = await reader.readexactly(4) + self.assertEqual(data, b'pong') + + # fill write backlog in a hacky way - renegotiation won't help + for _ in range(SIZE): + writer.transport._test__append_write_backlog(b'x' * CHUNK) + + try: + data = await reader.read() + self.assertEqual(data, b'') + except (BrokenPipeError, ConnectionResetError): + pass + + await future + + writer.close() + await self.wait_closed(writer) + + def run(meth): + def wrapper(sock): + try: + meth(sock) + except Exception as ex: + self.loop.call_soon_threadsafe(future.set_exception, ex) + else: + self.loop.call_soon_threadsafe(future.set_result, None) + return wrapper + + with self.tcp_server(run(server)) as srv: + self.loop.run_until_complete(client(srv.addr)) + + with self.tcp_server(run(eof_server)) as srv: + self.loop.run_until_complete(client(srv.addr)) + + def test_connect_timeout_warning(self): + s = socket.socket(socket.AF_INET) + s.bind(('127.0.0.1', 0)) + addr = s.getsockname() + + async def test(): + try: + await asyncio.wait_for( + self.loop.create_connection(asyncio.Protocol, + *addr, ssl=True), + 0.1) + except (ConnectionRefusedError, asyncio.TimeoutError): + pass + else: + self.fail('TimeoutError is not raised') + + with s: + try: + with self.assertWarns(ResourceWarning) as cm: + self.loop.run_until_complete(test()) + gc.collect() + gc.collect() + gc.collect() + except AssertionError as e: + self.assertEqual(str(e), 'ResourceWarning not triggered') + else: + self.fail('Unexpected ResourceWarning: {}'.format(cm.warning)) + + def test_handshake_timeout_handler_leak(self): + s = socket.socket(socket.AF_INET) + s.bind(('127.0.0.1', 0)) + s.listen(1) + addr = s.getsockname() + + async def test(ctx): + try: + await asyncio.wait_for( + self.loop.create_connection(asyncio.Protocol, *addr, + ssl=ctx), + 0.1) + except (ConnectionRefusedError, asyncio.TimeoutError): + pass + else: + self.fail('TimeoutError is not raised') + + with s: + ctx = ssl.create_default_context() + self.loop.run_until_complete(test(ctx)) + ctx = weakref.ref(ctx) + + # SSLProtocol should be DECREF to 0 + self.assertIsNone(ctx()) + + def test_shutdown_timeout_handler_leak(self): + loop = self.loop + + def server(sock): + sslctx = self._create_server_ssl_context( + test_utils.ONLYCERT, + test_utils.ONLYKEY + ) + sock = sslctx.wrap_socket(sock, server_side=True) + sock.recv(32) + sock.close() + + class Protocol(asyncio.Protocol): + def __init__(self): + self.fut = asyncio.Future(loop=loop) + + def connection_lost(self, exc): + self.fut.set_result(None) + + async def client(addr, ctx): + tr, pr = await loop.create_connection(Protocol, *addr, ssl=ctx) + tr.close() + await pr.fut + + with self.tcp_server(server) as srv: + ctx = self._create_client_ssl_context() + loop.run_until_complete(client(srv.addr, ctx)) + ctx = weakref.ref(ctx) + + # asyncio has no shutdown timeout, but it ends up with a circular + # reference loop - not ideal (introduces gc glitches), but at least + # not leaking + gc.collect() + gc.collect() + gc.collect() + + # SSLProtocol should be DECREF to 0 + self.assertIsNone(ctx()) + + def test_shutdown_timeout_handler_not_set(self): + loop = self.loop + eof = asyncio.Event() + extra = None + + def server(sock): + sslctx = self._create_server_ssl_context( + test_utils.ONLYCERT, + test_utils.ONLYKEY + ) + sock = sslctx.wrap_socket(sock, server_side=True) + sock.send(b'hello') + assert sock.recv(1024) == b'world' + sock.send(b'extra bytes') + # sending EOF here + sock.shutdown(socket.SHUT_WR) + loop.call_soon_threadsafe(eof.set) + # make sure we have enough time to reproduce the issue + assert sock.recv(1024) == b'' + sock.close() + + class Protocol(asyncio.Protocol): + def __init__(self): + self.fut = asyncio.Future(loop=loop) + self.transport = None + + def connection_made(self, transport): + self.transport = transport + + def data_received(self, data): + if data == b'hello': + self.transport.write(b'world') + # pause reading would make incoming data stay in the sslobj + self.transport.pause_reading() + else: + nonlocal extra + extra = data + + def connection_lost(self, exc): + if exc is None: + self.fut.set_result(None) + else: + self.fut.set_exception(exc) + + async def client(addr): + ctx = self._create_client_ssl_context() + tr, pr = await loop.create_connection(Protocol, *addr, ssl=ctx) + await eof.wait() + tr.resume_reading() + await pr.fut + tr.close() + assert extra == b'extra bytes' + + with self.tcp_server(server) as srv: + loop.run_until_complete(client(srv.addr)) + + +############################################################################### +# Socket Testing Utilities +############################################################################### + + +class TestSocketWrapper: + + def __init__(self, sock): + self.__sock = sock + + def recv_all(self, n): + buf = b'' + while len(buf) < n: + data = self.recv(n - len(buf)) + if data == b'': + raise ConnectionAbortedError + buf += data + return buf + + def starttls(self, ssl_context, *, + server_side=False, + server_hostname=None, + do_handshake_on_connect=True): + + assert isinstance(ssl_context, ssl.SSLContext) + + ssl_sock = ssl_context.wrap_socket( + self.__sock, server_side=server_side, + server_hostname=server_hostname, + do_handshake_on_connect=do_handshake_on_connect) + + if server_side: + ssl_sock.do_handshake() + + self.__sock.close() + self.__sock = ssl_sock + + def __getattr__(self, name): + return getattr(self.__sock, name) + + def __repr__(self): + return '<{} {!r}>'.format(type(self).__name__, self.__sock) + + +class SocketThread(threading.Thread): + + def stop(self): + self._active = False + self.join() + + def __enter__(self): + self.start() + return self + + def __exit__(self, *exc): + self.stop() + + +class TestThreadedClient(SocketThread): + + def __init__(self, test, sock, prog, timeout): + threading.Thread.__init__(self, None, None, 'test-client') + self.daemon = True + + self._timeout = timeout + self._sock = sock + self._active = True + self._prog = prog + self._test = test + + def run(self): + try: + self._prog(TestSocketWrapper(self._sock)) + except (KeyboardInterrupt, SystemExit): + raise + except BaseException as ex: + self._test._abort_socket_test(ex) + + +class TestThreadedServer(SocketThread): + + def __init__(self, test, sock, prog, timeout, max_clients): + threading.Thread.__init__(self, None, None, 'test-server') + self.daemon = True + + self._clients = 0 + self._finished_clients = 0 + self._max_clients = max_clients + self._timeout = timeout + self._sock = sock + self._active = True + + self._prog = prog + + self._s1, self._s2 = socket.socketpair() + self._s1.setblocking(False) + + self._test = test + + def stop(self): + try: + if self._s2 and self._s2.fileno() != -1: + try: + self._s2.send(b'stop') + except OSError: + pass + finally: + super().stop() + + def run(self): + try: + with self._sock: + self._sock.setblocking(0) + self._run() + finally: + self._s1.close() + self._s2.close() + + def _run(self): + while self._active: + if self._clients >= self._max_clients: + return + + r, w, x = select.select( + [self._sock, self._s1], [], [], self._timeout) + + if self._s1 in r: + return + + if self._sock in r: + try: + conn, addr = self._sock.accept() + except BlockingIOError: + continue + except socket.timeout: + if not self._active: + return + else: + raise + else: + self._clients += 1 + conn.settimeout(self._timeout) + try: + with conn: + self._handle_client(conn) + except (KeyboardInterrupt, SystemExit): + raise + except BaseException as ex: + self._active = False + try: + raise + finally: + self._test._abort_socket_test(ex) + + def _handle_client(self, sock): + self._prog(TestSocketWrapper(sock)) + + @property + def addr(self): + return self._sock.getsockname() diff --git a/Lib/test/test_asyncio/test_sslproto.py b/Lib/test/test_asyncio/test_sslproto.py index 22a216a83e3f9..6e8de7c84c0aa 100644 --- a/Lib/test/test_asyncio/test_sslproto.py +++ b/Lib/test/test_asyncio/test_sslproto.py @@ -15,7 +15,6 @@ from asyncio import log from asyncio import protocols from asyncio import sslproto -from test import support from test.test_asyncio import utils as test_utils from test.test_asyncio import functional as func_tests @@ -44,16 +43,13 @@ def ssl_protocol(self, *, waiter=None, proto=None): def connection_made(self, ssl_proto, *, do_handshake=None): transport = mock.Mock() - sslpipe = mock.Mock() - sslpipe.shutdown.return_value = b'' - if do_handshake: - sslpipe.do_handshake.side_effect = do_handshake - else: - def mock_handshake(callback): - return [] - sslpipe.do_handshake.side_effect = mock_handshake - with mock.patch('asyncio.sslproto._SSLPipe', return_value=sslpipe): - ssl_proto.connection_made(transport) + sslobj = mock.Mock() + # emulate reading decompressed data + sslobj.read.side_effect = ssl.SSLWantReadError + if do_handshake is not None: + sslobj.do_handshake = do_handshake + ssl_proto._sslobj = sslobj + ssl_proto.connection_made(transport) return transport def test_handshake_timeout_zero(self): @@ -75,7 +71,10 @@ def test_handshake_timeout_negative(self): def test_eof_received_waiter(self): waiter = self.loop.create_future() ssl_proto = self.ssl_protocol(waiter=waiter) - self.connection_made(ssl_proto) + self.connection_made( + ssl_proto, + do_handshake=mock.Mock(side_effect=ssl.SSLWantReadError) + ) ssl_proto.eof_received() test_utils.run_briefly(self.loop) self.assertIsInstance(waiter.exception(), ConnectionResetError) @@ -100,7 +99,10 @@ def test_connection_lost(self): # yield from waiter hang if lost_connection was called. waiter = self.loop.create_future() ssl_proto = self.ssl_protocol(waiter=waiter) - self.connection_made(ssl_proto) + self.connection_made( + ssl_proto, + do_handshake=mock.Mock(side_effect=ssl.SSLWantReadError) + ) ssl_proto.connection_lost(ConnectionAbortedError) test_utils.run_briefly(self.loop) self.assertIsInstance(waiter.exception(), ConnectionAbortedError) @@ -110,7 +112,10 @@ def test_close_during_handshake(self): waiter = self.loop.create_future() ssl_proto = self.ssl_protocol(waiter=waiter) - transport = self.connection_made(ssl_proto) + transport = self.connection_made( + ssl_proto, + do_handshake=mock.Mock(side_effect=ssl.SSLWantReadError) + ) test_utils.run_briefly(self.loop) ssl_proto._app_transport.close() @@ -143,7 +148,7 @@ def test_data_received_after_closing(self): transp.close() # should not raise - self.assertIsNone(ssl_proto.data_received(b'data')) + self.assertIsNone(ssl_proto.buffer_updated(5)) def test_write_after_closing(self): ssl_proto = self.ssl_protocol() diff --git a/Misc/NEWS.d/next/Library/2021-05-02-23-44-21.bpo-44011.hd8iUO.rst b/Misc/NEWS.d/next/Library/2021-05-02-23-44-21.bpo-44011.hd8iUO.rst new file mode 100644 index 0000000000000..1a48aa5ebae0c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-05-02-23-44-21.bpo-44011.hd8iUO.rst @@ -0,0 +1,2 @@ +Reimplement SSL/TLS support in asyncio, borrow the implementation from +uvloop library. From webhook-mailer at python.org Tue Feb 15 10:22:05 2022 From: webhook-mailer at python.org (Mariatta) Date: Tue, 15 Feb 2022 15:22:05 -0000 Subject: [Python-checkins] Fix copyright years in `README.rst` (#31347) Message-ID: https://github.com/python/cpython/commit/9a40a2d99f703402c81c0444aa6d369eef60847c commit: 9a40a2d99f703402c81c0444aa6d369eef60847c branch: main author: Nikita Sobolev committer: Mariatta date: 2022-02-15T07:22:00-08:00 summary: Fix copyright years in `README.rst` (#31347) files: M README.rst diff --git a/README.rst b/README.rst index e7bdd0f64efb5..bd80772393375 100644 --- a/README.rst +++ b/README.rst @@ -14,7 +14,7 @@ This is Python version 3.11.0 alpha 5 :target: https://discuss.python.org/ -Copyright ? 2001-2021 Python Software Foundation. All rights reserved. +Copyright ? 2001-2022 Python Software Foundation. All rights reserved. See the end of this file for further copyright and license information. @@ -244,7 +244,7 @@ Copyright and License Information --------------------------------- -Copyright ? 2001-2021 Python Software Foundation. All rights reserved. +Copyright ? 2001-2022 Python Software Foundation. All rights reserved. Copyright ? 2000 BeOpen.com. All rights reserved. From webhook-mailer at python.org Tue Feb 15 11:40:00 2022 From: webhook-mailer at python.org (asvetlov) Date: Tue, 15 Feb 2022 16:40:00 -0000 Subject: [Python-checkins] bpo-46741: Update `asyncio.protocols.BufferedProtocol` docstring (31327) Message-ID: https://github.com/python/cpython/commit/1d81fdc4c004511c25f74db0e04ddbbb8a04ce6d commit: 1d81fdc4c004511c25f74db0e04ddbbb8a04ce6d branch: main author: Alex Waygood committer: asvetlov date: 2022-02-15T18:39:28+02:00 summary: bpo-46741: Update `asyncio.protocols.BufferedProtocol` docstring (31327) The docstring for `BufferedProtocol` states that the class is still an "experimental API", but it has been considered stable since 3.8. files: M Lib/asyncio/protocols.py diff --git a/Lib/asyncio/protocols.py b/Lib/asyncio/protocols.py index 69fa43e8b6511..09987b164c66e 100644 --- a/Lib/asyncio/protocols.py +++ b/Lib/asyncio/protocols.py @@ -109,10 +109,6 @@ def eof_received(self): class BufferedProtocol(BaseProtocol): """Interface for stream protocol with manual buffer control. - Important: this has been added to asyncio in Python 3.7 - *on a provisional basis*! Consider it as an experimental API that - might be changed or removed in Python 3.8. - Event methods, such as `create_server` and `create_connection`, accept factories that return protocols that implement this interface. From webhook-mailer at python.org Tue Feb 15 12:08:41 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 15 Feb 2022 17:08:41 -0000 Subject: [Python-checkins] bpo-46741: Update `asyncio.protocols.BufferedProtocol` docstring (31327) Message-ID: https://github.com/python/cpython/commit/c292118ef3528df85a9d76ad21029009b560b088 commit: c292118ef3528df85a9d76ad21029009b560b088 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-15T09:08:24-08:00 summary: bpo-46741: Update `asyncio.protocols.BufferedProtocol` docstring (31327) The docstring for `BufferedProtocol` states that the class is still an "experimental API", but it has been considered stable since 3.8. (cherry picked from commit 1d81fdc4c004511c25f74db0e04ddbbb8a04ce6d) Co-authored-by: Alex Waygood files: M Lib/asyncio/protocols.py diff --git a/Lib/asyncio/protocols.py b/Lib/asyncio/protocols.py index 69fa43e8b6511..09987b164c66e 100644 --- a/Lib/asyncio/protocols.py +++ b/Lib/asyncio/protocols.py @@ -109,10 +109,6 @@ def eof_received(self): class BufferedProtocol(BaseProtocol): """Interface for stream protocol with manual buffer control. - Important: this has been added to asyncio in Python 3.7 - *on a provisional basis*! Consider it as an experimental API that - might be changed or removed in Python 3.8. - Event methods, such as `create_server` and `create_connection`, accept factories that return protocols that implement this interface. From webhook-mailer at python.org Tue Feb 15 12:14:18 2022 From: webhook-mailer at python.org (asvetlov) Date: Tue, 15 Feb 2022 17:14:18 -0000 Subject: [Python-checkins] bpo-46741: Update `asyncio.protocols.BufferedProtocol` docstring (GH-31327) (GH-31362) Message-ID: https://github.com/python/cpython/commit/72c46462aa9fd0a69317917ba127e62f94ce08d7 commit: 72c46462aa9fd0a69317917ba127e62f94ce08d7 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: asvetlov date: 2022-02-15T19:14:09+02:00 summary: bpo-46741: Update `asyncio.protocols.BufferedProtocol` docstring (GH-31327) (GH-31362) The docstring for `BufferedProtocol` states that the class is still an "experimental API", but it has been considered stable since 3.8. (cherry picked from commit 1d81fdc4c004511c25f74db0e04ddbbb8a04ce6d) Co-authored-by: Alex Waygood Co-authored-by: Alex Waygood files: M Lib/asyncio/protocols.py diff --git a/Lib/asyncio/protocols.py b/Lib/asyncio/protocols.py index 69fa43e8b6511..09987b164c66e 100644 --- a/Lib/asyncio/protocols.py +++ b/Lib/asyncio/protocols.py @@ -109,10 +109,6 @@ def eof_received(self): class BufferedProtocol(BaseProtocol): """Interface for stream protocol with manual buffer control. - Important: this has been added to asyncio in Python 3.7 - *on a provisional basis*! Consider it as an experimental API that - might be changed or removed in Python 3.8. - Event methods, such as `create_server` and `create_connection`, accept factories that return protocols that implement this interface. From webhook-mailer at python.org Tue Feb 15 12:17:08 2022 From: webhook-mailer at python.org (Mariatta) Date: Tue, 15 Feb 2022 17:17:08 -0000 Subject: [Python-checkins] Update the OMG link (GH-30383) Message-ID: https://github.com/python/cpython/commit/5b63ba3ff5ab56ebf0ed1173cf01dd23169e3dfe commit: 5b63ba3ff5ab56ebf0ed1173cf01dd23169e3dfe branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Mariatta date: 2022-02-15T09:17:00-08:00 summary: Update the OMG link (GH-30383) (cherry picked from commit 04215a427232e55efdc1adcda50a15197f518ca9) Co-authored-by: David Gilbertson Co-authored-by: David Gilbertson files: M Doc/library/xml.dom.rst diff --git a/Doc/library/xml.dom.rst b/Doc/library/xml.dom.rst index 98454e18ff1f8..b387240a3716c 100644 --- a/Doc/library/xml.dom.rst +++ b/Doc/library/xml.dom.rst @@ -73,7 +73,7 @@ implementations are free to support the strict mapping from IDL). See section `Document Object Model (DOM) Level 1 Specification `_ The W3C recommendation for the DOM supported by :mod:`xml.dom.minidom`. - `Python Language Mapping Specification `_ + `Python Language Mapping Specification `_ This specifies the mapping from OMG IDL to Python. From webhook-mailer at python.org Tue Feb 15 18:12:21 2022 From: webhook-mailer at python.org (rhettinger) Date: Tue, 15 Feb 2022 23:12:21 -0000 Subject: [Python-checkins] bpo-46737: Add default arguments to random.gauss and normalvariate (GH-31360) Message-ID: https://github.com/python/cpython/commit/08ec80113b3b7f7a9eaa3d217494536b63305181 commit: 08ec80113b3b7f7a9eaa3d217494536b63305181 branch: main author: Zackery Spytz committer: rhettinger date: 2022-02-15T17:12:15-06:00 summary: bpo-46737: Add default arguments to random.gauss and normalvariate (GH-31360) files: A Misc/NEWS.d/next/Library/2022-02-15-07-39-43.bpo-46737.6Pnblt.rst M Doc/library/random.rst M Lib/random.py M Lib/test/test_random.py diff --git a/Doc/library/random.rst b/Doc/library/random.rst index da4a4f61e4567..96c6300ea16f5 100644 --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -320,7 +320,7 @@ be found in any statistics text. math.gamma(alpha) * beta ** alpha -.. function:: gauss(mu, sigma) +.. function:: gauss(mu=0.0, sigma=1.0) Normal distribution, also called the Gaussian distribution. *mu* is the mean, and *sigma* is the standard deviation. This is slightly faster than @@ -333,6 +333,9 @@ be found in any statistics text. number generator. 2) Put locks around all calls. 3) Use the slower, but thread-safe :func:`normalvariate` function instead. + .. versionchanged:: 3.11 + *mu* and *sigma* now have default arguments. + .. function:: lognormvariate(mu, sigma) @@ -342,10 +345,13 @@ be found in any statistics text. zero. -.. function:: normalvariate(mu, sigma) +.. function:: normalvariate(mu=0.0, sigma=1.0) Normal distribution. *mu* is the mean, and *sigma* is the standard deviation. + .. versionchanged:: 3.11 + *mu* and *sigma* now have default arguments. + .. function:: vonmisesvariate(mu, kappa) diff --git a/Lib/random.py b/Lib/random.py index 6d7b617e33a30..1f3530e880fce 100644 --- a/Lib/random.py +++ b/Lib/random.py @@ -538,7 +538,7 @@ def triangular(self, low=0.0, high=1.0, mode=None): low, high = high, low return low + (high - low) * _sqrt(u * c) - def normalvariate(self, mu, sigma): + def normalvariate(self, mu=0.0, sigma=1.0): """Normal distribution. mu is the mean, and sigma is the standard deviation. @@ -559,7 +559,7 @@ def normalvariate(self, mu, sigma): break return mu + z * sigma - def gauss(self, mu, sigma): + def gauss(self, mu=0.0, sigma=1.0): """Gaussian distribution. mu is the mean, and sigma is the standard deviation. This is diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py index 5b066d23dd3fd..32e7868ba4de7 100644 --- a/Lib/test/test_random.py +++ b/Lib/test/test_random.py @@ -409,6 +409,10 @@ def test_randbytes(self): self.assertRaises(ValueError, self.gen.randbytes, -1) self.assertRaises(TypeError, self.gen.randbytes, 1.0) + def test_mu_sigma_default_args(self): + self.assertIsInstance(self.gen.normalvariate(), float) + self.assertIsInstance(self.gen.gauss(), float) + try: random.SystemRandom().random() diff --git a/Misc/NEWS.d/next/Library/2022-02-15-07-39-43.bpo-46737.6Pnblt.rst b/Misc/NEWS.d/next/Library/2022-02-15-07-39-43.bpo-46737.6Pnblt.rst new file mode 100644 index 0000000000000..c3f693aeb759e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-15-07-39-43.bpo-46737.6Pnblt.rst @@ -0,0 +1,2 @@ +:func:`random.gauss` and :func:`random.normalvariate` now have default +arguments. From webhook-mailer at python.org Tue Feb 15 18:42:24 2022 From: webhook-mailer at python.org (gvanrossum) Date: Tue, 15 Feb 2022 23:42:24 -0000 Subject: [Python-checkins] bpo-46752: Add TaskGroup; add Task..cancelled(),.uncancel() (GH-31270) Message-ID: https://github.com/python/cpython/commit/602630ac1855e38ef06361c68f6e216375a06180 commit: 602630ac1855e38ef06361c68f6e216375a06180 branch: main author: Guido van Rossum committer: gvanrossum date: 2022-02-15T15:42:04-08:00 summary: bpo-46752: Add TaskGroup; add Task..cancelled(),.uncancel() (GH-31270) asyncio/taskgroups.py is an adaptation of taskgroup.py from EdgeDb, with the following key changes: - Allow creating new tasks as long as the last task hasn't finished - Raise [Base]ExceptionGroup (directly) rather than TaskGroupError deriving from MultiError - Instead of monkey-patching the parent task's cancel() method, add a new public API to Task The Task class has a new internal flag, `_cancel_requested`, which is set when `.cancel()` is called successfully. The `.cancelling()` method returns the value of this flag. Further `.cancel()` calls while this flag is set return False. To reset this flag, call `.uncancel()`. Thus, a Task that catches and ignores `CancelledError` should call `.uncancel()` if it wants to be cancellable again; until it does so, it is deemed to be busy with uninterruptible cleanup. This new Task API helps solve the problem where TaskGroup needs to distinguish between whether the parent task being cancelled "from the outside" vs. "from inside". Co-authored-by: Yury Selivanov Co-authored-by: Andrew Svetlov files: A Lib/asyncio/taskgroups.py A Lib/test/test_asyncio/test_taskgroups.py A Misc/NEWS.d/next/Library/2022-02-14-21-21-49.bpo-46752.m6ldTm.rst M Lib/asyncio/__init__.py M Lib/asyncio/base_tasks.py M Lib/asyncio/tasks.py M Lib/test/test_asyncio/test_tasks.py M Modules/_asynciomodule.c M Modules/clinic/_asynciomodule.c.h diff --git a/Lib/asyncio/__init__.py b/Lib/asyncio/__init__.py index 200b14c2a3f21..db1124cc9bd1e 100644 --- a/Lib/asyncio/__init__.py +++ b/Lib/asyncio/__init__.py @@ -17,6 +17,7 @@ from .streams import * from .subprocess import * from .tasks import * +from .taskgroups import * from .threads import * from .transports import * diff --git a/Lib/asyncio/base_tasks.py b/Lib/asyncio/base_tasks.py index 09bb171a2ce75..1d623899f69a9 100644 --- a/Lib/asyncio/base_tasks.py +++ b/Lib/asyncio/base_tasks.py @@ -8,7 +8,7 @@ def _task_repr_info(task): info = base_futures._future_repr_info(task) - if task._must_cancel: + if task.cancelling() and not task.done(): # replace status info[0] = 'cancelling' diff --git a/Lib/asyncio/taskgroups.py b/Lib/asyncio/taskgroups.py new file mode 100644 index 0000000000000..718277892c51c --- /dev/null +++ b/Lib/asyncio/taskgroups.py @@ -0,0 +1,235 @@ +# Adapted with permission from the EdgeDB project. + + +__all__ = ["TaskGroup"] + +import itertools +import textwrap +import traceback +import types +import weakref + +from . import events +from . import exceptions +from . import tasks + +class TaskGroup: + + def __init__(self, *, name=None): + if name is None: + self._name = f'tg-{_name_counter()}' + else: + self._name = str(name) + + self._entered = False + self._exiting = False + self._aborting = False + self._loop = None + self._parent_task = None + self._parent_cancel_requested = False + self._tasks = weakref.WeakSet() + self._unfinished_tasks = 0 + self._errors = [] + self._base_error = None + self._on_completed_fut = None + + def get_name(self): + return self._name + + def __repr__(self): + msg = f' bool: + assert isinstance(exc, BaseException) + return isinstance(exc, (SystemExit, KeyboardInterrupt)) + + def _abort(self): + self._aborting = True + + for t in self._tasks: + if not t.done(): + t.cancel() + + def _on_task_done(self, task): + self._unfinished_tasks -= 1 + assert self._unfinished_tasks >= 0 + + if self._on_completed_fut is not None and not self._unfinished_tasks: + if not self._on_completed_fut.done(): + self._on_completed_fut.set_result(True) + + if task.cancelled(): + return + + exc = task.exception() + if exc is None: + return + + self._errors.append(exc) + if self._is_base_error(exc) and self._base_error is None: + self._base_error = exc + + if self._parent_task.done(): + # Not sure if this case is possible, but we want to handle + # it anyways. + self._loop.call_exception_handler({ + 'message': f'Task {task!r} has errored out but its parent ' + f'task {self._parent_task} is already completed', + 'exception': exc, + 'task': task, + }) + return + + self._abort() + if not self._parent_task.cancelling(): + # If parent task *is not* being cancelled, it means that we want + # to manually cancel it to abort whatever is being run right now + # in the TaskGroup. But we want to mark parent task as + # "not cancelled" later in __aexit__. Example situation that + # we need to handle: + # + # async def foo(): + # try: + # async with TaskGroup() as g: + # g.create_task(crash_soon()) + # await something # <- this needs to be canceled + # # by the TaskGroup, e.g. + # # foo() needs to be cancelled + # except Exception: + # # Ignore any exceptions raised in the TaskGroup + # pass + # await something_else # this line has to be called + # # after TaskGroup is finished. + self._parent_cancel_requested = True + self._parent_task.cancel() + + +_name_counter = itertools.count(1).__next__ diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py index 2bee5c050ded7..c11d0daaefea7 100644 --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -105,6 +105,7 @@ def __init__(self, coro, *, loop=None, name=None): else: self._name = str(name) + self._cancel_requested = False self._must_cancel = False self._fut_waiter = None self._coro = coro @@ -201,6 +202,9 @@ def cancel(self, msg=None): self._log_traceback = False if self.done(): return False + if self._cancel_requested: + return False + self._cancel_requested = True if self._fut_waiter is not None: if self._fut_waiter.cancel(msg=msg): # Leave self._fut_waiter; it may be a Task that @@ -212,6 +216,16 @@ def cancel(self, msg=None): self._cancel_message = msg return True + def cancelling(self): + return self._cancel_requested + + def uncancel(self): + if self._cancel_requested: + self._cancel_requested = False + return True + else: + return False + def __step(self, exc=None): if self.done(): raise exceptions.InvalidStateError( @@ -634,7 +648,7 @@ def _ensure_future(coro_or_future, *, loop=None): loop = events._get_event_loop(stacklevel=4) try: return loop.create_task(coro_or_future) - except RuntimeError: + except RuntimeError: if not called_wrap_awaitable: coro_or_future.close() raise diff --git a/Lib/test/test_asyncio/test_taskgroups.py b/Lib/test/test_asyncio/test_taskgroups.py new file mode 100644 index 0000000000000..ea6ee2ed43d2f --- /dev/null +++ b/Lib/test/test_asyncio/test_taskgroups.py @@ -0,0 +1,694 @@ +# Adapted with permission from the EdgeDB project. + + +import asyncio + +from asyncio import taskgroups +import unittest + + +# To prevent a warning "test altered the execution environment" +def tearDownModule(): + asyncio.set_event_loop_policy(None) + + +class MyExc(Exception): + pass + + +class MyBaseExc(BaseException): + pass + + +def get_error_types(eg): + return {type(exc) for exc in eg.exceptions} + + +class TestTaskGroup(unittest.IsolatedAsyncioTestCase): + + async def test_taskgroup_01(self): + + async def foo1(): + await asyncio.sleep(0.1) + return 42 + + async def foo2(): + await asyncio.sleep(0.2) + return 11 + + async with taskgroups.TaskGroup() as g: + t1 = g.create_task(foo1()) + t2 = g.create_task(foo2()) + + self.assertEqual(t1.result(), 42) + self.assertEqual(t2.result(), 11) + + async def test_taskgroup_02(self): + + async def foo1(): + await asyncio.sleep(0.1) + return 42 + + async def foo2(): + await asyncio.sleep(0.2) + return 11 + + async with taskgroups.TaskGroup() as g: + t1 = g.create_task(foo1()) + await asyncio.sleep(0.15) + t2 = g.create_task(foo2()) + + self.assertEqual(t1.result(), 42) + self.assertEqual(t2.result(), 11) + + async def test_taskgroup_03(self): + + async def foo1(): + await asyncio.sleep(1) + return 42 + + async def foo2(): + await asyncio.sleep(0.2) + return 11 + + async with taskgroups.TaskGroup() as g: + t1 = g.create_task(foo1()) + await asyncio.sleep(0.15) + # cancel t1 explicitly, i.e. everything should continue + # working as expected. + t1.cancel() + + t2 = g.create_task(foo2()) + + self.assertTrue(t1.cancelled()) + self.assertEqual(t2.result(), 11) + + async def test_taskgroup_04(self): + + NUM = 0 + t2_cancel = False + t2 = None + + async def foo1(): + await asyncio.sleep(0.1) + 1 / 0 + + async def foo2(): + nonlocal NUM, t2_cancel + try: + await asyncio.sleep(1) + except asyncio.CancelledError: + t2_cancel = True + raise + NUM += 1 + + async def runner(): + nonlocal NUM, t2 + + async with taskgroups.TaskGroup() as g: + g.create_task(foo1()) + t2 = g.create_task(foo2()) + + NUM += 10 + + with self.assertRaises(ExceptionGroup) as cm: + await asyncio.create_task(runner()) + + self.assertEqual(get_error_types(cm.exception), {ZeroDivisionError}) + + self.assertEqual(NUM, 0) + self.assertTrue(t2_cancel) + self.assertTrue(t2.cancelled()) + + async def test_taskgroup_05(self): + + NUM = 0 + t2_cancel = False + runner_cancel = False + + async def foo1(): + await asyncio.sleep(0.1) + 1 / 0 + + async def foo2(): + nonlocal NUM, t2_cancel + try: + await asyncio.sleep(5) + except asyncio.CancelledError: + t2_cancel = True + raise + NUM += 1 + + async def runner(): + nonlocal NUM, runner_cancel + + async with taskgroups.TaskGroup() as g: + g.create_task(foo1()) + g.create_task(foo1()) + g.create_task(foo1()) + g.create_task(foo2()) + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + runner_cancel = True + raise + + NUM += 10 + + # The 3 foo1 sub tasks can be racy when the host is busy - if the + # cancellation happens in the middle, we'll see partial sub errors here + with self.assertRaises(ExceptionGroup) as cm: + await asyncio.create_task(runner()) + + self.assertEqual(get_error_types(cm.exception), {ZeroDivisionError}) + self.assertEqual(NUM, 0) + self.assertTrue(t2_cancel) + self.assertTrue(runner_cancel) + + async def test_taskgroup_06(self): + + NUM = 0 + + async def foo(): + nonlocal NUM + try: + await asyncio.sleep(5) + except asyncio.CancelledError: + NUM += 1 + raise + + async def runner(): + async with taskgroups.TaskGroup() as g: + for _ in range(5): + g.create_task(foo()) + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.1) + + self.assertFalse(r.done()) + r.cancel() + with self.assertRaises(asyncio.CancelledError): + await r + + self.assertEqual(NUM, 5) + + async def test_taskgroup_07(self): + + NUM = 0 + + async def foo(): + nonlocal NUM + try: + await asyncio.sleep(5) + except asyncio.CancelledError: + NUM += 1 + raise + + async def runner(): + nonlocal NUM + async with taskgroups.TaskGroup() as g: + for _ in range(5): + g.create_task(foo()) + + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + NUM += 10 + raise + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.1) + + self.assertFalse(r.done()) + r.cancel() + with self.assertRaises(asyncio.CancelledError): + await r + + self.assertEqual(NUM, 15) + + async def test_taskgroup_08(self): + + async def foo(): + await asyncio.sleep(0.1) + 1 / 0 + + async def runner(): + async with taskgroups.TaskGroup() as g: + for _ in range(5): + g.create_task(foo()) + + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + raise + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.1) + + self.assertFalse(r.done()) + r.cancel() + with self.assertRaises(asyncio.CancelledError): + await r + + async def test_taskgroup_09(self): + + t1 = t2 = None + + async def foo1(): + await asyncio.sleep(1) + return 42 + + async def foo2(): + await asyncio.sleep(2) + return 11 + + async def runner(): + nonlocal t1, t2 + async with taskgroups.TaskGroup() as g: + t1 = g.create_task(foo1()) + t2 = g.create_task(foo2()) + await asyncio.sleep(0.1) + 1 / 0 + + try: + await runner() + except ExceptionGroup as t: + self.assertEqual(get_error_types(t), {ZeroDivisionError}) + else: + self.fail('ExceptionGroup was not raised') + + self.assertTrue(t1.cancelled()) + self.assertTrue(t2.cancelled()) + + async def test_taskgroup_10(self): + + t1 = t2 = None + + async def foo1(): + await asyncio.sleep(1) + return 42 + + async def foo2(): + await asyncio.sleep(2) + return 11 + + async def runner(): + nonlocal t1, t2 + async with taskgroups.TaskGroup() as g: + t1 = g.create_task(foo1()) + t2 = g.create_task(foo2()) + 1 / 0 + + try: + await runner() + except ExceptionGroup as t: + self.assertEqual(get_error_types(t), {ZeroDivisionError}) + else: + self.fail('ExceptionGroup was not raised') + + self.assertTrue(t1.cancelled()) + self.assertTrue(t2.cancelled()) + + async def test_taskgroup_11(self): + + async def foo(): + await asyncio.sleep(0.1) + 1 / 0 + + async def runner(): + async with taskgroups.TaskGroup(): + async with taskgroups.TaskGroup() as g2: + for _ in range(5): + g2.create_task(foo()) + + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + raise + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.1) + + self.assertFalse(r.done()) + r.cancel() + with self.assertRaises(asyncio.CancelledError): + await r + + async def test_taskgroup_12(self): + + async def foo(): + await asyncio.sleep(0.1) + 1 / 0 + + async def runner(): + async with taskgroups.TaskGroup() as g1: + g1.create_task(asyncio.sleep(10)) + + async with taskgroups.TaskGroup() as g2: + for _ in range(5): + g2.create_task(foo()) + + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + raise + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.1) + + self.assertFalse(r.done()) + r.cancel() + with self.assertRaises(asyncio.CancelledError): + await r + + async def test_taskgroup_13(self): + + async def crash_after(t): + await asyncio.sleep(t) + raise ValueError(t) + + async def runner(): + async with taskgroups.TaskGroup(name='g1') as g1: + g1.create_task(crash_after(0.1)) + + async with taskgroups.TaskGroup(name='g2') as g2: + g2.create_task(crash_after(0.2)) + + r = asyncio.create_task(runner()) + with self.assertRaises(ExceptionGroup) as cm: + await r + + self.assertEqual(get_error_types(cm.exception), {ValueError}) + + async def test_taskgroup_14(self): + + async def crash_after(t): + await asyncio.sleep(t) + raise ValueError(t) + + async def runner(): + async with taskgroups.TaskGroup(name='g1') as g1: + g1.create_task(crash_after(10)) + + async with taskgroups.TaskGroup(name='g2') as g2: + g2.create_task(crash_after(0.1)) + + r = asyncio.create_task(runner()) + with self.assertRaises(ExceptionGroup) as cm: + await r + + self.assertEqual(get_error_types(cm.exception), {ExceptionGroup}) + self.assertEqual(get_error_types(cm.exception.exceptions[0]), {ValueError}) + + async def test_taskgroup_15(self): + + async def crash_soon(): + await asyncio.sleep(0.3) + 1 / 0 + + async def runner(): + async with taskgroups.TaskGroup(name='g1') as g1: + g1.create_task(crash_soon()) + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + await asyncio.sleep(0.5) + raise + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.1) + + self.assertFalse(r.done()) + r.cancel() + with self.assertRaises(asyncio.CancelledError): + await r + + async def test_taskgroup_16(self): + + async def crash_soon(): + await asyncio.sleep(0.3) + 1 / 0 + + async def nested_runner(): + async with taskgroups.TaskGroup(name='g1') as g1: + g1.create_task(crash_soon()) + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + await asyncio.sleep(0.5) + raise + + async def runner(): + t = asyncio.create_task(nested_runner()) + await t + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.1) + + self.assertFalse(r.done()) + r.cancel() + with self.assertRaises(asyncio.CancelledError): + await r + + async def test_taskgroup_17(self): + NUM = 0 + + async def runner(): + nonlocal NUM + async with taskgroups.TaskGroup(): + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + NUM += 10 + raise + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.1) + + self.assertFalse(r.done()) + r.cancel() + with self.assertRaises(asyncio.CancelledError): + await r + + self.assertEqual(NUM, 10) + + async def test_taskgroup_18(self): + NUM = 0 + + async def runner(): + nonlocal NUM + async with taskgroups.TaskGroup(): + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + NUM += 10 + # This isn't a good idea, but we have to support + # this weird case. + raise MyExc + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.1) + + self.assertFalse(r.done()) + r.cancel() + + try: + await r + except ExceptionGroup as t: + self.assertEqual(get_error_types(t),{MyExc}) + else: + self.fail('ExceptionGroup was not raised') + + self.assertEqual(NUM, 10) + + async def test_taskgroup_19(self): + async def crash_soon(): + await asyncio.sleep(0.1) + 1 / 0 + + async def nested(): + try: + await asyncio.sleep(10) + finally: + raise MyExc + + async def runner(): + async with taskgroups.TaskGroup() as g: + g.create_task(crash_soon()) + await nested() + + r = asyncio.create_task(runner()) + try: + await r + except ExceptionGroup as t: + self.assertEqual(get_error_types(t), {MyExc, ZeroDivisionError}) + else: + self.fail('TasgGroupError was not raised') + + async def test_taskgroup_20(self): + async def crash_soon(): + await asyncio.sleep(0.1) + 1 / 0 + + async def nested(): + try: + await asyncio.sleep(10) + finally: + raise KeyboardInterrupt + + async def runner(): + async with taskgroups.TaskGroup() as g: + g.create_task(crash_soon()) + await nested() + + with self.assertRaises(KeyboardInterrupt): + await runner() + + async def test_taskgroup_20a(self): + async def crash_soon(): + await asyncio.sleep(0.1) + 1 / 0 + + async def nested(): + try: + await asyncio.sleep(10) + finally: + raise MyBaseExc + + async def runner(): + async with taskgroups.TaskGroup() as g: + g.create_task(crash_soon()) + await nested() + + with self.assertRaises(BaseExceptionGroup) as cm: + await runner() + + self.assertEqual( + get_error_types(cm.exception), {MyBaseExc, ZeroDivisionError} + ) + + async def _test_taskgroup_21(self): + # This test doesn't work as asyncio, currently, doesn't + # correctly propagate KeyboardInterrupt (or SystemExit) -- + # those cause the event loop itself to crash. + # (Compare to the previous (passing) test -- that one raises + # a plain exception but raises KeyboardInterrupt in nested(); + # this test does it the other way around.) + + async def crash_soon(): + await asyncio.sleep(0.1) + raise KeyboardInterrupt + + async def nested(): + try: + await asyncio.sleep(10) + finally: + raise TypeError + + async def runner(): + async with taskgroups.TaskGroup() as g: + g.create_task(crash_soon()) + await nested() + + with self.assertRaises(KeyboardInterrupt): + await runner() + + async def test_taskgroup_21a(self): + + async def crash_soon(): + await asyncio.sleep(0.1) + raise MyBaseExc + + async def nested(): + try: + await asyncio.sleep(10) + finally: + raise TypeError + + async def runner(): + async with taskgroups.TaskGroup() as g: + g.create_task(crash_soon()) + await nested() + + with self.assertRaises(BaseExceptionGroup) as cm: + await runner() + + self.assertEqual(get_error_types(cm.exception), {MyBaseExc, TypeError}) + + async def test_taskgroup_22(self): + + async def foo1(): + await asyncio.sleep(1) + return 42 + + async def foo2(): + await asyncio.sleep(2) + return 11 + + async def runner(): + async with taskgroups.TaskGroup() as g: + g.create_task(foo1()) + g.create_task(foo2()) + + r = asyncio.create_task(runner()) + await asyncio.sleep(0.05) + r.cancel() + + with self.assertRaises(asyncio.CancelledError): + await r + + async def test_taskgroup_23(self): + + async def do_job(delay): + await asyncio.sleep(delay) + + async with taskgroups.TaskGroup() as g: + for count in range(10): + await asyncio.sleep(0.1) + g.create_task(do_job(0.3)) + if count == 5: + self.assertLess(len(g._tasks), 5) + await asyncio.sleep(1.35) + self.assertEqual(len(g._tasks), 0) + + async def test_taskgroup_24(self): + + async def root(g): + await asyncio.sleep(0.1) + g.create_task(coro1(0.1)) + g.create_task(coro1(0.2)) + + async def coro1(delay): + await asyncio.sleep(delay) + + async def runner(): + async with taskgroups.TaskGroup() as g: + g.create_task(root(g)) + + await runner() + + async def test_taskgroup_25(self): + nhydras = 0 + + async def hydra(g): + nonlocal nhydras + nhydras += 1 + await asyncio.sleep(0.01) + g.create_task(hydra(g)) + g.create_task(hydra(g)) + + async def hercules(): + while nhydras < 10: + await asyncio.sleep(0.015) + 1 / 0 + + async def runner(): + async with taskgroups.TaskGroup() as g: + g.create_task(hydra(g)) + g.create_task(hercules()) + + with self.assertRaises(ExceptionGroup) as cm: + await runner() + + self.assertEqual(get_error_types(cm.exception), {ZeroDivisionError}) + self.assertGreaterEqual(nhydras, 10) diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index 8c4dceacdeec9..fe6bfb363f1c6 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -496,6 +496,51 @@ async def run(): # This also distinguishes from the initial has_cycle=None. self.assertEqual(has_cycle, False) + + def test_cancelling(self): + loop = asyncio.new_event_loop() + + async def task(): + await asyncio.sleep(10) + + try: + t = self.new_task(loop, task()) + self.assertFalse(t.cancelling()) + self.assertNotIn(" cancelling ", repr(t)) + self.assertTrue(t.cancel()) + self.assertTrue(t.cancelling()) + self.assertIn(" cancelling ", repr(t)) + self.assertFalse(t.cancel()) + + with self.assertRaises(asyncio.CancelledError): + loop.run_until_complete(t) + finally: + loop.close() + + def test_uncancel(self): + loop = asyncio.new_event_loop() + + async def task(): + try: + await asyncio.sleep(10) + except asyncio.CancelledError: + asyncio.current_task().uncancel() + await asyncio.sleep(10) + + try: + t = self.new_task(loop, task()) + loop.run_until_complete(asyncio.sleep(0.01)) + self.assertTrue(t.cancel()) # Cancel first sleep + self.assertIn(" cancelling ", repr(t)) + loop.run_until_complete(asyncio.sleep(0.01)) + self.assertNotIn(" cancelling ", repr(t)) # after .uncancel() + self.assertTrue(t.cancel()) # Cancel second sleep + + with self.assertRaises(asyncio.CancelledError): + loop.run_until_complete(t) + finally: + loop.close() + def test_cancel(self): def gen(): diff --git a/Misc/NEWS.d/next/Library/2022-02-14-21-21-49.bpo-46752.m6ldTm.rst b/Misc/NEWS.d/next/Library/2022-02-14-21-21-49.bpo-46752.m6ldTm.rst new file mode 100644 index 0000000000000..f460600c8c9dd --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-14-21-21-49.bpo-46752.m6ldTm.rst @@ -0,0 +1,2 @@ +Add task groups to asyncio (structured concurrency, inspired by Trio's nurseries). +This also introduces a change to task cancellation, where a cancelled task can't be cancelled again until it calls .uncancel(). diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 72dbdb8902f12..6725e2eba79bc 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -91,6 +91,7 @@ typedef struct { PyObject *task_context; int task_must_cancel; int task_log_destroy_pending; + int task_cancel_requested; } TaskObj; typedef struct { @@ -2039,6 +2040,7 @@ _asyncio_Task___init___impl(TaskObj *self, PyObject *coro, PyObject *loop, Py_CLEAR(self->task_fut_waiter); self->task_must_cancel = 0; self->task_log_destroy_pending = 1; + self->task_cancel_requested = 0; Py_INCREF(coro); Py_XSETREF(self->task_coro, coro); @@ -2205,6 +2207,11 @@ _asyncio_Task_cancel_impl(TaskObj *self, PyObject *msg) Py_RETURN_FALSE; } + if (self->task_cancel_requested) { + Py_RETURN_FALSE; + } + self->task_cancel_requested = 1; + if (self->task_fut_waiter) { PyObject *res; int is_true; @@ -2232,6 +2239,56 @@ _asyncio_Task_cancel_impl(TaskObj *self, PyObject *msg) Py_RETURN_TRUE; } +/*[clinic input] +_asyncio.Task.cancelling + +Return True if the task is in the process of being cancelled. + +This is set once .cancel() is called +and remains set until .uncancel() is called. + +As long as this flag is set, further .cancel() calls will be ignored, +until .uncancel() is called to reset it. +[clinic start generated code]*/ + +static PyObject * +_asyncio_Task_cancelling_impl(TaskObj *self) +/*[clinic end generated code: output=803b3af96f917d7e input=c50e50f9c3ca4676]*/ +/*[clinic end generated code]*/ +{ + if (self->task_cancel_requested) { + Py_RETURN_TRUE; + } + else { + Py_RETURN_FALSE; + } +} + +/*[clinic input] +_asyncio.Task.uncancel + +Reset the flag returned by cancelling(). + +This should be used by tasks that catch CancelledError +and wish to continue indefinitely until they are cancelled again. + +Returns the previous value of the flag. +[clinic start generated code]*/ + +static PyObject * +_asyncio_Task_uncancel_impl(TaskObj *self) +/*[clinic end generated code: output=58184d236a817d3c input=5db95e28fcb6f7cd]*/ +/*[clinic end generated code]*/ +{ + if (self->task_cancel_requested) { + self->task_cancel_requested = 0; + Py_RETURN_TRUE; + } + else { + Py_RETURN_FALSE; + } +} + /*[clinic input] _asyncio.Task.get_stack @@ -2455,6 +2512,8 @@ static PyMethodDef TaskType_methods[] = { _ASYNCIO_TASK_SET_RESULT_METHODDEF _ASYNCIO_TASK_SET_EXCEPTION_METHODDEF _ASYNCIO_TASK_CANCEL_METHODDEF + _ASYNCIO_TASK_CANCELLING_METHODDEF + _ASYNCIO_TASK_UNCANCEL_METHODDEF _ASYNCIO_TASK_GET_STACK_METHODDEF _ASYNCIO_TASK_PRINT_STACK_METHODDEF _ASYNCIO_TASK__MAKE_CANCELLED_ERROR_METHODDEF diff --git a/Modules/clinic/_asynciomodule.c.h b/Modules/clinic/_asynciomodule.c.h index c472e652fb7c5..5648e14f337f7 100644 --- a/Modules/clinic/_asynciomodule.c.h +++ b/Modules/clinic/_asynciomodule.c.h @@ -447,6 +447,53 @@ _asyncio_Task_cancel(TaskObj *self, PyObject *const *args, Py_ssize_t nargs, PyO return return_value; } +PyDoc_STRVAR(_asyncio_Task_cancelling__doc__, +"cancelling($self, /)\n" +"--\n" +"\n" +"Return True if the task is in the process of being cancelled.\n" +"\n" +"This is set once .cancel() is called\n" +"and remains set until .uncancel() is called.\n" +"\n" +"As long as this flag is set, further .cancel() calls will be ignored,\n" +"until .uncancel() is called to reset it."); + +#define _ASYNCIO_TASK_CANCELLING_METHODDEF \ + {"cancelling", (PyCFunction)_asyncio_Task_cancelling, METH_NOARGS, _asyncio_Task_cancelling__doc__}, + +static PyObject * +_asyncio_Task_cancelling_impl(TaskObj *self); + +static PyObject * +_asyncio_Task_cancelling(TaskObj *self, PyObject *Py_UNUSED(ignored)) +{ + return _asyncio_Task_cancelling_impl(self); +} + +PyDoc_STRVAR(_asyncio_Task_uncancel__doc__, +"uncancel($self, /)\n" +"--\n" +"\n" +"Reset the flag returned by cancelling().\n" +"\n" +"This should be used by tasks that catch CancelledError\n" +"and wish to continue indefinitely until they are cancelled again.\n" +"\n" +"Returns the previous value of the flag."); + +#define _ASYNCIO_TASK_UNCANCEL_METHODDEF \ + {"uncancel", (PyCFunction)_asyncio_Task_uncancel, METH_NOARGS, _asyncio_Task_uncancel__doc__}, + +static PyObject * +_asyncio_Task_uncancel_impl(TaskObj *self); + +static PyObject * +_asyncio_Task_uncancel(TaskObj *self, PyObject *Py_UNUSED(ignored)) +{ + return _asyncio_Task_uncancel_impl(self); +} + PyDoc_STRVAR(_asyncio_Task_get_stack__doc__, "get_stack($self, /, *, limit=None)\n" "--\n" @@ -871,4 +918,4 @@ _asyncio__leave_task(PyObject *module, PyObject *const *args, Py_ssize_t nargs, exit: return return_value; } -/*[clinic end generated code: output=0d127162ac92e0c0 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=c02708a9d6a774cc input=a9049054013a1b77]*/ From webhook-mailer at python.org Tue Feb 15 22:06:43 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Wed, 16 Feb 2022 03:06:43 -0000 Subject: [Python-checkins] bpo-46541: Drop the check for orphaned global strings. (gh-31363) Message-ID: https://github.com/python/cpython/commit/6c8958948666403f2370ca7b4c0a52b2010ec16d commit: 6c8958948666403f2370ca7b4c0a52b2010ec16d branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-15T20:06:38-07:00 summary: bpo-46541: Drop the check for orphaned global strings. (gh-31363) https://bugs.python.org/issue46541 files: M .github/workflows/build.yml M Makefile.pre.in M Tools/scripts/generate_global_objects.py diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 376359ccf2e2d..5d36dffa80108 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -100,8 +100,6 @@ jobs: run: make smelly - name: Check limited ABI symbols run: make check-limited-abi - - name: Check global objects - run: make check-global-objects build_win32: name: 'Windows (x86)' diff --git a/Makefile.pre.in b/Makefile.pre.in index 9b08929dd67c2..2464bbdf2ddd0 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -2436,9 +2436,6 @@ patchcheck: @DEF_MAKE_RULE@ check-limited-abi: all $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/stable_abi.py --all $(srcdir)/Misc/stable_abi.txt -check-global-objects: all - $(RUNSHARED) ./$(BUILDPYTHON) $(srcdir)/Tools/scripts/generate_global_objects.py --check - .PHONY: update-config update-config: curl -sL -o config.guess 'https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD' diff --git a/Tools/scripts/generate_global_objects.py b/Tools/scripts/generate_global_objects.py index e989f3c086f34..7a5f42f0d3238 100644 --- a/Tools/scripts/generate_global_objects.py +++ b/Tools/scripts/generate_global_objects.py @@ -252,147 +252,10 @@ def generate_runtime_init(identifiers, strings): printer.write(after) -####################################### -# checks - -def err(msg): - print(msg, file=sys.stderr) - - -GETTER_RE = re.compile(r''' - ^ - .*? - (?: - (?: - _Py_ID - [(] - ( \w+ ) # - [)] - ) - | - (?: - _Py_STR - [(] - ( \w+ ) # - [)] - ) - ) -''', re.VERBOSE) -TYPESLOTS_RE = re.compile(r''' - ^ - .*? - (?: - (?: - SLOT0 [(] .*?, \s* - ( \w+ ) # - [)] - ) - | - (?: - SLOT1 [(] .*?, \s* - ( \w+ ) # - , .* [)] - ) - | - (?: - SLOT1BIN [(] .*?, .*?, \s* - ( \w+ ) # - , \s* - ( \w+ ) # - [)] - ) - | - (?: - SLOT1BINFULL [(] .*?, .*?, .*?, \s* - ( \w+ ) # - , \s* - ( \w+ ) # - [)] - ) - | - ( SLOT \d .* [^)] $ ) # - ) -''', re.VERBOSE) - -def check_orphan_strings(identifiers): - literals = set(n for n, s in STRING_LITERALS.items() if s) - identifiers = set(identifiers) - files = glob.iglob(os.path.join(ROOT, '**', '*.[ch]'), recursive=True) - for i, filename in enumerate(files, start=1): - print('.', end='') - if i % 5 == 0: - print(' ', end='') - if i % 20 == 0: - print() - if i % 100 == 0: - print() - with open(filename) as infile: - wrapped = None - for line in infile: - identifier = literal = reverse = None - - line = line.splitlines()[0] - if wrapped: - line = f'{wrapped.rstrip()} {line}' - wrapped = None - - if os.path.basename(filename) == '_warnings.c': - m = re.match(r'^.* = GET_WARNINGS_ATTR[(][^,]*, (\w+),', line) - if m: - identifier, = m.groups() - elif os.path.basename(filename) == 'typeobject.c': - m = TYPESLOTS_RE.match(line) - if m: - (slot0, - slot1, - slot1bin, reverse, - slot1binfull, fullreverse, - wrapped, - ) = m.groups() - identifier = slot0 or slot1 or slot1bin or slot1binfull - reverse = reverse or fullreverse - - if not identifier and not literal: - m = GETTER_RE.match(line) - if not m: - continue - identifier, literal = m.groups() - - if literal: - if literals and literal in literals: - literals.remove(literal) - if identifier: - if identifiers and identifier in identifiers: - identifiers.remove(identifier) - if reverse: - if identifiers and reverse in identifiers: - identifiers.remove(reverse) - if not literals and not identifiers: - break - else: - continue - break - if i % 20: - print() - if not literals and not identifiers: - return - print('ERROR:', file=sys.stderr) - if literals: - err(' unused global string literals:') - for name in sorted(literals): - err(f' {name}') - if identifiers: - if literals: - print() - err(' unused global identifiers:') - for name in sorted(identifiers): - err(f' {name}') - - ####################################### # the script -def main(*, check=False) -> None: +def main() -> None: identifiers = set(IDENTIFIERS) strings = dict(STRING_LITERALS) for name, string, filename, lno, _ in iter_global_strings(): @@ -408,13 +271,9 @@ def main(*, check=False) -> None: generate_global_strings(identifiers, strings) generate_runtime_init(identifiers, strings) - if check: - check_orphan_strings(identifiers) - if __name__ == '__main__': import argparse parser = argparse.ArgumentParser() - parser.add_argument('--check', action='store_true') args = parser.parse_args() main(**vars(args)) From webhook-mailer at python.org Tue Feb 15 22:07:15 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Wed, 16 Feb 2022 03:07:15 -0000 Subject: [Python-checkins] bpo-46541: Scan Fewer Files in generate_global_objects.py (gh-31364) Message-ID: https://github.com/python/cpython/commit/4d8a515d193a4c9f3844704f974ddb870d7ee383 commit: 4d8a515d193a4c9f3844704f974ddb870d7ee383 branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-15T20:07:11-07:00 summary: bpo-46541: Scan Fewer Files in generate_global_objects.py (gh-31364) https://bugs.python.org/issue46541 files: M Tools/scripts/generate_global_objects.py diff --git a/Tools/scripts/generate_global_objects.py b/Tools/scripts/generate_global_objects.py index 7a5f42f0d3238..bad7865f1ff83 100644 --- a/Tools/scripts/generate_global_objects.py +++ b/Tools/scripts/generate_global_objects.py @@ -100,24 +100,28 @@ ####################################### # helpers +def iter_files(): + for name in ('Modules', 'Objects', 'Parser', 'PC', 'Programs', 'Python'): + root = os.path.join(ROOT, name) + for dirname, _, files in os.walk(root): + for name in files: + if not name.endswith(('.c', '.h')): + continue + yield os.path.join(dirname, name) + + def iter_global_strings(): id_regex = re.compile(r'\b_Py_ID\((\w+)\)') str_regex = re.compile(r'\b_Py_DECLARE_STR\((\w+), "(.*?)"\)') - for dirname, _, files in os.walk(ROOT): - if os.path.relpath(dirname, ROOT).startswith('Include'): - continue - for name in files: - if not name.endswith(('.c', '.h')): - continue - filename = os.path.join(dirname, name) - with open(os.path.join(filename), encoding='utf-8') as infile: - for lno, line in enumerate(infile, 1): - for m in id_regex.finditer(line): - identifier, = m.groups() - yield identifier, None, filename, lno, line - for m in str_regex.finditer(line): - varname, string = m.groups() - yield varname, string, filename, lno, line + for filename in iter_files(): + with open(filename, encoding='utf-8') as infile: + for lno, line in enumerate(infile, 1): + for m in id_regex.finditer(line): + identifier, = m.groups() + yield identifier, None, filename, lno, line + for m in str_regex.finditer(line): + varname, string = m.groups() + yield varname, string, filename, lno, line def iter_to_marker(lines, marker): for line in lines: From webhook-mailer at python.org Wed Feb 16 02:07:45 2022 From: webhook-mailer at python.org (sweeneyde) Date: Wed, 16 Feb 2022 07:07:45 -0000 Subject: [Python-checkins] bpo-46730: Add more info to @property AttributeError messages (GH-31311) Message-ID: https://github.com/python/cpython/commit/0cb765b2cec9b020224af016a83bf35c45b71932 commit: 0cb765b2cec9b020224af016a83bf35c45b71932 branch: main author: Alex-Blade <44120047+Alex-Blade at users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2022-02-16T02:07:34-05:00 summary: bpo-46730: Add more info to @property AttributeError messages (GH-31311) On `obj.read_only_property = x`, raise `AttributeError: property 'read_only_property' of 'A' object has no setter`. files: A Misc/NEWS.d/next/Core and Builtins/2022-02-14-21-04-43.bpo-46730.rYJ1w5.rst M Doc/howto/descriptor.rst M Lib/test/test_property.py M Misc/ACKS M Objects/descrobject.c diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index f8b1e00d96fad..4f6389e3b2d48 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -991,17 +991,17 @@ here is a pure Python equivalent: if obj is None: return self if self.fget is None: - raise AttributeError(f'unreadable attribute {self._name}') + raise AttributeError(f"property '{self._name}' has no getter") return self.fget(obj) def __set__(self, obj, value): if self.fset is None: - raise AttributeError(f"can't set attribute {self._name}") + raise AttributeError(f"property '{self._name}' has no setter") self.fset(obj, value) def __delete__(self, obj): if self.fdel is None: - raise AttributeError(f"can't delete attribute {self._name}") + raise AttributeError(f"property '{self._name}' has no deleter") self.fdel(obj) def getter(self, fget): @@ -1456,7 +1456,7 @@ attributes stored in ``__slots__``: >>> mark.dept = 'Space Pirate' Traceback (most recent call last): ... - AttributeError: can't set attribute + AttributeError: property 'dept' of 'Immutable' object has no setter >>> mark.location = 'Mars' Traceback (most recent call last): ... diff --git a/Lib/test/test_property.py b/Lib/test/test_property.py index 7f3813fc8cd15..7d1c4a1e12880 100644 --- a/Lib/test/test_property.py +++ b/Lib/test/test_property.py @@ -322,27 +322,27 @@ def setUpClass(cls): cls.obj = cls.cls() def test_get_property(self): - with self.assertRaisesRegex(AttributeError, self._format_exc_msg("unreadable attribute")): + with self.assertRaisesRegex(AttributeError, self._format_exc_msg("has no getter")): self.obj.foo def test_set_property(self): - with self.assertRaisesRegex(AttributeError, self._format_exc_msg("can't set attribute")): + with self.assertRaisesRegex(AttributeError, self._format_exc_msg("has no setter")): self.obj.foo = None def test_del_property(self): - with self.assertRaisesRegex(AttributeError, self._format_exc_msg("can't delete attribute")): + with self.assertRaisesRegex(AttributeError, self._format_exc_msg("has no deleter")): del self.obj.foo class PropertyUnreachableAttributeWithName(_PropertyUnreachableAttribute, unittest.TestCase): - msg_format = "^{} 'foo'$" + msg_format = r"^property 'foo' of 'PropertyUnreachableAttributeWithName\.cls' object {}$" class cls: foo = property() class PropertyUnreachableAttributeNoName(_PropertyUnreachableAttribute, unittest.TestCase): - msg_format = "^{}$" + msg_format = "^property of 'PropertyUnreachableAttributeNoName\.cls' object {}$" class cls: pass diff --git a/Misc/ACKS b/Misc/ACKS index 6df339c3e145b..64bd91d7d6905 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -985,6 +985,7 @@ Erno Kuusela Ross Lagerwall Cameron Laird Lo?c Lajeanne +Alexander Lakeev David Lam Thomas Lamb Valerie Lambert diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-14-21-04-43.bpo-46730.rYJ1w5.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-14-21-04-43.bpo-46730.rYJ1w5.rst new file mode 100644 index 0000000000000..473b595545370 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-14-21-04-43.bpo-46730.rYJ1w5.rst @@ -0,0 +1,3 @@ +Message of AttributeError caused by getting, setting or deleting a property +without the corresponding function now mentions that the attribute is in fact +a property and also specifies type of the class that it belongs to. diff --git a/Objects/descrobject.c b/Objects/descrobject.c index 95162683edd7d..9379ad65de43e 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -1463,17 +1463,17 @@ class property(object): if inst is None: return self if self.__get is None: - raise AttributeError, "unreadable attribute" + raise AttributeError, "property has no getter" return self.__get(inst) def __set__(self, inst, value): if self.__set is None: - raise AttributeError, "can't set attribute" + raise AttributeError, "property has no setter" return self.__set(inst, value) def __delete__(self, inst): if self.__del is None: - raise AttributeError, "can't delete attribute" + raise AttributeError, "property has no deleter" return self.__del(inst) */ @@ -1586,9 +1586,15 @@ property_descr_get(PyObject *self, PyObject *obj, PyObject *type) propertyobject *gs = (propertyobject *)self; if (gs->prop_get == NULL) { if (gs->prop_name != NULL) { - PyErr_Format(PyExc_AttributeError, "unreadable attribute %R", gs->prop_name); - } else { - PyErr_SetString(PyExc_AttributeError, "unreadable attribute"); + PyErr_Format(PyExc_AttributeError, + "property %R of %R object has no getter", + gs->prop_name, + PyType_GetQualName(Py_TYPE(obj))); + } + else { + PyErr_Format(PyExc_AttributeError, + "property of %R object has no getter", + PyType_GetQualName(Py_TYPE(obj))); } return NULL; @@ -1611,18 +1617,26 @@ property_descr_set(PyObject *self, PyObject *obj, PyObject *value) } if (func == NULL) { - if (gs->prop_name != NULL) { + if (gs->prop_name != NULL && obj != NULL) { PyErr_Format(PyExc_AttributeError, value == NULL ? - "can't delete attribute %R" : - "can't set attribute %R", - gs->prop_name); + "property %R of %R object has no deleter" : + "property %R of %R object has no setter", + gs->prop_name, + PyType_GetQualName(Py_TYPE(obj))); + } + else if (obj != NULL) { + PyErr_Format(PyExc_AttributeError, + value == NULL ? + "property of %R object has no deleter" : + "property of %R object has no setter", + PyType_GetQualName(Py_TYPE(obj))); } else { PyErr_SetString(PyExc_AttributeError, - value == NULL ? - "can't delete attribute" : - "can't set attribute"); + value == NULL ? + "property has no deleter" : + "property has no setter"); } return -1; } From webhook-mailer at python.org Wed Feb 16 04:57:53 2022 From: webhook-mailer at python.org (corona10) Date: Wed, 16 Feb 2022 09:57:53 -0000 Subject: [Python-checkins] bpo-46541: Remove usage of _Py_IDENTIFIER from dbms modules (GH-31358) Message-ID: https://github.com/python/cpython/commit/e59309b9d0969d5c8f493cb8df28afa6f38d6273 commit: e59309b9d0969d5c8f493cb8df28afa6f38d6273 branch: main author: Dong-hee Na committer: corona10 date: 2022-02-16T18:57:28+09:00 summary: bpo-46541: Remove usage of _Py_IDENTIFIER from dbms modules (GH-31358) files: M Modules/_dbmmodule.c M Modules/_gdbmmodule.c diff --git a/Modules/_dbmmodule.c b/Modules/_dbmmodule.c index 65778fa77bfec..de8b2304ed8f1 100644 --- a/Modules/_dbmmodule.c +++ b/Modules/_dbmmodule.c @@ -3,7 +3,6 @@ #define PY_SSIZE_T_CLEAN -#define NEEDS_PY_IDENTIFIER #include "Python.h" #include @@ -396,8 +395,7 @@ dbm__enter__(PyObject *self, PyObject *args) static PyObject * dbm__exit__(PyObject *self, PyObject *args) { - _Py_IDENTIFIER(close); - return _PyObject_CallMethodIdNoArgs(self, &PyId_close); + return _dbm_dbm_close_impl((dbmobject *)self); } static PyMethodDef dbm_methods[] = { diff --git a/Modules/_gdbmmodule.c b/Modules/_gdbmmodule.c index 2d690c0abfd08..e6440fa9cd364 100644 --- a/Modules/_gdbmmodule.c +++ b/Modules/_gdbmmodule.c @@ -4,7 +4,6 @@ /* Doc strings: Mitch Chapman */ #define PY_SSIZE_T_CLEAN -#define NEEDS_PY_IDENTIFIER #include "Python.h" #include "gdbm.h" @@ -545,8 +544,7 @@ gdbm__enter__(PyObject *self, PyObject *args) static PyObject * gdbm__exit__(PyObject *self, PyObject *args) { - _Py_IDENTIFIER(close); - return _PyObject_CallMethodIdNoArgs(self, &PyId_close); + return _gdbm_gdbm_close_impl((gdbmobject *)self); } static PyMethodDef gdbm_methods[] = { From webhook-mailer at python.org Wed Feb 16 05:54:20 2022 From: webhook-mailer at python.org (ericvsmith) Date: Wed, 16 Feb 2022 10:54:20 -0000 Subject: [Python-checkins] bpo-46762: Fix an assert failure in f-strings where > or < is the last character if the f-string is missing a trailing right brace. (#31365) Message-ID: https://github.com/python/cpython/commit/ffd9f8ff84ed53c956b16d027f7d2926ea631051 commit: ffd9f8ff84ed53c956b16d027f7d2926ea631051 branch: main author: Eric V. Smith committer: ericvsmith date: 2022-02-16T05:54:09-05:00 summary: bpo-46762: Fix an assert failure in f-strings where > or < is the last character if the f-string is missing a trailing right brace. (#31365) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst M Lib/test/test_fstring.py M Parser/string_parser.c diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index d0b1ade15137b..0c255c2af2217 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -1053,6 +1053,8 @@ def test_mismatched_braces(self): "f'{{{'", "f'{{}}{'", "f'{'", + "f'x{<'", # See bpo-46762. + "f'x{>'", ]) # But these are just normal strings. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst new file mode 100644 index 0000000000000..cd53eb4ffaddd --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst @@ -0,0 +1,2 @@ +Fix an assert failure in debug builds when a '<', '>', or '=' is the last +character in an f-string that's missing a closing right brace. diff --git a/Parser/string_parser.c b/Parser/string_parser.c index 0b5e30ba2ca6a..fae2a3648cf9f 100644 --- a/Parser/string_parser.c +++ b/Parser/string_parser.c @@ -666,12 +666,12 @@ fstring_find_expr(Parser *p, const char **str, const char *end, int raw, int rec *str += 1; continue; } - /* Don't get out of the loop for these, if they're single - chars (not part of 2-char tokens). If by themselves, they - don't end an expression (unlike say '!'). */ - if (ch == '>' || ch == '<') { - continue; - } + } + /* Don't get out of the loop for these, if they're single + chars (not part of 2-char tokens). If by themselves, they + don't end an expression (unlike say '!'). */ + if (ch == '>' || ch == '<') { + continue; } /* Normal way out of this loop. */ @@ -698,10 +698,10 @@ fstring_find_expr(Parser *p, const char **str, const char *end, int raw, int rec } } expr_end = *str; - /* If we leave this loop in a string or with mismatched parens, we - don't care. We'll get a syntax error when compiling the - expression. But, we can produce a better error message, so - let's just do that.*/ + /* If we leave the above loop in a string or with mismatched parens, we + don't really care. We'll get a syntax error when compiling the + expression. But, we can produce a better error message, so let's just + do that.*/ if (quote_char) { RAISE_SYNTAX_ERROR("f-string: unterminated string"); goto error; From webhook-mailer at python.org Wed Feb 16 06:16:12 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 16 Feb 2022 11:16:12 -0000 Subject: [Python-checkins] bpo-46762: Fix an assert failure in f-strings where > or < is the last character if the f-string is missing a trailing right brace. (GH-31365) Message-ID: https://github.com/python/cpython/commit/3d407b931156bc3ee5ae131de2f2babad831ccb2 commit: 3d407b931156bc3ee5ae131de2f2babad831ccb2 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-16T03:16:05-08:00 summary: bpo-46762: Fix an assert failure in f-strings where > or < is the last character if the f-string is missing a trailing right brace. (GH-31365) (cherry picked from commit ffd9f8ff84ed53c956b16d027f7d2926ea631051) Co-authored-by: Eric V. Smith files: A Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst M Lib/test/test_fstring.py M Parser/string_parser.c diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index d0b1ade15137b..0c255c2af2217 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -1053,6 +1053,8 @@ def test_mismatched_braces(self): "f'{{{'", "f'{{}}{'", "f'{'", + "f'x{<'", # See bpo-46762. + "f'x{>'", ]) # But these are just normal strings. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst new file mode 100644 index 0000000000000..cd53eb4ffaddd --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst @@ -0,0 +1,2 @@ +Fix an assert failure in debug builds when a '<', '>', or '=' is the last +character in an f-string that's missing a closing right brace. diff --git a/Parser/string_parser.c b/Parser/string_parser.c index 4c2043521c0cd..dac8dbb846471 100644 --- a/Parser/string_parser.c +++ b/Parser/string_parser.c @@ -666,12 +666,12 @@ fstring_find_expr(Parser *p, const char **str, const char *end, int raw, int rec *str += 1; continue; } - /* Don't get out of the loop for these, if they're single - chars (not part of 2-char tokens). If by themselves, they - don't end an expression (unlike say '!'). */ - if (ch == '>' || ch == '<') { - continue; - } + } + /* Don't get out of the loop for these, if they're single + chars (not part of 2-char tokens). If by themselves, they + don't end an expression (unlike say '!'). */ + if (ch == '>' || ch == '<') { + continue; } /* Normal way out of this loop. */ @@ -698,10 +698,10 @@ fstring_find_expr(Parser *p, const char **str, const char *end, int raw, int rec } } expr_end = *str; - /* If we leave this loop in a string or with mismatched parens, we - don't care. We'll get a syntax error when compiling the - expression. But, we can produce a better error message, so - let's just do that.*/ + /* If we leave the above loop in a string or with mismatched parens, we + don't really care. We'll get a syntax error when compiling the + expression. But, we can produce a better error message, so let's just + do that.*/ if (quote_char) { RAISE_SYNTAX_ERROR("f-string: unterminated string"); goto error; From webhook-mailer at python.org Wed Feb 16 06:18:22 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 16 Feb 2022 11:18:22 -0000 Subject: [Python-checkins] bpo-46762: Fix an assert failure in f-strings where > or < is the last character if the f-string is missing a trailing right brace. (GH-31365) Message-ID: https://github.com/python/cpython/commit/a657bff34945e40be24cd75d02560a93b7623cf5 commit: a657bff34945e40be24cd75d02560a93b7623cf5 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-16T03:18:16-08:00 summary: bpo-46762: Fix an assert failure in f-strings where > or < is the last character if the f-string is missing a trailing right brace. (GH-31365) (cherry picked from commit ffd9f8ff84ed53c956b16d027f7d2926ea631051) Co-authored-by: Eric V. Smith files: A Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst M Lib/test/test_fstring.py M Parser/pegen/parse_string.c diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index 92a4d22062f98..b62887f8f0099 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -1060,6 +1060,8 @@ def test_mismatched_braces(self): "f'{{{'", "f'{{}}{'", "f'{'", + "f'x{<'", # See bpo-46762. + "f'x{>'", ]) # But these are just normal strings. diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst new file mode 100644 index 0000000000000..cd53eb4ffaddd --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-15-20-26-46.bpo-46762.1H7vab.rst @@ -0,0 +1,2 @@ +Fix an assert failure in debug builds when a '<', '>', or '=' is the last +character in an f-string that's missing a closing right brace. diff --git a/Parser/pegen/parse_string.c b/Parser/pegen/parse_string.c index af350b340db68..15a132b4e0544 100644 --- a/Parser/pegen/parse_string.c +++ b/Parser/pegen/parse_string.c @@ -668,12 +668,12 @@ fstring_find_expr(Parser *p, const char **str, const char *end, int raw, int rec *str += 1; continue; } - /* Don't get out of the loop for these, if they're single - chars (not part of 2-char tokens). If by themselves, they - don't end an expression (unlike say '!'). */ - if (ch == '>' || ch == '<') { - continue; - } + } + /* Don't get out of the loop for these, if they're single + chars (not part of 2-char tokens). If by themselves, they + don't end an expression (unlike say '!'). */ + if (ch == '>' || ch == '<') { + continue; } /* Normal way out of this loop. */ @@ -700,10 +700,10 @@ fstring_find_expr(Parser *p, const char **str, const char *end, int raw, int rec } } expr_end = *str; - /* If we leave this loop in a string or with mismatched parens, we - don't care. We'll get a syntax error when compiling the - expression. But, we can produce a better error message, so - let's just do that.*/ + /* If we leave the above loop in a string or with mismatched parens, we + don't really care. We'll get a syntax error when compiling the + expression. But, we can produce a better error message, so let's just + do that.*/ if (quote_char) { RAISE_SYNTAX_ERROR("f-string: unterminated string"); goto error; From webhook-mailer at python.org Wed Feb 16 06:26:18 2022 From: webhook-mailer at python.org (markshannon) Date: Wed, 16 Feb 2022 11:26:18 -0000 Subject: [Python-checkins] [3.10] bpo-46724: Use `JUMP_ABSOLUTE` for all backward jumps. (GH-31326) (GH-31354) Message-ID: https://github.com/python/cpython/commit/d4e4ef107a9fea257981d7701f023177b704a44f commit: d4e4ef107a9fea257981d7701f023177b704a44f branch: 3.10 author: Mark Shannon committer: markshannon date: 2022-02-16T11:26:02Z summary: [3.10] bpo-46724: Use `JUMP_ABSOLUTE` for all backward jumps. (GH-31326) (GH-31354) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-14-14-44-06.bpo-46724.jym_K6.rst M Lib/test/test_dis.py M Python/compile.c M Python/importlib.h M Python/importlib_external.h M Python/importlib_zipimport.h diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index 000549f5b6f11..23a171853636f 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -1044,7 +1044,7 @@ def jumpy(): Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=34, starts_line=None, is_jump_target=False), Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=21, argval=42, argrepr='to 42', offset=36, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=38, starts_line=8, is_jump_target=False), - Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=26, argval=52, argrepr='to 52', offset=40, starts_line=None, is_jump_target=False), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=5, argval=52, argrepr='to 52', offset=40, starts_line=None, is_jump_target=False), Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=4, argval=8, argrepr='to 8', offset=42, starts_line=7, is_jump_target=True), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=44, starts_line=10, is_jump_target=True), Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='I can haz else clause?', argrepr="'I can haz else clause?'", offset=46, starts_line=None, is_jump_target=False), @@ -1069,7 +1069,7 @@ def jumpy(): Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=84, starts_line=None, is_jump_target=False), Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=86, starts_line=None, is_jump_target=False), Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=46, argval=92, argrepr='to 92', offset=88, starts_line=None, is_jump_target=False), - Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=52, argval=104, argrepr='to 104', offset=90, starts_line=17, is_jump_target=False), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=6, argval=104, argrepr='to 104', offset=90, starts_line=17, is_jump_target=False), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=92, starts_line=11, is_jump_target=True), Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=28, argval=56, argrepr='to 56', offset=94, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=96, starts_line=19, is_jump_target=True), diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-14-14-44-06.bpo-46724.jym_K6.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-14-14-44-06.bpo-46724.jym_K6.rst new file mode 100644 index 0000000000000..7324182677abf --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-14-14-44-06.bpo-46724.jym_K6.rst @@ -0,0 +1,2 @@ +Make sure that all backwards jumps use the ``JUMP_ABSOLUTE`` instruction, rather +than ``JUMP_FORWARD`` with an argument of ``(2**32)+offset``. diff --git a/Python/compile.c b/Python/compile.c index a7f62bbcbb33e..1416eb119c9e7 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -128,6 +128,8 @@ typedef struct basicblock_ { unsigned b_nofallthrough : 1; /* Basic block exits scope (it ends with a return or raise) */ unsigned b_exit : 1; + /* Used by compiler passes to mark whether they have visited a basic block. */ + unsigned b_visited : 1; /* depth of stack upon entry of block, computed by stackdepth() */ int b_startdepth; /* instruction offset for block, computed by assemble_jump_offsets() */ @@ -6701,6 +6703,31 @@ assemble_emit(struct assembler *a, struct instr *i) return 1; } +static void +normalize_jumps(struct assembler *a) +{ + for (basicblock *b = a->a_entry; b != NULL; b = b->b_next) { + b->b_visited = 0; + } + for (basicblock *b = a->a_entry; b != NULL; b = b->b_next) { + b->b_visited = 1; + if (b->b_iused == 0) { + continue; + } + struct instr *last = &b->b_instr[b->b_iused-1]; + if (last->i_opcode == JUMP_ABSOLUTE) { + if (last->i_target->b_visited == 0) { + last->i_opcode = JUMP_FORWARD; + } + } + if (last->i_opcode == JUMP_FORWARD) { + if (last->i_target->b_visited == 1) { + last->i_opcode = JUMP_ABSOLUTE; + } + } + } +} + static void assemble_jump_offsets(struct assembler *a, struct compiler *c) { @@ -7137,6 +7164,10 @@ assemble(struct compiler *c, int addNone) } propagate_line_numbers(&a); guarantee_lineno_for_exits(&a, c->u->u_firstlineno); + + /* Order of basic blocks must have been determined by now */ + normalize_jumps(&a); + /* Can't modify the bytecode after computing jump offsets. */ assemble_jump_offsets(&a, c); diff --git a/Python/importlib.h b/Python/importlib.h index ab3e69b22b662..dd1a9f172c04f 100644 --- a/Python/importlib.h +++ b/Python/importlib.h @@ -782,8 +782,8 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 103,0,110,1,100,2,125,5,124,4,124,0,124,1,124,5, 100,4,141,3,83,0,124,3,100,2,117,0,114,67,116,0, 124,1,100,5,131,2,114,65,122,7,124,1,160,4,124,0, - 161,1,125,3,87,0,113,67,4,0,116,5,121,64,1,0, - 1,0,1,0,100,2,125,3,89,0,113,67,119,0,100,6, + 161,1,125,3,87,0,110,13,4,0,116,5,121,64,1,0, + 1,0,1,0,100,2,125,3,89,0,110,3,119,0,100,6, 125,3,116,6,124,0,124,1,124,2,124,3,100,7,141,4, 83,0,41,8,122,53,82,101,116,117,114,110,32,97,32,109, 111,100,117,108,101,32,115,112,101,99,32,98,97,115,101,100, @@ -812,9 +812,9 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 122,5,124,0,106,4,125,5,87,0,110,11,4,0,116,1, 121,59,1,0,1,0,1,0,100,0,125,5,89,0,110,1, 119,0,124,2,100,0,117,0,114,87,124,5,100,0,117,0, - 114,85,122,5,124,1,106,5,125,2,87,0,113,87,4,0, + 114,85,122,5,124,1,106,5,125,2,87,0,110,13,4,0, 116,1,121,84,1,0,1,0,1,0,100,0,125,2,89,0, - 113,87,119,0,124,5,125,2,122,5,124,0,106,6,125,6, + 110,3,119,0,124,5,125,2,122,5,124,0,106,6,125,6, 87,0,110,11,4,0,116,1,121,103,1,0,1,0,1,0, 100,0,125,6,89,0,110,1,119,0,122,7,116,7,124,0, 106,8,131,1,125,7,87,0,110,11,4,0,116,1,121,122, diff --git a/Python/importlib_external.h b/Python/importlib_external.h index 62e47b13804b4..dcf5505fa74fc 100644 --- a/Python/importlib_external.h +++ b/Python/importlib_external.h @@ -977,8 +977,8 @@ const unsigned char _Py_M__importlib_bootstrap_external[] = { 8,0,0,0,67,0,0,0,115,54,1,0,0,124,1,100, 1,117,0,114,29,100,2,125,1,116,0,124,2,100,3,131, 2,114,28,122,7,124,2,160,1,124,0,161,1,125,1,87, - 0,113,57,4,0,116,2,121,27,1,0,1,0,1,0,89, - 0,113,57,119,0,110,28,116,3,160,4,124,1,161,1,125, + 0,110,38,4,0,116,2,121,27,1,0,1,0,1,0,89, + 0,110,30,119,0,110,28,116,3,160,4,124,1,161,1,125, 1,116,5,124,1,131,1,115,57,122,9,116,6,116,3,160, 7,161,0,124,1,131,2,125,1,87,0,110,9,4,0,116, 8,121,56,1,0,1,0,1,0,89,0,110,1,119,0,116, @@ -986,11 +986,11 @@ const unsigned char _Py_M__importlib_bootstrap_external[] = { 5,124,4,95,11,124,2,100,1,117,0,114,99,116,12,131, 0,68,0,93,21,92,2,125,5,125,6,124,1,160,13,116, 14,124,6,131,1,161,1,114,96,124,5,124,0,124,1,131, - 2,125,2,124,2,124,4,95,15,1,0,113,99,113,75,100, + 2,125,2,124,2,124,4,95,15,1,0,110,3,113,75,100, 1,83,0,124,3,116,16,117,0,114,131,116,0,124,2,100, 6,131,2,114,130,122,7,124,2,160,17,124,0,161,1,125, 7,87,0,110,9,4,0,116,2,121,124,1,0,1,0,1, - 0,89,0,113,134,119,0,124,7,114,130,103,0,124,4,95, + 0,89,0,110,10,119,0,124,7,114,130,103,0,124,4,95, 18,110,3,124,3,124,4,95,18,124,4,106,18,103,0,107, 2,114,153,124,1,114,153,116,19,124,1,131,1,100,7,25, 0,125,8,124,4,106,18,160,20,124,8,161,1,1,0,124, diff --git a/Python/importlib_zipimport.h b/Python/importlib_zipimport.h index 07c9dd574040f..3c476841a8c20 100644 --- a/Python/importlib_zipimport.h +++ b/Python/importlib_zipimport.h @@ -130,7 +130,7 @@ const unsigned char _Py_M__zipimport[] = { 3,141,2,130,1,124,5,125,1,124,3,160,13,124,6,161, 1,1,0,89,0,110,15,119,0,124,4,106,14,100,6,64, 0,100,7,107,3,114,89,116,4,100,5,124,1,100,3,141, - 2,130,1,113,91,113,33,122,6,116,15,124,1,25,0,125, + 2,130,1,110,1,113,33,122,6,116,15,124,1,25,0,125, 7,87,0,110,17,4,0,116,16,121,114,1,0,1,0,1, 0,116,17,124,1,131,1,125,7,124,7,116,15,124,1,60, 0,89,0,110,1,119,0,124,7,124,0,95,18,124,1,124, @@ -653,7 +653,7 @@ const unsigned char _Py_M__zipimport[] = { 100,2,141,2,130,1,119,0,9,0,124,1,160,7,100,16, 161,1,125,3,116,8,124,3,131,1,100,5,107,0,144,1, 114,55,116,14,100,17,131,1,130,1,124,3,100,0,100,5, - 133,2,25,0,100,18,107,3,144,1,114,66,144,2,113,85, + 133,2,25,0,100,18,107,3,144,1,114,66,144,1,110,19, 116,8,124,3,131,1,100,16,107,3,144,1,114,77,116,14, 100,17,131,1,130,1,116,15,124,3,100,19,100,20,133,2, 25,0,131,1,125,13,116,15,124,3,100,20,100,9,133,2, @@ -1004,9 +1004,9 @@ const unsigned char _Py_M__zipimport[] = { 1,0,1,0,89,0,113,9,119,0,124,8,100,4,25,0, 125,9,116,8,124,0,106,4,124,8,131,2,125,10,100,0, 125,11,124,5,114,91,122,10,116,9,124,0,124,9,124,7, - 124,1,124,10,131,5,125,11,87,0,113,96,4,0,116,10, + 124,1,124,10,131,5,125,11,87,0,110,25,4,0,116,10, 121,90,1,0,125,12,1,0,122,8,124,12,125,3,87,0, - 89,0,100,0,125,12,126,12,113,96,100,0,125,12,126,12, + 89,0,100,0,125,12,126,12,110,10,100,0,125,12,126,12, 119,1,119,0,116,11,124,9,124,10,131,2,125,11,124,11, 100,0,117,0,114,101,113,9,124,8,100,4,25,0,125,9, 124,11,124,6,124,9,102,3,2,0,1,0,83,0,124,3, From webhook-mailer at python.org Wed Feb 16 07:19:23 2022 From: webhook-mailer at python.org (markshannon) Date: Wed, 16 Feb 2022 12:19:23 -0000 Subject: [Python-checkins] Remove offsets from expected output in test.test_dis (GH-31369) Message-ID: https://github.com/python/cpython/commit/3954fcb83fe471911ff01c0410a71d184d9984e7 commit: 3954fcb83fe471911ff01c0410a71d184d9984e7 branch: main author: Mark Shannon committer: markshannon date: 2022-02-16T12:19:09Z summary: Remove offsets from expected output in test.test_dis (GH-31369) files: A Misc/NEWS.d/next/Tests/2022-02-16-10-38-18.bpo-46760.O3ovJo.rst M Lib/test/test_dis.py diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index 29da5701cdaa3..4feeb8c5be8a2 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -38,49 +38,49 @@ def cm(cls, x): cls.x = x == 1 dis_c_instance_method = """\ -%3d 0 RESUME 0 - -%3d 2 LOAD_FAST 1 (x) - 4 LOAD_CONST 1 (1) - 6 COMPARE_OP 2 (==) - 8 LOAD_FAST 0 (self) - 10 STORE_ATTR 0 (x) - 12 LOAD_CONST 0 (None) - 14 RETURN_VALUE +%3d RESUME 0 + +%3d LOAD_FAST 1 (x) + LOAD_CONST 1 (1) + COMPARE_OP 2 (==) + LOAD_FAST 0 (self) + STORE_ATTR 0 (x) + LOAD_CONST 0 (None) + RETURN_VALUE """ % (_C.__init__.__code__.co_firstlineno, _C.__init__.__code__.co_firstlineno + 1,) dis_c_instance_method_bytes = """\ - 0 RESUME 0 - 2 LOAD_FAST 1 - 4 LOAD_CONST 1 - 6 COMPARE_OP 2 (==) - 8 LOAD_FAST 0 - 10 STORE_ATTR 0 - 12 LOAD_CONST 0 - 14 RETURN_VALUE + RESUME 0 + LOAD_FAST 1 + LOAD_CONST 1 + COMPARE_OP 2 (==) + LOAD_FAST 0 + STORE_ATTR 0 + LOAD_CONST 0 + RETURN_VALUE """ dis_c_class_method = """\ -%3d 0 RESUME 0 - -%3d 2 LOAD_FAST 1 (x) - 4 LOAD_CONST 1 (1) - 6 COMPARE_OP 2 (==) - 8 LOAD_FAST 0 (cls) - 10 STORE_ATTR 0 (x) - 12 LOAD_CONST 0 (None) - 14 RETURN_VALUE +%3d RESUME 0 + +%3d LOAD_FAST 1 (x) + LOAD_CONST 1 (1) + COMPARE_OP 2 (==) + LOAD_FAST 0 (cls) + STORE_ATTR 0 (x) + LOAD_CONST 0 (None) + RETURN_VALUE """ % (_C.cm.__code__.co_firstlineno, _C.cm.__code__.co_firstlineno + 2,) dis_c_static_method = """\ -%3d 0 RESUME 0 - -%3d 2 LOAD_FAST 0 (x) - 4 LOAD_CONST 1 (1) - 6 COMPARE_OP 2 (==) - 8 STORE_FAST 0 (x) - 10 LOAD_CONST 0 (None) - 12 RETURN_VALUE +%3d RESUME 0 + +%3d LOAD_FAST 0 (x) + LOAD_CONST 1 (1) + COMPARE_OP 2 (==) + STORE_FAST 0 (x) + LOAD_CONST 0 (None) + RETURN_VALUE """ % (_C.sm.__code__.co_firstlineno, _C.sm.__code__.co_firstlineno + 2,) # Class disassembling info has an extra newline at end. @@ -100,30 +100,30 @@ def _f(a): return 1 dis_f = """\ -%3d 0 RESUME 0 +%3d RESUME 0 -%3d 2 LOAD_GLOBAL 0 (print) - 4 LOAD_FAST 0 (a) - 6 PRECALL_FUNCTION 1 - 8 CALL 0 - 10 POP_TOP +%3d LOAD_GLOBAL 0 (print) + LOAD_FAST 0 (a) + PRECALL_FUNCTION 1 + CALL 0 + POP_TOP -%3d 12 LOAD_CONST 1 (1) - 14 RETURN_VALUE +%3d LOAD_CONST 1 (1) + RETURN_VALUE """ % (_f.__code__.co_firstlineno, _f.__code__.co_firstlineno + 1, _f.__code__.co_firstlineno + 2) dis_f_co_code = """\ - 0 RESUME 0 - 2 LOAD_GLOBAL 0 - 4 LOAD_FAST 0 - 6 PRECALL_FUNCTION 1 - 8 CALL 0 - 10 POP_TOP - 12 LOAD_CONST 1 - 14 RETURN_VALUE + RESUME 0 + LOAD_GLOBAL 0 + LOAD_FAST 0 + PRECALL_FUNCTION 1 + CALL 0 + POP_TOP + LOAD_CONST 1 + RETURN_VALUE """ @@ -133,23 +133,23 @@ def bug708901(): pass dis_bug708901 = """\ -%3d 0 RESUME 0 +%3d RESUME 0 -%3d 2 LOAD_GLOBAL 0 (range) - 4 LOAD_CONST 1 (1) +%3d LOAD_GLOBAL 0 (range) + LOAD_CONST 1 (1) -%3d 6 LOAD_CONST 2 (10) +%3d LOAD_CONST 2 (10) -%3d 8 PRECALL_FUNCTION 2 - 10 CALL 0 - 12 GET_ITER - >> 14 FOR_ITER 2 (to 20) - 16 STORE_FAST 0 (res) +%3d PRECALL_FUNCTION 2 + CALL 0 + GET_ITER + >> FOR_ITER 2 (to 20) + STORE_FAST 0 (res) -%3d 18 JUMP_ABSOLUTE 7 (to 14) +%3d JUMP_ABSOLUTE 7 (to 14) -%3d >> 20 LOAD_CONST 0 (None) - 22 RETURN_VALUE +%3d >> LOAD_CONST 0 (None) + RETURN_VALUE """ % (bug708901.__code__.co_firstlineno, bug708901.__code__.co_firstlineno + 1, bug708901.__code__.co_firstlineno + 2, @@ -164,22 +164,22 @@ def bug1333982(x=[]): pass dis_bug1333982 = """\ -%3d 0 RESUME 0 - -%3d 2 LOAD_ASSERTION_ERROR - 4 LOAD_CONST 2 ( at 0x..., file "%s", line %d>) - 6 MAKE_FUNCTION 0 - 8 LOAD_FAST 0 (x) - 10 GET_ITER - 12 PRECALL_FUNCTION 1 - 14 CALL 0 - -%3d 16 LOAD_CONST 3 (1) - -%3d 18 BINARY_OP 0 (+) - 20 PRECALL_FUNCTION 1 - 22 CALL 0 - 24 RAISE_VARARGS 1 +%3d RESUME 0 + +%3d LOAD_ASSERTION_ERROR + LOAD_CONST 2 ( at 0x..., file "%s", line %d>) + MAKE_FUNCTION 0 + LOAD_FAST 0 (x) + GET_ITER + PRECALL_FUNCTION 1 + CALL 0 + +%3d LOAD_CONST 3 (1) + +%3d BINARY_OP 0 (+) + PRECALL_FUNCTION 1 + CALL 0 + RAISE_VARARGS 1 """ % (bug1333982.__code__.co_firstlineno, bug1333982.__code__.co_firstlineno + 1, __file__, @@ -197,9 +197,9 @@ def bug42562(): dis_bug42562 = """\ - 0 RESUME 0 - 2 LOAD_CONST 0 (None) - 4 RETURN_VALUE + RESUME 0 + LOAD_CONST 0 (None) + RETURN_VALUE """ # Extended arg followed by NOP @@ -212,66 +212,66 @@ def bug42562(): ]) dis_bug_45757 = """\ - 0 EXTENDED_ARG 1 - 2 NOP - 4 EXTENDED_ARG 1 - 6 LOAD_CONST 297 - 8 RETURN_VALUE + EXTENDED_ARG 1 + NOP + EXTENDED_ARG 1 + LOAD_CONST 297 + RETURN_VALUE """ _BIG_LINENO_FORMAT = """\ - 1 0 RESUME 0 + 1 RESUME 0 -%3d 2 LOAD_GLOBAL 0 (spam) - 4 POP_TOP - 6 LOAD_CONST 0 (None) - 8 RETURN_VALUE +%3d LOAD_GLOBAL 0 (spam) + POP_TOP + LOAD_CONST 0 (None) + RETURN_VALUE """ _BIG_LINENO_FORMAT2 = """\ - 1 0 RESUME 0 + 1 RESUME 0 -%4d 2 LOAD_GLOBAL 0 (spam) - 4 POP_TOP - 6 LOAD_CONST 0 (None) - 8 RETURN_VALUE +%4d LOAD_GLOBAL 0 (spam) + POP_TOP + LOAD_CONST 0 (None) + RETURN_VALUE """ dis_module_expected_results = """\ Disassembly of f: - 4 0 RESUME 0 - 2 LOAD_CONST 0 (None) - 4 RETURN_VALUE + 4 RESUME 0 + LOAD_CONST 0 (None) + RETURN_VALUE Disassembly of g: - 5 0 RESUME 0 - 2 LOAD_CONST 0 (None) - 4 RETURN_VALUE + 5 RESUME 0 + LOAD_CONST 0 (None) + RETURN_VALUE """ expr_str = "x + 1" dis_expr_str = """\ - 0 RESUME 0 + RESUME 0 - 1 2 LOAD_NAME 0 (x) - 4 LOAD_CONST 0 (1) - 6 BINARY_OP 0 (+) - 8 RETURN_VALUE + 1 LOAD_NAME 0 (x) + LOAD_CONST 0 (1) + BINARY_OP 0 (+) + RETURN_VALUE """ simple_stmt_str = "x = x + 1" dis_simple_stmt_str = """\ - 0 RESUME 0 - - 1 2 LOAD_NAME 0 (x) - 4 LOAD_CONST 0 (1) - 6 BINARY_OP 0 (+) - 8 STORE_NAME 0 (x) - 10 LOAD_CONST 1 (None) - 12 RETURN_VALUE + RESUME 0 + + 1 LOAD_NAME 0 (x) + LOAD_CONST 0 (1) + BINARY_OP 0 (+) + STORE_NAME 0 (x) + LOAD_CONST 1 (None) + RETURN_VALUE """ annot_stmt_str = """\ @@ -283,35 +283,35 @@ def bug42562(): # leading newline is for a reason (tests lineno) dis_annot_stmt_str = """\ - 0 RESUME 0 - - 2 2 SETUP_ANNOTATIONS - 4 LOAD_CONST 0 (1) - 6 STORE_NAME 0 (x) - 8 LOAD_NAME 1 (int) - 10 LOAD_NAME 2 (__annotations__) - 12 LOAD_CONST 1 ('x') - 14 STORE_SUBSCR - - 3 16 LOAD_NAME 3 (fun) - 18 LOAD_CONST 0 (1) - 20 PRECALL_FUNCTION 1 - 22 CALL 0 - 24 LOAD_NAME 2 (__annotations__) - 26 LOAD_CONST 2 ('y') - 28 STORE_SUBSCR - - 4 30 LOAD_CONST 0 (1) - 32 LOAD_NAME 4 (lst) - 34 LOAD_NAME 3 (fun) - 36 LOAD_CONST 3 (0) - 38 PRECALL_FUNCTION 1 - 40 CALL 0 - 42 STORE_SUBSCR - 44 LOAD_NAME 1 (int) - 46 POP_TOP - 48 LOAD_CONST 4 (None) - 50 RETURN_VALUE + RESUME 0 + + 2 SETUP_ANNOTATIONS + LOAD_CONST 0 (1) + STORE_NAME 0 (x) + LOAD_NAME 1 (int) + LOAD_NAME 2 (__annotations__) + LOAD_CONST 1 ('x') + STORE_SUBSCR + + 3 LOAD_NAME 3 (fun) + LOAD_CONST 0 (1) + PRECALL_FUNCTION 1 + CALL 0 + LOAD_NAME 2 (__annotations__) + LOAD_CONST 2 ('y') + STORE_SUBSCR + + 4 LOAD_CONST 0 (1) + LOAD_NAME 4 (lst) + LOAD_NAME 3 (fun) + LOAD_CONST 3 (0) + PRECALL_FUNCTION 1 + CALL 0 + STORE_SUBSCR + LOAD_NAME 1 (int) + POP_TOP + LOAD_CONST 4 (None) + RETURN_VALUE """ compound_stmt_str = """\ @@ -321,63 +321,59 @@ def bug42562(): # Trailing newline has been deliberately omitted dis_compound_stmt_str = """\ - 0 RESUME 0 + RESUME 0 - 1 2 LOAD_CONST 0 (0) - 4 STORE_NAME 0 (x) + 1 LOAD_CONST 0 (0) + STORE_NAME 0 (x) - 2 6 NOP + 2 NOP - 3 >> 8 LOAD_NAME 0 (x) - 10 LOAD_CONST 1 (1) - 12 BINARY_OP 13 (+=) - 14 STORE_NAME 0 (x) + 3 >> LOAD_NAME 0 (x) + LOAD_CONST 1 (1) + BINARY_OP 13 (+=) + STORE_NAME 0 (x) - 2 16 JUMP_ABSOLUTE 4 (to 8) + 2 JUMP_ABSOLUTE 4 (to 8) """ dis_traceback = """\ -%3d 0 RESUME 0 - -%3d 2 NOP - -%3d 4 LOAD_CONST 1 (1) - 6 LOAD_CONST 2 (0) - --> 8 BINARY_OP 11 (/) - 10 POP_TOP - -%3d 12 LOAD_FAST 1 (tb) - 14 RETURN_VALUE - >> 16 PUSH_EXC_INFO - -%3d 18 LOAD_GLOBAL 0 (Exception) - 20 JUMP_IF_NOT_EXC_MATCH 25 (to 50) - 22 STORE_FAST 0 (e) - -%3d 24 LOAD_FAST 0 (e) - 26 LOAD_ATTR 1 (__traceback__) - 28 STORE_FAST 1 (tb) - 30 POP_EXCEPT - 32 LOAD_CONST 0 (None) - 34 STORE_FAST 0 (e) - 36 DELETE_FAST 0 (e) - -%3d 38 LOAD_FAST 1 (tb) - 40 RETURN_VALUE - >> 42 LOAD_CONST 0 (None) - 44 STORE_FAST 0 (e) - 46 DELETE_FAST 0 (e) - 48 RERAISE 1 - -%3d >> 50 RERAISE 0 - >> 52 COPY 3 - 54 POP_EXCEPT - 56 RERAISE 1 +%3d RESUME 0 + +%3d NOP + +%3d LOAD_CONST 1 (1) + LOAD_CONST 2 (0) + --> BINARY_OP 11 (/) + POP_TOP + +%3d LOAD_FAST 1 (tb) + RETURN_VALUE + >> PUSH_EXC_INFO + +%3d LOAD_GLOBAL 0 (Exception) + JUMP_IF_NOT_EXC_MATCH 25 (to 50) + STORE_FAST 0 (e) + +%3d LOAD_FAST 0 (e) + LOAD_ATTR 1 (__traceback__) + STORE_FAST 1 (tb) + POP_EXCEPT + LOAD_CONST 0 (None) + STORE_FAST 0 (e) + DELETE_FAST 0 (e) + +%3d LOAD_FAST 1 (tb) + RETURN_VALUE + >> LOAD_CONST 0 (None) + STORE_FAST 0 (e) + DELETE_FAST 0 (e) + RERAISE 1 + +%3d >> RERAISE 0 + >> COPY 3 + POP_EXCEPT + RERAISE 1 ExceptionTable: - 4 to 10 -> 16 [0] - 16 to 22 -> 52 [1] lasti - 24 to 28 -> 42 [1] lasti - 42 to 50 -> 52 [1] lasti """ % (TRACEBACK_CODE.co_firstlineno, TRACEBACK_CODE.co_firstlineno + 1, TRACEBACK_CODE.co_firstlineno + 2, @@ -391,23 +387,23 @@ def _fstring(a, b, c, d): return f'{a} {b:4} {c!r} {d!r:4}' dis_fstring = """\ -%3d 0 RESUME 0 - -%3d 2 LOAD_FAST 0 (a) - 4 FORMAT_VALUE 0 - 6 LOAD_CONST 1 (' ') - 8 LOAD_FAST 1 (b) - 10 LOAD_CONST 2 ('4') - 12 FORMAT_VALUE 4 (with format) - 14 LOAD_CONST 1 (' ') - 16 LOAD_FAST 2 (c) - 18 FORMAT_VALUE 2 (repr) - 20 LOAD_CONST 1 (' ') - 22 LOAD_FAST 3 (d) - 24 LOAD_CONST 2 ('4') - 26 FORMAT_VALUE 6 (repr, with format) - 28 BUILD_STRING 7 - 30 RETURN_VALUE +%3d RESUME 0 + +%3d LOAD_FAST 0 (a) + FORMAT_VALUE 0 + LOAD_CONST 1 (' ') + LOAD_FAST 1 (b) + LOAD_CONST 2 ('4') + FORMAT_VALUE 4 (with format) + LOAD_CONST 1 (' ') + LOAD_FAST 2 (c) + FORMAT_VALUE 2 (repr) + LOAD_CONST 1 (' ') + LOAD_FAST 3 (d) + LOAD_CONST 2 ('4') + FORMAT_VALUE 6 (repr, with format) + BUILD_STRING 7 + RETURN_VALUE """ % (_fstring.__code__.co_firstlineno, _fstring.__code__.co_firstlineno + 1) def _tryfinally(a, b): @@ -423,29 +419,27 @@ def _tryfinallyconst(b): b() dis_tryfinally = """\ -%3d 0 RESUME 0 - -%3d 2 NOP - -%3d 4 LOAD_FAST 0 (a) - -%3d 6 LOAD_FAST 1 (b) - 8 PRECALL_FUNCTION 0 - 10 CALL 0 - 12 POP_TOP - 14 RETURN_VALUE - >> 16 PUSH_EXC_INFO - 18 LOAD_FAST 1 (b) - 20 PRECALL_FUNCTION 0 - 22 CALL 0 - 24 POP_TOP - 26 RERAISE 0 - >> 28 COPY 3 - 30 POP_EXCEPT - 32 RERAISE 1 +%3d RESUME 0 + +%3d NOP + +%3d LOAD_FAST 0 (a) + +%3d LOAD_FAST 1 (b) + PRECALL_FUNCTION 0 + CALL 0 + POP_TOP + RETURN_VALUE + >> PUSH_EXC_INFO + LOAD_FAST 1 (b) + PRECALL_FUNCTION 0 + CALL 0 + POP_TOP + RERAISE 0 + >> COPY 3 + POP_EXCEPT + RERAISE 1 ExceptionTable: - 4 to 4 -> 16 [0] - 16 to 26 -> 28 [1] lasti """ % (_tryfinally.__code__.co_firstlineno, _tryfinally.__code__.co_firstlineno + 1, _tryfinally.__code__.co_firstlineno + 2, @@ -453,29 +447,28 @@ def _tryfinallyconst(b): ) dis_tryfinallyconst = """\ -%3d 0 RESUME 0 - -%3d 2 NOP - -%3d 4 NOP - -%3d 6 LOAD_FAST 0 (b) - 8 PRECALL_FUNCTION 0 - 10 CALL 0 - 12 POP_TOP - 14 LOAD_CONST 1 (1) - 16 RETURN_VALUE - 18 PUSH_EXC_INFO - 20 LOAD_FAST 0 (b) - 22 PRECALL_FUNCTION 0 - 24 CALL 0 - 26 POP_TOP - 28 RERAISE 0 - >> 30 COPY 3 - 32 POP_EXCEPT - 34 RERAISE 1 +%3d RESUME 0 + +%3d NOP + +%3d NOP + +%3d LOAD_FAST 0 (b) + PRECALL_FUNCTION 0 + CALL 0 + POP_TOP + LOAD_CONST 1 (1) + RETURN_VALUE + PUSH_EXC_INFO + LOAD_FAST 0 (b) + PRECALL_FUNCTION 0 + CALL 0 + POP_TOP + RERAISE 0 + >> COPY 3 + POP_EXCEPT + RERAISE 1 ExceptionTable: - 18 to 28 -> 30 [1] lasti """ % (_tryfinallyconst.__code__.co_firstlineno, _tryfinallyconst.__code__.co_firstlineno + 1, _tryfinallyconst.__code__.co_firstlineno + 2, @@ -499,18 +492,18 @@ def foo(x): return foo dis_nested_0 = """\ - 0 MAKE_CELL 0 (y) + MAKE_CELL 0 (y) -%3d 2 RESUME 0 +%3d RESUME 0 -%3d 4 LOAD_CLOSURE 0 (y) - 6 BUILD_TUPLE 1 - 8 LOAD_CONST 1 () - 10 MAKE_FUNCTION 8 (closure) - 12 STORE_FAST 1 (foo) +%3d LOAD_CLOSURE 0 (y) + BUILD_TUPLE 1 + LOAD_CONST 1 () + MAKE_FUNCTION 8 (closure) + STORE_FAST 1 (foo) -%3d 14 LOAD_FAST 1 (foo) - 16 RETURN_VALUE +%3d LOAD_FAST 1 (foo) + RETURN_VALUE """ % (_h.__code__.co_firstlineno, _h.__code__.co_firstlineno + 1, __file__, @@ -520,20 +513,20 @@ def foo(x): dis_nested_1 = """%s Disassembly of : - 0 COPY_FREE_VARS 1 - 2 MAKE_CELL 0 (x) - -%3d 4 RESUME 0 - -%3d 6 LOAD_CLOSURE 0 (x) - 8 BUILD_TUPLE 1 - 10 LOAD_CONST 1 ( at 0x..., file "%s", line %d>) - 12 MAKE_FUNCTION 8 (closure) - 14 LOAD_DEREF 1 (y) - 16 GET_ITER - 18 PRECALL_FUNCTION 1 - 20 CALL 0 - 22 RETURN_VALUE + COPY_FREE_VARS 1 + MAKE_CELL 0 (x) + +%3d RESUME 0 + +%3d LOAD_CLOSURE 0 (x) + BUILD_TUPLE 1 + LOAD_CONST 1 ( at 0x..., file "%s", line %d>) + MAKE_FUNCTION 8 (closure) + LOAD_DEREF 1 (y) + GET_ITER + PRECALL_FUNCTION 1 + CALL 0 + RETURN_VALUE """ % (dis_nested_0, __file__, _h.__code__.co_firstlineno + 1, @@ -545,27 +538,90 @@ def foo(x): dis_nested_2 = """%s Disassembly of at 0x..., file "%s", line %d>: - 0 COPY_FREE_VARS 1 - -%3d 2 RESUME 0 - 4 BUILD_LIST 0 - 6 LOAD_FAST 0 (.0) - >> 8 FOR_ITER 6 (to 22) - 10 STORE_FAST 1 (z) - 12 LOAD_DEREF 2 (x) - 14 LOAD_FAST 1 (z) - 16 BINARY_OP 0 (+) - 18 LIST_APPEND 2 - 20 JUMP_ABSOLUTE 4 (to 8) - >> 22 RETURN_VALUE + COPY_FREE_VARS 1 + +%3d RESUME 0 + BUILD_LIST 0 + LOAD_FAST 0 (.0) + >> FOR_ITER 6 (to 22) + STORE_FAST 1 (z) + LOAD_DEREF 2 (x) + LOAD_FAST 1 (z) + BINARY_OP 0 (+) + LIST_APPEND 2 + JUMP_ABSOLUTE 4 (to 8) + >> RETURN_VALUE """ % (dis_nested_1, __file__, _h.__code__.co_firstlineno + 3, _h.__code__.co_firstlineno + 3, ) +class DisTestBase(unittest.TestCase): + "Common utilities for DisTests and TestDisTraceback" -class DisTests(unittest.TestCase): + def strip_addresses(self, text): + return re.sub(r'\b0x[0-9A-Fa-f]+\b', '0x...', text) + + def find_offset_column(self, lines): + for line in lines: + if line and not line.startswith("Disassembly"): + break + else: + return 0, 0 + offset = 5 + while (line[offset] == " "): + offset += 1 + if (line[offset] == ">"): + offset += 2 + while (line[offset] == " "): + offset += 1 + end = offset + while line[end] in "0123456789": + end += 1 + return end-5, end + + def assert_offsets_increasing(self, text, delta): + expected_offset = 0 + lines = text.splitlines() + start, end = self.find_offset_column(lines) + for line in lines: + if not line: + continue + if line.startswith("Disassembly"): + expected_offset = 0 + continue + if line.startswith("Exception"): + break + offset = int(line[start:end]) + self.assertEqual(offset, expected_offset, line) + expected_offset += delta + + def strip_offsets(self, text): + lines = text.splitlines(True) + start, end = self.find_offset_column(lines) + res = [] + lines = iter(lines) + for line in lines: + if line.startswith("Exception"): + res.append(line) + break + if not line or line.startswith("Disassembly"): + res.append(line) + else: + res.append(line[:start] + line[end:]) + return "".join(res) + + def do_disassembly_compare(self, got, expected, with_offsets=False): + if not with_offsets: + self.assert_offsets_increasing(got, 2) + got = self.strip_offsets(got) + if got != expected: + got = self.strip_addresses(got) + self.assertEqual(got, expected) + + +class DisTests(DisTestBase): maxDiff = None @@ -582,15 +638,10 @@ def get_disassembly(self, func, lasti=-1, wrapper=True, **kwargs): def get_disassemble_as_string(self, func, lasti=-1): return self.get_disassembly(func, lasti, False) - def strip_addresses(self, text): - return re.sub(r'\b0x[0-9A-Fa-f]+\b', '0x...', text) - - def do_disassembly_test(self, func, expected): + def do_disassembly_test(self, func, expected, with_offsets=False): self.maxDiff = None got = self.get_disassembly(func, depth=0) - if got != expected: - got = self.strip_addresses(got) - self.assertEqual(got, expected) + self.do_disassembly_compare(got, expected, with_offsets) def test_opmap(self): self.assertEqual(dis.opmap["NOP"], 9) @@ -690,9 +741,9 @@ def expected(count, w): return ''.join(s) for i in range(1, 5): - self.do_disassembly_test(func(i), expected(i, 4)) - self.do_disassembly_test(func(1248), expected(1248, 4)) - self.do_disassembly_test(func(1250), expected(1250, 5)) + self.do_disassembly_test(func(i), expected(i, 4), True) + self.do_disassembly_test(func(1248), expected(1248, 4), True) + self.do_disassembly_test(func(1250), expected(1250, 5), True) def test_disassemble_str(self): self.do_disassembly_test(expr_str, dis_expr_str) @@ -764,7 +815,7 @@ def test_dis_traceback(self): sys.last_traceback = tb tb_dis = self.get_disassemble_as_string(tb.tb_frame.f_code, tb.tb_lasti) - self.do_disassembly_test(None, tb_dis) + self.do_disassembly_test(None, tb_dis, True) def test_dis_object(self): self.assertRaises(TypeError, dis.dis, object()) @@ -773,6 +824,7 @@ def test_disassemble_recursive(self): def check(expected, **kwargs): dis = self.get_disassembly(_h, **kwargs) dis = self.strip_addresses(dis) + dis = self.strip_offsets(dis) self.assertEqual(dis, expected) check(dis_nested_0, depth=0) @@ -1369,7 +1421,7 @@ def test_co_positions_missing_info(self): # get_instructions has its own tests above, so can rely on it to validate # the object oriented API -class BytecodeTests(InstructionTestCase): +class BytecodeTests(InstructionTestCase, DisTestBase): def test_instantiation(self): # Test with function, method, code string and code object @@ -1410,7 +1462,7 @@ def test_info(self): def test_disassembled(self): actual = dis.Bytecode(_f).dis() - self.assertEqual(actual, dis_f) + self.do_disassembly_compare(actual, dis_f) def test_from_traceback(self): tb = get_tb() @@ -1423,7 +1475,7 @@ def test_from_traceback_dis(self): self.maxDiff = None tb = get_tb() b = dis.Bytecode.from_traceback(tb) - self.assertEqual(b.dis(), dis_traceback) + self.assertEqual(self.strip_offsets(b.dis()), dis_traceback) @requires_debug_ranges() def test_bytecode_co_positions(self): @@ -1492,7 +1544,7 @@ def test_findlabels(self): self.assertEqual(sorted(labels), sorted(jumps)) -class TestDisTraceback(unittest.TestCase): +class TestDisTraceback(DisTestBase): def setUp(self) -> None: try: # We need to clean up existing tracebacks del sys.last_traceback @@ -1516,13 +1568,13 @@ def test_distb_last_traceback(self): tb = get_tb() sys.last_traceback = tb - self.assertEqual(self.get_disassembly(None), dis_traceback) + self.do_disassembly_compare(self.get_disassembly(None), dis_traceback) def test_distb_explicit_arg(self): self.maxDiff = None tb = get_tb() - self.assertEqual(self.get_disassembly(tb), dis_traceback) + self.do_disassembly_compare(self.get_disassembly(tb), dis_traceback) class TestDisTracebackWithFile(TestDisTraceback): diff --git a/Misc/NEWS.d/next/Tests/2022-02-16-10-38-18.bpo-46760.O3ovJo.rst b/Misc/NEWS.d/next/Tests/2022-02-16-10-38-18.bpo-46760.O3ovJo.rst new file mode 100644 index 0000000000000..c7dfc7c559939 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-16-10-38-18.bpo-46760.O3ovJo.rst @@ -0,0 +1,2 @@ +Remove bytecode offsets from expected values in test.test_dis module. +Reduces the obstacles to modifying the VM or compiler. From webhook-mailer at python.org Wed Feb 16 07:46:38 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 16 Feb 2022 12:46:38 -0000 Subject: [Python-checkins] bpo-29418: Implement inspect.ismethodwrapper and fix inspect.isroutine for cases where methodwrapper is given (GH-19261) Message-ID: https://github.com/python/cpython/commit/562c13f5734d406b2283cfca673611f4b496fdc7 commit: 562c13f5734d406b2283cfca673611f4b496fdc7 branch: main author: Hakan ?elik committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-16T04:46:20-08:00 summary: bpo-29418: Implement inspect.ismethodwrapper and fix inspect.isroutine for cases where methodwrapper is given (GH-19261) Automerge-Triggered-By: GH:isidentical files: A Misc/NEWS.d/next/Library/2020-03-31-20-53-11.bpo-29418.8Qa9cQ.rst M Doc/library/inspect.rst M Lib/inspect.py M Lib/test/test_inspect.py diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index 711f510d7dcb0..7a6dc9c0337d8 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -429,6 +429,14 @@ attributes: Return ``True`` if the object is a built-in function or a bound built-in method. +.. function:: ismethodwrapper(object) + + Return ``True`` if the type of object is a :class:`~types.MethodWrapperType`. + + These are instances of :class:`~types.MethodWrapperType`, such as :meth:`~object().__str__`, + :meth:`~object().__eq__` and :meth:`~object().__repr__` + + .. function:: isroutine(object) Return ``True`` if the object is a user-defined or built-in function or method. diff --git a/Lib/inspect.py b/Lib/inspect.py index eb45f81aa2d95..9c1283ab3734b 100644 --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -121,6 +121,7 @@ "ismemberdescriptor", "ismethod", "ismethoddescriptor", + "ismethodwrapper", "ismodule", "isroutine", "istraceback", @@ -509,12 +510,17 @@ def isbuiltin(object): __self__ instance to which a method is bound, or None""" return isinstance(object, types.BuiltinFunctionType) +def ismethodwrapper(object): + """Return true if the object is a method wrapper.""" + return isinstance(object, types.MethodWrapperType) + def isroutine(object): """Return true if the object is any kind of function or method.""" return (isbuiltin(object) or isfunction(object) or ismethod(object) - or ismethoddescriptor(object)) + or ismethoddescriptor(object) + or ismethodwrapper(object)) def isabstract(object): """Return true if the object is an abstract base class (ABC).""" @@ -1887,13 +1893,9 @@ def getcoroutinelocals(coroutine): ############################################################################### -_WrapperDescriptor = type(type.__call__) -_MethodWrapper = type(all.__call__) -_ClassMethodWrapper = type(int.__dict__['from_bytes']) - -_NonUserDefinedCallables = (_WrapperDescriptor, - _MethodWrapper, - _ClassMethodWrapper, +_NonUserDefinedCallables = (types.WrapperDescriptorType, + types.MethodWrapperType, + types.ClassMethodDescriptorType, types.BuiltinFunctionType) @@ -2533,7 +2535,7 @@ def _signature_from_callable(obj, *, elif not isinstance(obj, _NonUserDefinedCallables): # An object with __call__ # We also check that the 'obj' is not an instance of - # _WrapperDescriptor or _MethodWrapper to avoid + # types.WrapperDescriptorType or types.MethodWrapperType to avoid # infinite recursion (and even potential segfault) call = _signature_get_user_defined_method(type(obj), '__call__') if call is not None: diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py index 29589a726768f..9e3c77056d70a 100644 --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -44,7 +44,8 @@ # isbuiltin, isroutine, isgenerator, isgeneratorfunction, getmembers, # getdoc, getfile, getmodule, getsourcefile, getcomments, getsource, # getclasstree, getargvalues, formatargvalues, -# currentframe, stack, trace, isdatadescriptor +# currentframe, stack, trace, isdatadescriptor, +# ismethodwrapper # NOTE: There are some additional tests relating to interaction with # zipimport in the test_zipimport_support test module. @@ -93,7 +94,8 @@ class IsTestBase(unittest.TestCase): inspect.ismodule, inspect.istraceback, inspect.isgenerator, inspect.isgeneratorfunction, inspect.iscoroutine, inspect.iscoroutinefunction, - inspect.isasyncgen, inspect.isasyncgenfunction]) + inspect.isasyncgen, inspect.isasyncgenfunction, + inspect.ismethodwrapper]) def istest(self, predicate, exp): obj = eval(exp) @@ -169,6 +171,14 @@ def test_excluding_predicates(self): self.istest(inspect.ismemberdescriptor, 'datetime.timedelta.days') else: self.assertFalse(inspect.ismemberdescriptor(datetime.timedelta.days)) + self.istest(inspect.ismethodwrapper, "object().__str__") + self.istest(inspect.ismethodwrapper, "object().__eq__") + self.istest(inspect.ismethodwrapper, "object().__repr__") + self.assertFalse(inspect.ismethodwrapper(type)) + self.assertFalse(inspect.ismethodwrapper(int)) + self.assertFalse(inspect.ismethodwrapper(type("AnyClass", (), {}))) + + def test_iscoroutine(self): async_gen_coro = async_generator_function_example(1) @@ -241,8 +251,38 @@ class NotFuture: pass coro.close(); gen_coro.close() # silence warnings def test_isroutine(self): - self.assertTrue(inspect.isroutine(mod.spam)) + # method + self.assertTrue(inspect.isroutine(git.argue)) + self.assertTrue(inspect.isroutine(mod.custom_method)) self.assertTrue(inspect.isroutine([].count)) + # function + self.assertTrue(inspect.isroutine(mod.spam)) + self.assertTrue(inspect.isroutine(mod.StupidGit.abuse)) + # slot-wrapper + self.assertTrue(inspect.isroutine(object.__init__)) + self.assertTrue(inspect.isroutine(object.__str__)) + self.assertTrue(inspect.isroutine(object.__lt__)) + self.assertTrue(inspect.isroutine(int.__lt__)) + # method-wrapper + self.assertTrue(inspect.isroutine(object().__init__)) + self.assertTrue(inspect.isroutine(object().__str__)) + self.assertTrue(inspect.isroutine(object().__lt__)) + self.assertTrue(inspect.isroutine((42).__lt__)) + # method-descriptor + self.assertTrue(inspect.isroutine(str.join)) + self.assertTrue(inspect.isroutine(list.append)) + self.assertTrue(inspect.isroutine(''.join)) + self.assertTrue(inspect.isroutine([].append)) + # object + self.assertFalse(inspect.isroutine(object)) + self.assertFalse(inspect.isroutine(object())) + self.assertFalse(inspect.isroutine(str())) + # module + self.assertFalse(inspect.isroutine(mod)) + # type + self.assertFalse(inspect.isroutine(type)) + self.assertFalse(inspect.isroutine(int)) + self.assertFalse(inspect.isroutine(type('some_class', (), {}))) def test_isclass(self): self.istest(inspect.isclass, 'mod.StupidGit') diff --git a/Misc/NEWS.d/next/Library/2020-03-31-20-53-11.bpo-29418.8Qa9cQ.rst b/Misc/NEWS.d/next/Library/2020-03-31-20-53-11.bpo-29418.8Qa9cQ.rst new file mode 100644 index 0000000000000..b188ac3992d5a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-03-31-20-53-11.bpo-29418.8Qa9cQ.rst @@ -0,0 +1 @@ +Implement :func:`inspect.ismethodwrapper` and fix :func:`inspect.isroutine` for cases where methodwrapper is given. Patch by Hakan ?elik. From webhook-mailer at python.org Wed Feb 16 10:24:19 2022 From: webhook-mailer at python.org (corona10) Date: Wed, 16 Feb 2022 15:24:19 -0000 Subject: [Python-checkins] bpo-46541: Remove usage of _Py_IDENTIFIER from csv module (GH-31372) Message-ID: https://github.com/python/cpython/commit/d64f3caebe8f8e31ecd193e0bd25105400153ece commit: d64f3caebe8f8e31ecd193e0bd25105400153ece branch: main author: Dong-hee Na committer: corona10 date: 2022-02-17T00:24:03+09:00 summary: bpo-46541: Remove usage of _Py_IDENTIFIER from csv module (GH-31372) files: M Modules/_csv.c diff --git a/Modules/_csv.c b/Modules/_csv.c index f59d42a022e41..ebc3328d0cf0d 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -10,8 +10,6 @@ module instead. #define MODULE_VERSION "1.0" -#define NEEDS_PY_IDENTIFIER - #include "Python.h" #include "structmember.h" // PyMemberDef #include @@ -27,6 +25,7 @@ typedef struct { PyTypeObject *reader_type; PyTypeObject *writer_type; long field_limit; /* max parsed field size */ + PyObject *str_write; } _csvstate; static struct PyModuleDef _csvmodule; @@ -48,6 +47,7 @@ _csv_clear(PyObject *module) Py_CLEAR(module_state->dialect_type); Py_CLEAR(module_state->reader_type); Py_CLEAR(module_state->writer_type); + Py_CLEAR(module_state->str_write); return 0; } @@ -60,6 +60,7 @@ _csv_traverse(PyObject *module, visitproc visit, void *arg) Py_VISIT(module_state->dialect_type); Py_VISIT(module_state->reader_type); Py_VISIT(module_state->writer_type); + Py_VISIT(module_state->str_write); return 0; } @@ -1430,7 +1431,6 @@ csv_writer(PyObject *module, PyObject *args, PyObject *keyword_args) PyObject * output_file, * dialect = NULL; _csvstate *module_state = get_csv_state(module); WriterObj * self = PyObject_GC_New(WriterObj, module_state->writer_type); - _Py_IDENTIFIER(write); if (!self) return NULL; @@ -1449,7 +1449,9 @@ csv_writer(PyObject *module, PyObject *args, PyObject *keyword_args) Py_DECREF(self); return NULL; } - if (_PyObject_LookupAttrId(output_file, &PyId_write, &self->write) < 0) { + if (_PyObject_LookupAttr(output_file, + module_state->str_write, + &self->write) < 0) { Py_DECREF(self); return NULL; } @@ -1751,6 +1753,10 @@ csv_exec(PyObject *module) { return -1; } + module_state->str_write = PyUnicode_InternFromString("write"); + if (module_state->str_write == NULL) { + return -1; + } return 0; } From webhook-mailer at python.org Wed Feb 16 10:24:50 2022 From: webhook-mailer at python.org (corona10) Date: Wed, 16 Feb 2022 15:24:50 -0000 Subject: [Python-checkins] bpo-46541: Replace _Py_IDENTIFIER with _Py_ID in sqlite3 (GH-31351) Message-ID: https://github.com/python/cpython/commit/b2077117d125925210148294eefee28797b7ff4c commit: b2077117d125925210148294eefee28797b7ff4c branch: main author: Erlend Egeberg Aasland committer: corona10 date: 2022-02-17T00:24:44+09:00 summary: bpo-46541: Replace _Py_IDENTIFIER with _Py_ID in sqlite3 (GH-31351) files: M Modules/_sqlite/connection.c M Modules/_sqlite/cursor.c M Modules/_sqlite/microprotocols.c M Modules/_sqlite/module.c M Modules/_sqlite/module.h diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index cb8cd2a3d1807..0efb5ae35a7f5 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -21,8 +21,6 @@ * 3. This notice may not be removed or altered from any source distribution. */ -#define NEEDS_PY_IDENTIFIER - #include "module.h" #include "structmember.h" // PyMemberDef #include "connection.h" @@ -125,13 +123,12 @@ class _sqlite3.Connection "pysqlite_Connection *" "clinic_state()->ConnectionTyp [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=67369db2faf80891]*/ -_Py_IDENTIFIER(cursor); - static void _pysqlite_drop_unused_cursor_references(pysqlite_Connection* self); static void free_callback_context(callback_context *ctx); static void set_callback_context(callback_context **ctx_pp, callback_context *ctx); static void connection_close(pysqlite_Connection *self); +PyObject *_pysqlite_query_execute(pysqlite_Cursor *, int, PyObject *, PyObject *); static PyObject * new_statement_cache(pysqlite_Connection *self, pysqlite_state *state, @@ -782,7 +779,6 @@ final_callback(sqlite3_context *context) PyObject* function_result; PyObject** aggregate_instance; - _Py_IDENTIFIER(finalize); int ok; PyObject *exception, *value, *tb; @@ -801,8 +797,10 @@ final_callback(sqlite3_context *context) /* Keep the exception (if any) of the last call to step() */ PyErr_Fetch(&exception, &value, &tb); - function_result = _PyObject_CallMethodIdNoArgs(*aggregate_instance, &PyId_finalize); - + callback_context *ctx = (callback_context *)sqlite3_user_data(context); + assert(ctx != NULL); + function_result = PyObject_CallMethodNoArgs(*aggregate_instance, + ctx->state->str_finalize); Py_DECREF(*aggregate_instance); ok = 0; @@ -1432,16 +1430,14 @@ pysqlite_connection_execute_impl(pysqlite_Connection *self, PyObject *sql, PyObject *parameters) /*[clinic end generated code: output=5be05ae01ee17ee4 input=fbd17c75c7140271]*/ { - _Py_IDENTIFIER(execute); - PyObject* cursor = 0; PyObject* result = 0; - cursor = _PyObject_CallMethodIdNoArgs((PyObject*)self, &PyId_cursor); + PyObject *cursor = pysqlite_connection_cursor_impl(self, NULL); if (!cursor) { goto error; } - result = _PyObject_CallMethodIdObjArgs(cursor, &PyId_execute, sql, parameters, NULL); + result = _pysqlite_query_execute((pysqlite_Cursor *)cursor, 0, sql, parameters); if (!result) { Py_CLEAR(cursor); } @@ -1467,17 +1463,14 @@ pysqlite_connection_executemany_impl(pysqlite_Connection *self, PyObject *sql, PyObject *parameters) /*[clinic end generated code: output=776cd2fd20bfe71f input=4feab80659ffc82b]*/ { - _Py_IDENTIFIER(executemany); - PyObject* cursor = 0; PyObject* result = 0; - cursor = _PyObject_CallMethodIdNoArgs((PyObject*)self, &PyId_cursor); + PyObject *cursor = pysqlite_connection_cursor_impl(self, NULL); if (!cursor) { goto error; } - result = _PyObject_CallMethodIdObjArgs(cursor, &PyId_executemany, sql, - parameters, NULL); + result = _pysqlite_query_execute((pysqlite_Cursor *)cursor, 1, sql, parameters); if (!result) { Py_CLEAR(cursor); } @@ -1502,17 +1495,15 @@ pysqlite_connection_executescript(pysqlite_Connection *self, PyObject *script_obj) /*[clinic end generated code: output=4c4f9d77aa0ae37d input=b27ae5c24ffb8b43]*/ { - _Py_IDENTIFIER(executescript); - PyObject* cursor = 0; PyObject* result = 0; - cursor = _PyObject_CallMethodIdNoArgs((PyObject*)self, &PyId_cursor); + PyObject *cursor = pysqlite_connection_cursor_impl(self, NULL); if (!cursor) { goto error; } - result = _PyObject_CallMethodIdObjArgs(cursor, &PyId_executescript, - script_obj, NULL); + PyObject *meth = self->state->str_executescript; // borrowed ref. + result = PyObject_CallMethodObjArgs(cursor, meth, script_obj, NULL); if (!result) { Py_CLEAR(cursor); } @@ -1620,7 +1611,6 @@ static PyObject * pysqlite_connection_iterdump_impl(pysqlite_Connection *self) /*[clinic end generated code: output=586997aaf9808768 input=53bc907cb5eedb85]*/ { - _Py_IDENTIFIER(_iterdump); PyObject* retval = NULL; PyObject* module = NULL; PyObject* module_dict; @@ -1640,7 +1630,12 @@ pysqlite_connection_iterdump_impl(pysqlite_Connection *self) goto finally; } - pyfn_iterdump = _PyDict_GetItemIdWithError(module_dict, &PyId__iterdump); + PyObject *meth = PyUnicode_InternFromString("_iterdump"); + if (meth == NULL) { + goto finally; + } + pyfn_iterdump = PyDict_GetItemWithError(module_dict, meth); + Py_DECREF(meth); if (!pyfn_iterdump) { if (!PyErr_Occurred()) { PyErr_SetString(self->OperationalError, diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c index 395b10f33ff3a..b0dd9dd39c8ee 100644 --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -21,8 +21,6 @@ * 3. This notice may not be removed or altered from any source distribution. */ -#define NEEDS_PY_IDENTIFIER - #include "cursor.h" #include "module.h" #include "util.h" @@ -131,13 +129,12 @@ _pysqlite_get_converter(pysqlite_state *state, const char *keystr, PyObject *key; PyObject *upcase_key; PyObject *retval; - _Py_IDENTIFIER(upper); key = PyUnicode_FromStringAndSize(keystr, keylen); if (!key) { return NULL; } - upcase_key = _PyObject_CallMethodIdNoArgs(key, &PyId_upper); + upcase_key = PyObject_CallMethodNoArgs(key, state->str_upper); Py_DECREF(key); if (!upcase_key) { return NULL; @@ -457,7 +454,7 @@ get_statement_from_cache(pysqlite_Cursor *self, PyObject *operation) return PyObject_Vectorcall(cache, args + 1, nargsf, NULL); } -static PyObject * +PyObject * _pysqlite_query_execute(pysqlite_Cursor* self, int multiple, PyObject* operation, PyObject* second_argument) { PyObject* parameters_list = NULL; diff --git a/Modules/_sqlite/microprotocols.c b/Modules/_sqlite/microprotocols.c index 697d1a788a3b0..a79f0067b17e8 100644 --- a/Modules/_sqlite/microprotocols.c +++ b/Modules/_sqlite/microprotocols.c @@ -23,8 +23,6 @@ * 3. This notice may not be removed or altered from any source distribution. */ -#define NEEDS_PY_IDENTIFIER - #include #include "cursor.h" @@ -76,8 +74,6 @@ PyObject * pysqlite_microprotocols_adapt(pysqlite_state *state, PyObject *obj, PyObject *proto, PyObject *alt) { - _Py_IDENTIFIER(__adapt__); - _Py_IDENTIFIER(__conform__); PyObject *adapter, *key, *adapted; /* we don't check for exact type conformance as specified in PEP 246 @@ -102,7 +98,7 @@ pysqlite_microprotocols_adapt(pysqlite_state *state, PyObject *obj, } /* try to have the protocol adapt this object */ - if (_PyObject_LookupAttrId(proto, &PyId___adapt__, &adapter) < 0) { + if (_PyObject_LookupAttr(proto, state->str___adapt__, &adapter) < 0) { return NULL; } if (adapter) { @@ -121,7 +117,7 @@ pysqlite_microprotocols_adapt(pysqlite_state *state, PyObject *obj, } /* and finally try to have the object adapt itself */ - if (_PyObject_LookupAttrId(obj, &PyId___conform__, &adapter) < 0) { + if (_PyObject_LookupAttr(obj, state->str___conform__, &adapter) < 0) { return NULL; } if (adapter) { diff --git a/Modules/_sqlite/module.c b/Modules/_sqlite/module.c index cfa4b970e7320..35cdcbe0a5c38 100644 --- a/Modules/_sqlite/module.c +++ b/Modules/_sqlite/module.c @@ -21,8 +21,6 @@ * 3. This notice may not be removed or altered from any source distribution. */ -#define NEEDS_PY_IDENTIFIER - #include "connection.h" #include "statement.h" #include "cursor.h" @@ -185,15 +183,14 @@ pysqlite_register_converter_impl(PyObject *module, PyObject *orig_name, { PyObject* name = NULL; PyObject* retval = NULL; - _Py_IDENTIFIER(upper); /* convert the name to upper case */ - name = _PyObject_CallMethodIdNoArgs(orig_name, &PyId_upper); + pysqlite_state *state = pysqlite_get_state(module); + name = PyObject_CallMethodNoArgs(orig_name, state->str_upper); if (!name) { goto error; } - pysqlite_state *state = pysqlite_get_state(module); if (PyDict_SetItem(state->converters, name, callable) != 0) { goto error; } @@ -593,6 +590,13 @@ module_traverse(PyObject *module, visitproc visit, void *arg) Py_VISIT(state->lru_cache); Py_VISIT(state->psyco_adapters); + // Interned strings + Py_VISIT(state->str___adapt__); + Py_VISIT(state->str___conform__); + Py_VISIT(state->str_executescript); + Py_VISIT(state->str_finalize); + Py_VISIT(state->str_upper); + return 0; } @@ -625,6 +629,13 @@ module_clear(PyObject *module) Py_CLEAR(state->lru_cache); Py_CLEAR(state->psyco_adapters); + // Interned strings + Py_CLEAR(state->str___adapt__); + Py_CLEAR(state->str___conform__); + Py_CLEAR(state->str_executescript); + Py_CLEAR(state->str_finalize); + Py_CLEAR(state->str_upper); + return 0; } @@ -650,6 +661,15 @@ do { \ ADD_TYPE(module, (PyTypeObject *)state->exc); \ } while (0) +#define ADD_INTERNED(state, string) \ +do { \ + PyObject *tmp = PyUnicode_InternFromString(#string); \ + if (tmp == NULL) { \ + goto error; \ + } \ + state->str_ ## string = tmp; \ +} while (0) + static int module_exec(PyObject *module) { @@ -695,6 +715,13 @@ module_exec(PyObject *module) ADD_EXCEPTION(module, state, DataError, state->DatabaseError); ADD_EXCEPTION(module, state, NotSupportedError, state->DatabaseError); + /* Add interned strings */ + ADD_INTERNED(state, __adapt__); + ADD_INTERNED(state, __conform__); + ADD_INTERNED(state, executescript); + ADD_INTERNED(state, finalize); + ADD_INTERNED(state, upper); + /* Set error constants */ if (add_error_constants(module) < 0) { goto error; diff --git a/Modules/_sqlite/module.h b/Modules/_sqlite/module.h index a248d044af8fd..35c6f38552606 100644 --- a/Modules/_sqlite/module.h +++ b/Modules/_sqlite/module.h @@ -58,6 +58,13 @@ typedef struct { PyTypeObject *PrepareProtocolType; PyTypeObject *RowType; PyTypeObject *StatementType; + + /* Pointers to interned strings */ + PyObject *str___adapt__; + PyObject *str___conform__; + PyObject *str_executescript; + PyObject *str_finalize; + PyObject *str_upper; } pysqlite_state; extern pysqlite_state pysqlite_global_state; From webhook-mailer at python.org Wed Feb 16 11:04:43 2022 From: webhook-mailer at python.org (corona10) Date: Wed, 16 Feb 2022 16:04:43 -0000 Subject: [Python-checkins] bpo-46541: Remove usage of _Py_IDENTIFIER from mmap module (GH-31375) Message-ID: https://github.com/python/cpython/commit/e8a19b092fc0551581e10d6f310dd5feabac7609 commit: e8a19b092fc0551581e10d6f310dd5feabac7609 branch: main author: Dong-hee Na committer: corona10 date: 2022-02-17T01:04:38+09:00 summary: bpo-46541: Remove usage of _Py_IDENTIFIER from mmap module (GH-31375) files: M Modules/mmapmodule.c diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c index 0a216941d0248..26cedf1b9006d 100644 --- a/Modules/mmapmodule.c +++ b/Modules/mmapmodule.c @@ -21,7 +21,6 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif -#define NEEDS_PY_IDENTIFIER #define PY_SSIZE_T_CLEAN #include @@ -771,9 +770,7 @@ mmap__enter__method(mmap_object *self, PyObject *args) static PyObject * mmap__exit__method(PyObject *self, PyObject *args) { - _Py_IDENTIFIER(close); - - return _PyObject_CallMethodIdNoArgs(self, &PyId_close); + return mmap_close_method((mmap_object *)self, NULL); } static PyObject * From webhook-mailer at python.org Wed Feb 16 11:48:21 2022 From: webhook-mailer at python.org (brandtbucher) Date: Wed, 16 Feb 2022 16:48:21 -0000 Subject: [Python-checkins] bpo-46702: Specialize UNPACK_SEQUENCE (GH-31240) Message-ID: https://github.com/python/cpython/commit/a9da085015db8cbb81f660158864ac94fe6c67a2 commit: a9da085015db8cbb81f660158864ac94fe6c67a2 branch: main author: Brandt Bucher committer: brandtbucher date: 2022-02-16T08:48:16-08:00 summary: bpo-46702: Specialize UNPACK_SEQUENCE (GH-31240) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-09-16-36-11.bpo-46702.LcaEuC.rst M Include/internal/pycore_code.h M Include/opcode.h M Lib/opcode.py M Python/ceval.c M Python/opcode_targets.h M Python/specialize.c diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index 2d8fe20e1a6f0..ead9541c29843 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -276,6 +276,8 @@ int _Py_Specialize_CallNoKw(PyObject *callable, _Py_CODEUNIT *instr, int nargs, void _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); void _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); +void _Py_Specialize_UnpackSequence(PyObject *seq, _Py_CODEUNIT *instr, + SpecializedCacheEntry *cache); /* Deallocator function for static codeobjects used in deepfreeze.py */ void _PyStaticCode_Dealloc(PyCodeObject *co); diff --git a/Include/opcode.h b/Include/opcode.h index 58fc62808937f..df45e7b5e5a30 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -169,11 +169,15 @@ extern "C" { #define STORE_ATTR_INSTANCE_VALUE 81 #define STORE_ATTR_SLOT 131 #define STORE_ATTR_WITH_HINT 140 -#define LOAD_FAST__LOAD_FAST 141 -#define STORE_FAST__LOAD_FAST 143 -#define LOAD_FAST__LOAD_CONST 150 -#define LOAD_CONST__LOAD_FAST 153 -#define STORE_FAST__STORE_FAST 154 +#define UNPACK_SEQUENCE_ADAPTIVE 141 +#define UNPACK_SEQUENCE_LIST 143 +#define UNPACK_SEQUENCE_TUPLE 150 +#define UNPACK_SEQUENCE_TWO_TUPLE 153 +#define LOAD_FAST__LOAD_FAST 154 +#define STORE_FAST__LOAD_FAST 158 +#define LOAD_FAST__LOAD_CONST 159 +#define LOAD_CONST__LOAD_FAST 161 +#define STORE_FAST__STORE_FAST 166 #define DO_TRACING 255 #ifdef NEED_OPCODE_JUMP_TABLES static uint32_t _PyOpcode_RelativeJump[8] = { diff --git a/Lib/opcode.py b/Lib/opcode.py index a1f0c6e4326d1..8237aa7fa3d11 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -283,6 +283,10 @@ def jabs_op(name, op): "STORE_ATTR_INSTANCE_VALUE", "STORE_ATTR_SLOT", "STORE_ATTR_WITH_HINT", + "UNPACK_SEQUENCE_ADAPTIVE", + "UNPACK_SEQUENCE_LIST", + "UNPACK_SEQUENCE_TUPLE", + "UNPACK_SEQUENCE_TWO_TUPLE", # Super instructions "LOAD_FAST__LOAD_FAST", "STORE_FAST__LOAD_FAST", diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-09-16-36-11.bpo-46702.LcaEuC.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-09-16-36-11.bpo-46702.LcaEuC.rst new file mode 100644 index 0000000000000..8fe758528960f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-09-16-36-11.bpo-46702.LcaEuC.rst @@ -0,0 +1,2 @@ +Specialize :opcode:`UNPACK_SEQUENCE` for :class:`tuple` and :class:`list` +unpackings. diff --git a/Python/ceval.c b/Python/ceval.c index ad8b05400d564..b900de53c8c92 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2738,52 +2738,84 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(UNPACK_SEQUENCE) { PREDICTED(UNPACK_SEQUENCE); - PyObject *seq = POP(), *item, **items; -#ifdef Py_STATS - extern int _PySpecialization_ClassifySequence(PyObject *, int); - _py_stats.opcode_stats[UNPACK_SEQUENCE].specialization.failure++; - _py_stats.opcode_stats[UNPACK_SEQUENCE].specialization. - failure_kinds[_PySpecialization_ClassifySequence(seq, oparg)]++; -#endif - if (PyTuple_CheckExact(seq) && - PyTuple_GET_SIZE(seq) == oparg) { - items = ((PyTupleObject *)seq)->ob_item; - while (oparg--) { - item = items[oparg]; - Py_INCREF(item); - PUSH(item); - } - } else if (PyList_CheckExact(seq) && - PyList_GET_SIZE(seq) == oparg) { - items = ((PyListObject *)seq)->ob_item; - while (oparg--) { - item = items[oparg]; - Py_INCREF(item); - PUSH(item); - } - } else if (unpack_iterable(tstate, seq, oparg, -1, - stack_pointer + oparg)) { - STACK_GROW(oparg); - } else { - /* unpack_iterable() raised an exception */ + PyObject *seq = POP(); + PyObject **top = stack_pointer + oparg; + if (!unpack_iterable(tstate, seq, oparg, -1, top)) { Py_DECREF(seq); goto error; } + STACK_GROW(oparg); Py_DECREF(seq); DISPATCH(); } + TARGET(UNPACK_SEQUENCE_ADAPTIVE) { + assert(cframe.use_tracing == 0); + SpecializedCacheEntry *cache = GET_CACHE(); + if (cache->adaptive.counter == 0) { + PyObject *seq = TOP(); + next_instr--; + _Py_Specialize_UnpackSequence(seq, next_instr, cache); + DISPATCH(); + } + else { + STAT_INC(UNPACK_SEQUENCE, deferred); + cache->adaptive.counter--; + oparg = cache->adaptive.original_oparg; + JUMP_TO_INSTRUCTION(UNPACK_SEQUENCE); + } + } + + TARGET(UNPACK_SEQUENCE_TWO_TUPLE) { + PyObject *seq = TOP(); + DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); + DEOPT_IF(PyTuple_GET_SIZE(seq) != 2, UNPACK_SEQUENCE); + STAT_INC(UNPACK_SEQUENCE, hit); + SET_TOP(Py_NewRef(PyTuple_GET_ITEM(seq, 1))); + PUSH(Py_NewRef(PyTuple_GET_ITEM(seq, 0))); + Py_DECREF(seq); + NOTRACE_DISPATCH(); + } + + TARGET(UNPACK_SEQUENCE_TUPLE) { + PyObject *seq = TOP(); + int len = GET_CACHE()->adaptive.original_oparg; + DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); + DEOPT_IF(PyTuple_GET_SIZE(seq) != len, UNPACK_SEQUENCE); + STAT_INC(UNPACK_SEQUENCE, hit); + STACK_SHRINK(1); + PyObject **items = _PyTuple_ITEMS(seq); + while (len--) { + PUSH(Py_NewRef(items[len])); + } + Py_DECREF(seq); + NOTRACE_DISPATCH(); + } + + TARGET(UNPACK_SEQUENCE_LIST) { + PyObject *seq = TOP(); + int len = GET_CACHE()->adaptive.original_oparg; + DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE); + DEOPT_IF(PyList_GET_SIZE(seq) != len, UNPACK_SEQUENCE); + STAT_INC(UNPACK_SEQUENCE, hit); + STACK_SHRINK(1); + PyObject **items = _PyList_ITEMS(seq); + while (len--) { + PUSH(Py_NewRef(items[len])); + } + Py_DECREF(seq); + NOTRACE_DISPATCH(); + } + TARGET(UNPACK_EX) { int totalargs = 1 + (oparg & 0xFF) + (oparg >> 8); PyObject *seq = POP(); - - if (unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, - stack_pointer + totalargs)) { - stack_pointer += totalargs; - } else { + PyObject **top = stack_pointer + totalargs; + if (!unpack_iterable(tstate, seq, oparg & 0xFF, oparg >> 8, top)) { Py_DECREF(seq); goto error; } + STACK_GROW(totalargs); Py_DECREF(seq); DISPATCH(); } @@ -5396,6 +5428,7 @@ MISS_WITH_CACHE(CALL) MISS_WITH_CACHE(BINARY_OP) MISS_WITH_CACHE(COMPARE_OP) MISS_WITH_CACHE(BINARY_SUBSCR) +MISS_WITH_CACHE(UNPACK_SEQUENCE) MISS_WITH_OPARG_COUNTER(STORE_SUBSCR) binary_subscr_dict_error: diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index f47da2bbb1ef0..1e137f93c7292 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -140,32 +140,32 @@ static void *opcode_targets[256] = { &&TARGET_STORE_DEREF, &&TARGET_DELETE_DEREF, &&TARGET_STORE_ATTR_WITH_HINT, - &&TARGET_LOAD_FAST__LOAD_FAST, + &&TARGET_UNPACK_SEQUENCE_ADAPTIVE, &&TARGET_CALL_FUNCTION_EX, - &&TARGET_STORE_FAST__LOAD_FAST, + &&TARGET_UNPACK_SEQUENCE_LIST, &&TARGET_EXTENDED_ARG, &&TARGET_LIST_APPEND, &&TARGET_SET_ADD, &&TARGET_MAP_ADD, &&TARGET_LOAD_CLASSDEREF, &&TARGET_COPY_FREE_VARS, - &&TARGET_LOAD_FAST__LOAD_CONST, + &&TARGET_UNPACK_SEQUENCE_TUPLE, &&TARGET_RESUME, &&TARGET_MATCH_CLASS, - &&TARGET_LOAD_CONST__LOAD_FAST, - &&TARGET_STORE_FAST__STORE_FAST, + &&TARGET_UNPACK_SEQUENCE_TWO_TUPLE, + &&TARGET_LOAD_FAST__LOAD_FAST, &&TARGET_FORMAT_VALUE, &&TARGET_BUILD_CONST_KEY_MAP, &&TARGET_BUILD_STRING, - &&_unknown_opcode, - &&_unknown_opcode, + &&TARGET_STORE_FAST__LOAD_FAST, + &&TARGET_LOAD_FAST__LOAD_CONST, &&TARGET_LOAD_METHOD, - &&_unknown_opcode, + &&TARGET_LOAD_CONST__LOAD_FAST, &&TARGET_LIST_EXTEND, &&TARGET_SET_UPDATE, &&TARGET_DICT_MERGE, &&TARGET_DICT_UPDATE, - &&_unknown_opcode, + &&TARGET_STORE_FAST__STORE_FAST, &&TARGET_PRECALL_FUNCTION, &&TARGET_PRECALL_METHOD, &&_unknown_opcode, diff --git a/Python/specialize.c b/Python/specialize.c index 0e1ffad1b9159..ab2363cab9f3e 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -51,6 +51,7 @@ static uint8_t adaptive_opcodes[256] = { [STORE_ATTR] = STORE_ATTR_ADAPTIVE, [BINARY_OP] = BINARY_OP_ADAPTIVE, [COMPARE_OP] = COMPARE_OP_ADAPTIVE, + [UNPACK_SEQUENCE] = UNPACK_SEQUENCE_ADAPTIVE, }; /* The number of cache entries required for a "family" of instructions. */ @@ -64,6 +65,7 @@ static uint8_t cache_requirements[256] = { [STORE_ATTR] = 2, /* _PyAdaptiveEntry and _PyAttrCache */ [BINARY_OP] = 1, // _PyAdaptiveEntry [COMPARE_OP] = 1, /* _PyAdaptiveEntry */ + [UNPACK_SEQUENCE] = 1, // _PyAdaptiveEntry }; Py_ssize_t _Py_QuickenedCount = 0; @@ -155,6 +157,7 @@ _Py_GetSpecializationStats(void) { err += add_stat_dict(stats, CALL, "call"); err += add_stat_dict(stats, BINARY_OP, "binary_op"); err += add_stat_dict(stats, COMPARE_OP, "compare_op"); + err += add_stat_dict(stats, UNPACK_SEQUENCE, "unpack_sequence"); if (err < 0) { Py_DECREF(stats); return NULL; @@ -607,27 +610,10 @@ initial_counter_value(void) { #define SPEC_FAIL_FOR_ITER_DICT_VALUES 22 #define SPEC_FAIL_FOR_ITER_ENUMERATE 23 -/* UNPACK_SEQUENCE */ -#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_0 9 -#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_1 10 -#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_2 11 -#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_3 12 -#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_4 13 -#define SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_N 14 - -#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_0 15 -#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_1 16 -#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_2 17 -#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_3 18 -#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_4 19 -#define SPEC_FAIL_UNPACK_SEQUENCE_LIST_N 20 - -#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_0 21 -#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_1 22 -#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_2 23 -#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_3 24 -#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_4 25 -#define SPEC_FAIL_UNPACK_SEQUENCE_OTHER_N 26 +// UNPACK_SEQUENCE + +#define SPEC_FAIL_UNPACK_SEQUENCE_ITERATOR 8 +#define SPEC_FAIL_UNPACK_SEQUENCE_SEQUENCE 9 static int @@ -1949,6 +1935,56 @@ _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, adaptive->counter = initial_counter_value(); } +#ifdef Py_STATS +static int +unpack_sequence_fail_kind(PyObject *seq) +{ + if (PySequence_Check(seq)) { + return SPEC_FAIL_UNPACK_SEQUENCE_SEQUENCE; + } + if (PyIter_Check(seq)) { + return SPEC_FAIL_UNPACK_SEQUENCE_ITERATOR; + } + return SPEC_FAIL_OTHER; +} +#endif + +void +_Py_Specialize_UnpackSequence(PyObject *seq, _Py_CODEUNIT *instr, + SpecializedCacheEntry *cache) +{ + _PyAdaptiveEntry *adaptive = &cache->adaptive; + if (PyTuple_CheckExact(seq)) { + if (PyTuple_GET_SIZE(seq) != adaptive->original_oparg) { + SPECIALIZATION_FAIL(UNPACK_SEQUENCE, SPEC_FAIL_EXPECTED_ERROR); + goto failure; + } + if (PyTuple_GET_SIZE(seq) == 2) { + *instr = _Py_MAKECODEUNIT(UNPACK_SEQUENCE_TWO_TUPLE, + _Py_OPARG(*instr)); + goto success; + } + *instr = _Py_MAKECODEUNIT(UNPACK_SEQUENCE_TUPLE, _Py_OPARG(*instr)); + goto success; + } + if (PyList_CheckExact(seq)) { + if (PyList_GET_SIZE(seq) != adaptive->original_oparg) { + SPECIALIZATION_FAIL(UNPACK_SEQUENCE, SPEC_FAIL_EXPECTED_ERROR); + goto failure; + } + *instr = _Py_MAKECODEUNIT(UNPACK_SEQUENCE_LIST, _Py_OPARG(*instr)); + goto success; + } + SPECIALIZATION_FAIL(UNPACK_SEQUENCE, unpack_sequence_fail_kind(seq)); +failure: + STAT_INC(UNPACK_SEQUENCE, failure); + cache_backoff(adaptive); + return; +success: + STAT_INC(UNPACK_SEQUENCE, success); + adaptive->counter = initial_counter_value(); +} + #ifdef Py_STATS int @@ -2001,22 +2037,6 @@ int return SPEC_FAIL_OTHER; } -int -_PySpecialization_ClassifySequence(PyObject *seq, int n) -{ - assert(n >= 0); - if (n > 4) { - n = 5; - } - if (PyTuple_CheckExact(seq)) { - return SPEC_FAIL_UNPACK_SEQUENCE_TUPLE_0 + n; - } - if (PyList_CheckExact(seq)) { - return SPEC_FAIL_UNPACK_SEQUENCE_LIST_0 + n; - } - return SPEC_FAIL_UNPACK_SEQUENCE_OTHER_0 + n; -} - int _PySpecialization_ClassifyCallable(PyObject *callable) { From webhook-mailer at python.org Wed Feb 16 11:50:03 2022 From: webhook-mailer at python.org (brandtbucher) Date: Wed, 16 Feb 2022 16:50:03 -0000 Subject: [Python-checkins] bpo-46072: Add detailed failure stats for BINARY_OP (GH-31289) Message-ID: https://github.com/python/cpython/commit/580cd9ab2992b7df6f4815020b5841e14a5a6977 commit: 580cd9ab2992b7df6f4815020b5841e14a5a6977 branch: main author: Brandt Bucher committer: brandtbucher date: 2022-02-16T08:49:58-08:00 summary: bpo-46072: Add detailed failure stats for BINARY_OP (GH-31289) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-11-13-47-58.bpo-46072.PDS6Ke.rst M Python/specialize.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-11-13-47-58.bpo-46072.PDS6Ke.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-11-13-47-58.bpo-46072.PDS6Ke.rst new file mode 100644 index 0000000000000..aa9ea64c129f6 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-11-13-47-58.bpo-46072.PDS6Ke.rst @@ -0,0 +1 @@ +Add more detailed specialization failure statistics for :opcode:`BINARY_OP`. diff --git a/Python/specialize.c b/Python/specialize.c index ab2363cab9f3e..5fd7d0923a698 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -552,9 +552,28 @@ initial_counter_value(void) { #define SPEC_FAIL_SUBSCR_PY_OTHER 21 #define SPEC_FAIL_SUBSCR_DICT_SUBCLASS_NO_OVERRIDE 22 -/* Binary add */ - -#define SPEC_FAIL_BINARY_OP_DIFFERENT_TYPES 12 +/* Binary op */ + +#define SPEC_FAIL_BINARY_OP_ADD_DIFFERENT_TYPES 8 +#define SPEC_FAIL_BINARY_OP_ADD_OTHER 9 +#define SPEC_FAIL_BINARY_OP_AND_DIFFERENT_TYPES 10 +#define SPEC_FAIL_BINARY_OP_AND_INT 11 +#define SPEC_FAIL_BINARY_OP_AND_OTHER 12 +#define SPEC_FAIL_BINARY_OP_FLOOR_DIVIDE 13 +#define SPEC_FAIL_BINARY_OP_LSHIFT 14 +#define SPEC_FAIL_BINARY_OP_MATRIX_MULTIPLY 15 +#define SPEC_FAIL_BINARY_OP_MULTIPLY_DIFFERENT_TYPES 16 +#define SPEC_FAIL_BINARY_OP_MULTIPLY_OTHER 17 +#define SPEC_FAIL_BINARY_OP_OR 18 +#define SPEC_FAIL_BINARY_OP_POWER 19 +#define SPEC_FAIL_BINARY_OP_REMAINDER 20 +#define SPEC_FAIL_BINARY_OP_RSHIFT 21 +#define SPEC_FAIL_BINARY_OP_SUBTRACT_DIFFERENT_TYPES 22 +#define SPEC_FAIL_BINARY_OP_SUBTRACT_OTHER 23 +#define SPEC_FAIL_BINARY_OP_TRUE_DIVIDE_DIFFERENT_TYPES 24 +#define SPEC_FAIL_BINARY_OP_TRUE_DIVIDE_FLOAT 25 +#define SPEC_FAIL_BINARY_OP_TRUE_DIVIDE_OTHER 26 +#define SPEC_FAIL_BINARY_OP_XOR 27 /* Calls */ #define SPEC_FAIL_CALL_COMPLEX_PARAMETERS 9 @@ -1745,6 +1764,76 @@ _Py_Specialize_CallNoKw( return 0; } +#ifdef Py_STATS +static int +binary_op_fail_kind(int oparg, PyObject *lhs, PyObject *rhs) +{ + switch (oparg) { + case NB_ADD: + case NB_INPLACE_ADD: + if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { + return SPEC_FAIL_BINARY_OP_ADD_DIFFERENT_TYPES; + } + return SPEC_FAIL_BINARY_OP_ADD_OTHER; + case NB_AND: + case NB_INPLACE_AND: + if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { + return SPEC_FAIL_BINARY_OP_AND_DIFFERENT_TYPES; + } + if (PyLong_CheckExact(lhs)) { + return SPEC_FAIL_BINARY_OP_AND_INT; + } + return SPEC_FAIL_BINARY_OP_AND_OTHER; + case NB_FLOOR_DIVIDE: + case NB_INPLACE_FLOOR_DIVIDE: + return SPEC_FAIL_BINARY_OP_FLOOR_DIVIDE; + case NB_LSHIFT: + case NB_INPLACE_LSHIFT: + return SPEC_FAIL_BINARY_OP_LSHIFT; + case NB_MATRIX_MULTIPLY: + case NB_INPLACE_MATRIX_MULTIPLY: + return SPEC_FAIL_BINARY_OP_MATRIX_MULTIPLY; + case NB_MULTIPLY: + case NB_INPLACE_MULTIPLY: + if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { + return SPEC_FAIL_BINARY_OP_MULTIPLY_DIFFERENT_TYPES; + } + return SPEC_FAIL_BINARY_OP_MULTIPLY_OTHER; + case NB_OR: + case NB_INPLACE_OR: + return SPEC_FAIL_BINARY_OP_OR; + case NB_POWER: + case NB_INPLACE_POWER: + return SPEC_FAIL_BINARY_OP_POWER; + case NB_REMAINDER: + case NB_INPLACE_REMAINDER: + return SPEC_FAIL_BINARY_OP_REMAINDER; + case NB_RSHIFT: + case NB_INPLACE_RSHIFT: + return SPEC_FAIL_BINARY_OP_RSHIFT; + case NB_SUBTRACT: + case NB_INPLACE_SUBTRACT: + if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { + return SPEC_FAIL_BINARY_OP_SUBTRACT_DIFFERENT_TYPES; + } + return SPEC_FAIL_BINARY_OP_SUBTRACT_OTHER; + case NB_TRUE_DIVIDE: + case NB_INPLACE_TRUE_DIVIDE: + if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { + return SPEC_FAIL_BINARY_OP_TRUE_DIVIDE_DIFFERENT_TYPES; + } + if (PyFloat_CheckExact(lhs)) { + return SPEC_FAIL_BINARY_OP_TRUE_DIVIDE_FLOAT; + } + return SPEC_FAIL_BINARY_OP_TRUE_DIVIDE_OTHER; + case NB_XOR: + case NB_INPLACE_XOR: + return SPEC_FAIL_BINARY_OP_XOR; + } + Py_UNREACHABLE(); +} +#endif + void _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache) @@ -1754,8 +1843,7 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, case NB_ADD: case NB_INPLACE_ADD: if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { - SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_BINARY_OP_DIFFERENT_TYPES); - goto failure; + break; } if (PyUnicode_CheckExact(lhs)) { if (_Py_OPCODE(instr[1]) == STORE_FAST && Py_REFCNT(lhs) == 2) { @@ -1780,8 +1868,7 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, case NB_MULTIPLY: case NB_INPLACE_MULTIPLY: if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { - SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_BINARY_OP_DIFFERENT_TYPES); - goto failure; + break; } if (PyLong_CheckExact(lhs)) { *instr = _Py_MAKECODEUNIT(BINARY_OP_MULTIPLY_INT, @@ -1797,8 +1884,7 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, case NB_SUBTRACT: case NB_INPLACE_SUBTRACT: if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { - SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_BINARY_OP_DIFFERENT_TYPES); - goto failure; + break; } if (PyLong_CheckExact(lhs)) { *instr = _Py_MAKECODEUNIT(BINARY_OP_SUBTRACT_INT, @@ -1811,14 +1897,19 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, goto success; } break; +#ifndef Py_STATS default: // These operators don't have any available specializations. Rather // than repeatedly attempting to specialize them, just convert them - // back to BINARY_OP (while still recording a failure, of course)! + // back to BINARY_OP (unless we're collecting stats, where it's more + // important to get accurate hit counts for the unadaptive version + // and each of the different failure types): *instr = _Py_MAKECODEUNIT(BINARY_OP, adaptive->original_oparg); + return; +#endif } - SPECIALIZATION_FAIL(BINARY_OP, SPEC_FAIL_OTHER); -failure: + SPECIALIZATION_FAIL( + BINARY_OP, binary_op_fail_kind(adaptive->original_oparg, lhs, rhs)); STAT_INC(BINARY_OP, failure); cache_backoff(adaptive); return; From webhook-mailer at python.org Wed Feb 16 14:02:15 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Wed, 16 Feb 2022 19:02:15 -0000 Subject: [Python-checkins] Become a CODEOWNER for typing (#31374) Message-ID: https://github.com/python/cpython/commit/6f1efd19a70839d480e4b1fcd9fecd3a8725824b commit: 6f1efd19a70839d480e4b1fcd9fecd3a8725824b branch: main author: Jelle Zijlstra committer: JelleZijlstra date: 2022-02-16T11:02:06-08:00 summary: Become a CODEOWNER for typing (#31374) files: M .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index f484664f7b712..a884713ec500f 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -136,7 +136,7 @@ Lib/ast.py @isidentical **/*idlelib* @terryjreedy -**/*typing* @gvanrossum @Fidget-Spinner +**/*typing* @gvanrossum @Fidget-Spinner @JelleZijlstra **/*asyncore @giampaolo **/*asynchat @giampaolo From webhook-mailer at python.org Wed Feb 16 22:26:18 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Thu, 17 Feb 2022 03:26:18 -0000 Subject: [Python-checkins] bpo-46066: Deprecate kwargs syntax for TypedDict definitions (GH-31126) Message-ID: https://github.com/python/cpython/commit/de6043e596492201cc1a1eb28038970bb69f3107 commit: de6043e596492201cc1a1eb28038970bb69f3107 branch: main author: 97littleleaf11 <11172084+97littleleaf11 at users.noreply.github.com> committer: JelleZijlstra date: 2022-02-16T19:26:07-08:00 summary: bpo-46066: Deprecate kwargs syntax for TypedDict definitions (GH-31126) Closes python/typing#981 https://bugs.python.org/issue46066 files: A Misc/NEWS.d/next/Library/2022-02-08-16-42-20.bpo-46066.m32Hl0.rst M Doc/library/typing.rst M Lib/test/test_typing.py M Lib/typing.py M Misc/ACKS diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 94a46b01a1a8c..8240c912c6497 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1470,11 +1470,20 @@ These are not used in annotations. They are building blocks for declaring types. ``Point2D.__optional_keys__``. To allow using this feature with older versions of Python that do not support :pep:`526`, ``TypedDict`` supports two additional equivalent - syntactic forms:: + syntactic forms: + + * Using a literal :class:`dict` as the second argument:: - Point2D = TypedDict('Point2D', x=int, y=int, label=str) Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str}) + * Using keyword arguments:: + + Point2D = TypedDict('Point2D', x=int, y=int, label=str) + + .. deprecated-removed:: 3.11 3.13 + The keyword-argument syntax is deprecated in 3.11 and will be removed + in 3.13. It may also be unsupported by static type checkers. + By default, all keys must be present in a ``TypedDict``. It is possible to override this by specifying totality. Usage:: @@ -1483,6 +1492,9 @@ These are not used in annotations. They are building blocks for declaring types. x: int y: int + # Alternative syntax + Point2D = TypedDict('Point2D', {'x': int, 'y': int}, total=False) + This means that a ``Point2D`` ``TypedDict`` can have any of the keys omitted. A type checker is only expected to support a literal ``False`` or ``True`` as the value of the ``total`` argument. ``True`` is the default, diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 2bb5d61068a78..9548a0cab1d1e 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4380,7 +4380,8 @@ def test_basics_functional_syntax(self): self.assertEqual(Emp.__total__, True) def test_basics_keywords_syntax(self): - Emp = TypedDict('Emp', name=str, id=int) + with self.assertWarns(DeprecationWarning): + Emp = TypedDict('Emp', name=str, id=int) self.assertIsSubclass(Emp, dict) self.assertIsSubclass(Emp, typing.MutableMapping) self.assertNotIsSubclass(Emp, collections.abc.Sequence) @@ -4395,7 +4396,8 @@ def test_basics_keywords_syntax(self): self.assertEqual(Emp.__total__, True) def test_typeddict_special_keyword_names(self): - TD = TypedDict("TD", cls=type, self=object, typename=str, _typename=int, fields=list, _fields=dict) + with self.assertWarns(DeprecationWarning): + TD = TypedDict("TD", cls=type, self=object, typename=str, _typename=int, fields=list, _fields=dict) self.assertEqual(TD.__name__, 'TD') self.assertEqual(TD.__annotations__, {'cls': type, 'self': object, 'typename': str, '_typename': int, 'fields': list, '_fields': dict}) a = TD(cls=str, self=42, typename='foo', _typename=53, fields=[('bar', tuple)], _fields={'baz', set}) @@ -4451,7 +4453,7 @@ def test_py36_class_syntax_usage(self): def test_pickle(self): global EmpD # pickle wants to reference the class by name - EmpD = TypedDict('EmpD', name=str, id=int) + EmpD = TypedDict('EmpD', {'name': str, 'id': int}) jane = EmpD({'name': 'jane', 'id': 37}) for proto in range(pickle.HIGHEST_PROTOCOL + 1): z = pickle.dumps(jane, proto) @@ -4463,7 +4465,7 @@ def test_pickle(self): self.assertEqual(EmpDnew({'name': 'jane', 'id': 37}), jane) def test_optional(self): - EmpD = TypedDict('EmpD', name=str, id=int) + EmpD = TypedDict('EmpD', {'name': str, 'id': int}) self.assertEqual(typing.Optional[EmpD], typing.Union[None, EmpD]) self.assertNotEqual(typing.List[EmpD], typing.Tuple[EmpD]) diff --git a/Lib/typing.py b/Lib/typing.py index 4a8bdf8132861..3233827d816d1 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -2569,9 +2569,8 @@ class Point2D(TypedDict): The type info can be accessed via the Point2D.__annotations__ dict, and the Point2D.__required_keys__ and Point2D.__optional_keys__ frozensets. - TypedDict supports two additional equivalent forms:: + TypedDict supports an additional equivalent form:: - Point2D = TypedDict('Point2D', x=int, y=int, label=str) Point2D = TypedDict('Point2D', {'x': int, 'y': int, 'label': str}) By default, all keys must be present in a TypedDict. It is possible @@ -2587,14 +2586,22 @@ class point2D(TypedDict, total=False): the total argument. True is the default, and makes all items defined in the class body be required. - The class syntax is only supported in Python 3.6+, while two other - syntax forms work for Python 2.7 and 3.2+ + The class syntax is only supported in Python 3.6+, while the other + syntax form works for Python 2.7 and 3.2+ """ if fields is None: fields = kwargs elif kwargs: raise TypeError("TypedDict takes either a dict or keyword arguments," " but not both") + if kwargs: + warnings.warn( + "The kwargs-based syntax for TypedDict definitions is deprecated " + "in Python 3.11, will be removed in Python 3.13, and may not be " + "understood by third-party type checkers.", + DeprecationWarning, + stacklevel=2, + ) ns = {'__annotations__': dict(fields)} module = _caller() diff --git a/Misc/ACKS b/Misc/ACKS index 64bd91d7d6905..244d7579b25eb 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1974,6 +1974,7 @@ Arnon Yaari Alakshendra Yadav Hirokazu Yamamoto Masayuki Yamamoto +Jingchen Ye Ka-Ping Yee Chi Hsuan Yen Jason Yeo diff --git a/Misc/NEWS.d/next/Library/2022-02-08-16-42-20.bpo-46066.m32Hl0.rst b/Misc/NEWS.d/next/Library/2022-02-08-16-42-20.bpo-46066.m32Hl0.rst new file mode 100644 index 0000000000000..d13d9421e748b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-08-16-42-20.bpo-46066.m32Hl0.rst @@ -0,0 +1,3 @@ +Deprecate kwargs-based syntax for :class:`typing.TypedDict` definitions. +It had confusing semantics when specifying totality, and was largely unused. +Patch by Jingchen Ye. From webhook-mailer at python.org Wed Feb 16 22:28:30 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Thu, 17 Feb 2022 03:28:30 -0000 Subject: [Python-checkins] bpo-46333: Honor `module` parameter in ForwardRef (GH-30536) Message-ID: https://github.com/python/cpython/commit/6e7b813195f9bd6a2a15c1f00ef2c0180f6c751a commit: 6e7b813195f9bd6a2a15c1f00ef2c0180f6c751a branch: main author: aha79 <34090357+aha79 at users.noreply.github.com> committer: JelleZijlstra date: 2022-02-16T19:28:18-08:00 summary: bpo-46333: Honor `module` parameter in ForwardRef (GH-30536) The `module` parameter carries semantic information about the forward ref. Forward refs are different if they refer to different module even if they have the same name. This affects the `__eq__`, `__repr__` and `__hash__` methods. Co-authored-by: Andreas Hangauer Co-authored-by: Alex Waygood Co-authored-by: Ken Jin <28750310+Fidget-Spinner at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst M Lib/test/test_typing.py M Lib/typing.py M Misc/ACKS diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 9548a0cab1d1e..1b3eb0667668f 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -2800,6 +2800,10 @@ def test_forward_equality(self): fr = typing.ForwardRef('int') self.assertEqual(fr, typing.ForwardRef('int')) self.assertNotEqual(List['int'], List[int]) + self.assertNotEqual(fr, typing.ForwardRef('int', module=__name__)) + frm = typing.ForwardRef('int', module=__name__) + self.assertEqual(frm, typing.ForwardRef('int', module=__name__)) + self.assertNotEqual(frm, typing.ForwardRef('int', module='__other_name__')) def test_forward_equality_gth(self): c1 = typing.ForwardRef('C') @@ -2836,6 +2840,14 @@ def foo(a: c1_gth, b: c2_gth): self.assertEqual(hash(c1_gth), hash(c2_gth)) self.assertEqual(hash(c1), hash(c1_gth)) + c3 = typing.ForwardRef('int', module=__name__) + c4 = typing.ForwardRef('int', module='__other_name__') + + self.assertNotEqual(hash(c3), hash(c1)) + self.assertNotEqual(hash(c3), hash(c1_gth)) + self.assertNotEqual(hash(c3), hash(c4)) + self.assertEqual(hash(c3), hash(typing.ForwardRef('int', module=__name__))) + def test_forward_equality_namespace(self): class A: pass diff --git a/Lib/typing.py b/Lib/typing.py index 3233827d816d1..8f923fa20f534 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -767,10 +767,11 @@ def __eq__(self, other): if self.__forward_evaluated__ and other.__forward_evaluated__: return (self.__forward_arg__ == other.__forward_arg__ and self.__forward_value__ == other.__forward_value__) - return self.__forward_arg__ == other.__forward_arg__ + return (self.__forward_arg__ == other.__forward_arg__ and + self.__forward_module__ == other.__forward_module__) def __hash__(self): - return hash(self.__forward_arg__) + return hash((self.__forward_arg__, self.__forward_module__)) def __or__(self, other): return Union[self, other] diff --git a/Misc/ACKS b/Misc/ACKS index 244d7579b25eb..dceb2b628eb20 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -680,6 +680,7 @@ Anders Hammarquist Mark Hammond Harald Hanche-Olsen Manus Hand +Andreas Hangauer Milton L. Hankins Carl Bordum Hansen Stephen Hansen diff --git a/Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst b/Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst new file mode 100644 index 0000000000000..ec3c6d54ee499 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst @@ -0,0 +1,4 @@ +The :meth:`__eq__` and :meth:`__hash__` methods of +:class:`typing.ForwardRef` now honor the ``module`` parameter of +:class:`typing.ForwardRef`. Forward references from different +modules are now differentiated. From webhook-mailer at python.org Wed Feb 16 22:53:12 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 17 Feb 2022 03:53:12 -0000 Subject: [Python-checkins] [3.10] bpo-46333: Honor `module` parameter in ForwardRef (GH-30536) (GH-31379) Message-ID: https://github.com/python/cpython/commit/a17d59a6df146077b5420b36c35a1b82ab3f071a commit: a17d59a6df146077b5420b36c35a1b82ab3f071a branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-16T19:53:03-08:00 summary: [3.10] bpo-46333: Honor `module` parameter in ForwardRef (GH-30536) (GH-31379) The `module` parameter carries semantic information about the forward ref. Forward refs are different if they refer to different module even if they have the same name. This affects the `__eq__`, `__repr__` and `__hash__` methods. Co-authored-by: Andreas Hangauer Co-authored-by: Alex Waygood Co-authored-by: Ken Jin <28750310+Fidget-Spinner at users.noreply.github.com> (cherry picked from commit 6e7b813195f9bd6a2a15c1f00ef2c0180f6c751a) Co-authored-by: aha79 <34090357+aha79 at users.noreply.github.com> Automerge-Triggered-By: GH:JelleZijlstra files: A Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst M Lib/test/test_typing.py M Lib/typing.py M Misc/ACKS diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 2842e28c5ab59..c4c06008d91fa 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -2658,6 +2658,10 @@ def test_forward_equality(self): fr = typing.ForwardRef('int') self.assertEqual(fr, typing.ForwardRef('int')) self.assertNotEqual(List['int'], List[int]) + self.assertNotEqual(fr, typing.ForwardRef('int', module=__name__)) + frm = typing.ForwardRef('int', module=__name__) + self.assertEqual(frm, typing.ForwardRef('int', module=__name__)) + self.assertNotEqual(frm, typing.ForwardRef('int', module='__other_name__')) def test_forward_equality_gth(self): c1 = typing.ForwardRef('C') @@ -2694,6 +2698,14 @@ def foo(a: c1_gth, b: c2_gth): self.assertEqual(hash(c1_gth), hash(c2_gth)) self.assertEqual(hash(c1), hash(c1_gth)) + c3 = typing.ForwardRef('int', module=__name__) + c4 = typing.ForwardRef('int', module='__other_name__') + + self.assertNotEqual(hash(c3), hash(c1)) + self.assertNotEqual(hash(c3), hash(c1_gth)) + self.assertNotEqual(hash(c3), hash(c4)) + self.assertEqual(hash(c3), hash(typing.ForwardRef('int', module=__name__))) + def test_forward_equality_namespace(self): class A: pass diff --git a/Lib/typing.py b/Lib/typing.py index 135ca582177b9..5d900dd6c33ab 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -705,10 +705,11 @@ def __eq__(self, other): if self.__forward_evaluated__ and other.__forward_evaluated__: return (self.__forward_arg__ == other.__forward_arg__ and self.__forward_value__ == other.__forward_value__) - return self.__forward_arg__ == other.__forward_arg__ + return (self.__forward_arg__ == other.__forward_arg__ and + self.__forward_module__ == other.__forward_module__) def __hash__(self): - return hash(self.__forward_arg__) + return hash((self.__forward_arg__, self.__forward_module__)) def __repr__(self): return f'ForwardRef({self.__forward_arg__!r})' diff --git a/Misc/ACKS b/Misc/ACKS index 7f9166cd74cfa..4bd05b59a9982 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -677,6 +677,7 @@ Anders Hammarquist Mark Hammond Harald Hanche-Olsen Manus Hand +Andreas Hangauer Milton L. Hankins Carl Bordum Hansen Stephen Hansen diff --git a/Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst b/Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst new file mode 100644 index 0000000000000..ec3c6d54ee499 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst @@ -0,0 +1,4 @@ +The :meth:`__eq__` and :meth:`__hash__` methods of +:class:`typing.ForwardRef` now honor the ``module`` parameter of +:class:`typing.ForwardRef`. Forward references from different +modules are now differentiated. From webhook-mailer at python.org Wed Feb 16 22:53:35 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 17 Feb 2022 03:53:35 -0000 Subject: [Python-checkins] bpo-46333: Honor `module` parameter in ForwardRef (GH-30536) Message-ID: https://github.com/python/cpython/commit/29ae7d3996e073ec3f15c4e7486b17c1d0e3c8f5 commit: 29ae7d3996e073ec3f15c4e7486b17c1d0e3c8f5 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-16T19:53:31-08:00 summary: bpo-46333: Honor `module` parameter in ForwardRef (GH-30536) The `module` parameter carries semantic information about the forward ref. Forward refs are different if they refer to different module even if they have the same name. This affects the `__eq__`, `__repr__` and `__hash__` methods. Co-authored-by: Andreas Hangauer Co-authored-by: Alex Waygood Co-authored-by: Ken Jin <28750310+Fidget-Spinner at users.noreply.github.com> (cherry picked from commit 6e7b813195f9bd6a2a15c1f00ef2c0180f6c751a) Co-authored-by: aha79 <34090357+aha79 at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst M Lib/test/test_typing.py M Lib/typing.py M Misc/ACKS diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 641527329501f..08eddb095d561 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -2463,6 +2463,10 @@ def test_forward_equality(self): fr = typing.ForwardRef('int') self.assertEqual(fr, typing.ForwardRef('int')) self.assertNotEqual(List['int'], List[int]) + self.assertNotEqual(fr, typing.ForwardRef('int', module=__name__)) + frm = typing.ForwardRef('int', module=__name__) + self.assertEqual(frm, typing.ForwardRef('int', module=__name__)) + self.assertNotEqual(frm, typing.ForwardRef('int', module='__other_name__')) def test_forward_equality_gth(self): c1 = typing.ForwardRef('C') @@ -2499,6 +2503,14 @@ def foo(a: c1_gth, b: c2_gth): self.assertEqual(hash(c1_gth), hash(c2_gth)) self.assertEqual(hash(c1), hash(c1_gth)) + c3 = typing.ForwardRef('int', module=__name__) + c4 = typing.ForwardRef('int', module='__other_name__') + + self.assertNotEqual(hash(c3), hash(c1)) + self.assertNotEqual(hash(c3), hash(c1_gth)) + self.assertNotEqual(hash(c3), hash(c4)) + self.assertEqual(hash(c3), hash(typing.ForwardRef('int', module=__name__))) + def test_forward_equality_namespace(self): class A: pass diff --git a/Lib/typing.py b/Lib/typing.py index 18b21e96435b3..af2d8c758a454 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -568,10 +568,11 @@ def __eq__(self, other): if self.__forward_evaluated__ and other.__forward_evaluated__: return (self.__forward_arg__ == other.__forward_arg__ and self.__forward_value__ == other.__forward_value__) - return self.__forward_arg__ == other.__forward_arg__ + return (self.__forward_arg__ == other.__forward_arg__ and + self.__forward_module__ == other.__forward_module__) def __hash__(self): - return hash(self.__forward_arg__) + return hash((self.__forward_arg__, self.__forward_module__)) def __repr__(self): return f'ForwardRef({self.__forward_arg__!r})' diff --git a/Misc/ACKS b/Misc/ACKS index 61267d2a23bea..fe7e692e04967 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -665,6 +665,7 @@ Anders Hammarquist Mark Hammond Harald Hanche-Olsen Manus Hand +Andreas Hangauer Milton L. Hankins Carl Bordum Hansen Stephen Hansen diff --git a/Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst b/Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst new file mode 100644 index 0000000000000..ec3c6d54ee499 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-11-15-54-15.bpo-46333.B1faiF.rst @@ -0,0 +1,4 @@ +The :meth:`__eq__` and :meth:`__hash__` methods of +:class:`typing.ForwardRef` now honor the ``module`` parameter of +:class:`typing.ForwardRef`. Forward references from different +modules are now differentiated. From webhook-mailer at python.org Wed Feb 16 23:02:27 2022 From: webhook-mailer at python.org (corona10) Date: Thu, 17 Feb 2022 04:02:27 -0000 Subject: [Python-checkins] bpo-46541: Remove usage of _Py_IDENTIFIER from array module (GH-31376) Message-ID: https://github.com/python/cpython/commit/8cb5f707a841832aeac4b01c8b6a0e72dced52ae commit: 8cb5f707a841832aeac4b01c8b6a0e72dced52ae branch: main author: Dong-hee Na committer: corona10 date: 2022-02-17T13:02:17+09:00 summary: bpo-46541: Remove usage of _Py_IDENTIFIER from array module (GH-31376) files: M Modules/arraymodule.c M Modules/clinic/arraymodule.c.h diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c index e516f54ab61e6..2d6da1aaac85a 100644 --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -6,7 +6,6 @@ #ifndef Py_BUILD_CORE_BUILTIN # define Py_BUILD_CORE_MODULE 1 #endif -#define NEEDS_PY_IDENTIFIER #define PY_SSIZE_T_CLEAN #include "Python.h" @@ -58,6 +57,12 @@ typedef struct { typedef struct { PyTypeObject *ArrayType; PyTypeObject *ArrayIterType; + + PyObject *str_read; + PyObject *str_write; + PyObject *str__array_reconstructor; + PyObject *str___dict__; + PyObject *str_iter; } array_state; static array_state * @@ -1464,6 +1469,7 @@ array_array_reverse_impl(arrayobject *self) /*[clinic input] array.array.fromfile + cls: defining_class f: object n: Py_ssize_t / @@ -1472,13 +1478,13 @@ Read n objects from the file object f and append them to the end of the array. [clinic start generated code]*/ static PyObject * -array_array_fromfile_impl(arrayobject *self, PyObject *f, Py_ssize_t n) -/*[clinic end generated code: output=ec9f600e10f53510 input=e188afe8e58adf40]*/ +array_array_fromfile_impl(arrayobject *self, PyTypeObject *cls, PyObject *f, + Py_ssize_t n) +/*[clinic end generated code: output=83a667080b345ebc input=3822e907c1c11f1a]*/ { PyObject *b, *res; Py_ssize_t itemsize = self->ob_descr->itemsize; Py_ssize_t nbytes; - _Py_IDENTIFIER(read); int not_enough_bytes; if (n < 0) { @@ -1489,9 +1495,14 @@ array_array_fromfile_impl(arrayobject *self, PyObject *f, Py_ssize_t n) PyErr_NoMemory(); return NULL; } + + + array_state *state = get_array_state_by_class(cls); + assert(state != NULL); + nbytes = n * itemsize; - b = _PyObject_CallMethodId(f, &PyId_read, "n", nbytes); + b = _PyObject_CallMethod(f, state->str_read, "n", nbytes); if (b == NULL) return NULL; @@ -1522,6 +1533,7 @@ array_array_fromfile_impl(arrayobject *self, PyObject *f, Py_ssize_t n) /*[clinic input] array.array.tofile + cls: defining_class f: object / @@ -1529,8 +1541,8 @@ Write all items (as machine values) to the file object f. [clinic start generated code]*/ static PyObject * -array_array_tofile(arrayobject *self, PyObject *f) -/*[clinic end generated code: output=3a2cfa8128df0777 input=b0669a484aab0831]*/ +array_array_tofile_impl(arrayobject *self, PyTypeObject *cls, PyObject *f) +/*[clinic end generated code: output=4560c628d9c18bc2 input=5a24da7a7b407b52]*/ { Py_ssize_t nbytes = Py_SIZE(self) * self->ob_descr->itemsize; /* Write 64K blocks at a time */ @@ -1542,18 +1554,21 @@ array_array_tofile(arrayobject *self, PyObject *f) if (Py_SIZE(self) == 0) goto done; + + array_state *state = get_array_state_by_class(cls); + assert(state != NULL); + for (i = 0; i < nblocks; i++) { char* ptr = self->ob_item + i*BLOCKSIZE; Py_ssize_t size = BLOCKSIZE; PyObject *bytes, *res; - _Py_IDENTIFIER(write); if (i*BLOCKSIZE + size > nbytes) size = nbytes - i*BLOCKSIZE; bytes = PyBytes_FromStringAndSize(ptr, size); if (bytes == NULL) return NULL; - res = _PyObject_CallMethodIdOneArg(f, &PyId_write, bytes); + res = PyObject_CallMethodOneArg(f, state->str_write, bytes); Py_DECREF(bytes); if (res == NULL) return NULL; @@ -2176,6 +2191,7 @@ array__array_reconstructor_impl(PyObject *module, PyTypeObject *arraytype, /*[clinic input] array.array.__reduce_ex__ + cls: defining_class value: object / @@ -2183,8 +2199,9 @@ Return state information for pickling. [clinic start generated code]*/ static PyObject * -array_array___reduce_ex__(arrayobject *self, PyObject *value) -/*[clinic end generated code: output=051e0a6175d0eddb input=c36c3f85de7df6cd]*/ +array_array___reduce_ex___impl(arrayobject *self, PyTypeObject *cls, + PyObject *value) +/*[clinic end generated code: output=4958ee5d79452ad5 input=19968cf0f91d3eea]*/ { PyObject *dict; PyObject *result; @@ -2193,16 +2210,17 @@ array_array___reduce_ex__(arrayobject *self, PyObject *value) int mformat_code; static PyObject *array_reconstructor = NULL; long protocol; - _Py_IDENTIFIER(_array_reconstructor); - _Py_IDENTIFIER(__dict__); + + array_state *state = get_array_state_by_class(cls); + assert(state != NULL); if (array_reconstructor == NULL) { PyObject *array_module = PyImport_ImportModule("array"); if (array_module == NULL) return NULL; - array_reconstructor = _PyObject_GetAttrId( + array_reconstructor = PyObject_GetAttr( array_module, - &PyId__array_reconstructor); + state->str__array_reconstructor); Py_DECREF(array_module); if (array_reconstructor == NULL) return NULL; @@ -2217,7 +2235,7 @@ array_array___reduce_ex__(arrayobject *self, PyObject *value) if (protocol == -1 && PyErr_Occurred()) return NULL; - if (_PyObject_LookupAttrId((PyObject *)self, &PyId___dict__, &dict) < 0) { + if (_PyObject_LookupAttr((PyObject *)self, state->str___dict__, &dict) < 0) { return NULL; } if (dict == NULL) { @@ -2939,15 +2957,20 @@ arrayiter_traverse(arrayiterobject *it, visitproc visit, void *arg) /*[clinic input] array.arrayiterator.__reduce__ + cls: defining_class + / + Return state information for pickling. [clinic start generated code]*/ static PyObject * -array_arrayiterator___reduce___impl(arrayiterobject *self) -/*[clinic end generated code: output=7898a52e8e66e016 input=a062ea1e9951417a]*/ +array_arrayiterator___reduce___impl(arrayiterobject *self, PyTypeObject *cls) +/*[clinic end generated code: output=4b032417a2c8f5e6 input=ac64e65a87ad452e]*/ { - _Py_IDENTIFIER(iter); - PyObject *func = _PyEval_GetBuiltinId(&PyId_iter); + + array_state *state = get_array_state_by_class(cls); + assert(state != NULL); + PyObject *func = _PyEval_GetBuiltin(state->str_iter); if (self->ao == NULL) { return Py_BuildValue("N(())", func); } @@ -3011,6 +3034,11 @@ array_traverse(PyObject *module, visitproc visit, void *arg) array_state *state = get_array_state(module); Py_VISIT(state->ArrayType); Py_VISIT(state->ArrayIterType); + Py_VISIT(state->str_read); + Py_VISIT(state->str_write); + Py_VISIT(state->str__array_reconstructor); + Py_VISIT(state->str___dict__); + Py_VISIT(state->str_iter); return 0; } @@ -3020,6 +3048,11 @@ array_clear(PyObject *module) array_state *state = get_array_state(module); Py_CLEAR(state->ArrayType); Py_CLEAR(state->ArrayIterType); + Py_CLEAR(state->str_read); + Py_CLEAR(state->str_write); + Py_CLEAR(state->str__array_reconstructor); + Py_CLEAR(state->str___dict__); + Py_CLEAR(state->str_iter); return 0; } @@ -3043,6 +3076,15 @@ do { \ } \ } while (0) +#define ADD_INTERNED(state, string) \ +do { \ + PyObject *tmp = PyUnicode_InternFromString(#string); \ + if (tmp == NULL) { \ + return -1; \ + } \ + state->str_ ## string = tmp; \ +} while (0) + static int array_modexec(PyObject *m) { @@ -3051,6 +3093,13 @@ array_modexec(PyObject *m) PyObject *typecodes; const struct arraydescr *descr; + /* Add interned strings */ + ADD_INTERNED(state, read); + ADD_INTERNED(state, write); + ADD_INTERNED(state, _array_reconstructor); + ADD_INTERNED(state, __dict__); + ADD_INTERNED(state, iter); + CREATE_TYPE(m, state->ArrayType, &array_spec); CREATE_TYPE(m, state->ArrayIterType, &arrayiter_spec); Py_SET_TYPE(state->ArrayIterType, &PyType_Type); diff --git a/Modules/clinic/arraymodule.c.h b/Modules/clinic/arraymodule.c.h index c46cc738de91b..583ff28bf3287 100644 --- a/Modules/clinic/arraymodule.c.h +++ b/Modules/clinic/arraymodule.c.h @@ -285,35 +285,26 @@ PyDoc_STRVAR(array_array_fromfile__doc__, "Read n objects from the file object f and append them to the end of the array."); #define ARRAY_ARRAY_FROMFILE_METHODDEF \ - {"fromfile", (PyCFunction)(void(*)(void))array_array_fromfile, METH_FASTCALL, array_array_fromfile__doc__}, + {"fromfile", (PyCFunction)(void(*)(void))array_array_fromfile, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, array_array_fromfile__doc__}, static PyObject * -array_array_fromfile_impl(arrayobject *self, PyObject *f, Py_ssize_t n); +array_array_fromfile_impl(arrayobject *self, PyTypeObject *cls, PyObject *f, + Py_ssize_t n); static PyObject * -array_array_fromfile(arrayobject *self, PyObject *const *args, Py_ssize_t nargs) +array_array_fromfile(arrayobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", "", NULL}; + static _PyArg_Parser _parser = {"On:fromfile", _keywords, 0}; PyObject *f; Py_ssize_t n; - if (!_PyArg_CheckPositional("fromfile", nargs, 2, 2)) { + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &f, &n)) { goto exit; } - f = args[0]; - { - Py_ssize_t ival = -1; - PyObject *iobj = _PyNumber_Index(args[1]); - if (iobj != NULL) { - ival = PyLong_AsSsize_t(iobj); - Py_DECREF(iobj); - } - if (ival == -1 && PyErr_Occurred()) { - goto exit; - } - n = ival; - } - return_value = array_array_fromfile_impl(self, f, n); + return_value = array_array_fromfile_impl(self, cls, f, n); exit: return return_value; @@ -326,7 +317,28 @@ PyDoc_STRVAR(array_array_tofile__doc__, "Write all items (as machine values) to the file object f."); #define ARRAY_ARRAY_TOFILE_METHODDEF \ - {"tofile", (PyCFunction)array_array_tofile, METH_O, array_array_tofile__doc__}, + {"tofile", (PyCFunction)(void(*)(void))array_array_tofile, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, array_array_tofile__doc__}, + +static PyObject * +array_array_tofile_impl(arrayobject *self, PyTypeObject *cls, PyObject *f); + +static PyObject * +array_array_tofile(arrayobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"", NULL}; + static _PyArg_Parser _parser = {"O:tofile", _keywords, 0}; + PyObject *f; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &f)) { + goto exit; + } + return_value = array_array_tofile_impl(self, cls, f); + +exit: + return return_value; +} PyDoc_STRVAR(array_array_fromlist__doc__, "fromlist($self, list, /)\n" @@ -544,7 +556,29 @@ PyDoc_STRVAR(array_array___reduce_ex____doc__, "Return state information for pickling."); #define ARRAY_ARRAY___REDUCE_EX___METHODDEF \ - {"__reduce_ex__", (PyCFunction)array_array___reduce_ex__, METH_O, array_array___reduce_ex____doc__}, + {"__reduce_ex__", (PyCFunction)(void(*)(void))array_array___reduce_ex__, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, array_array___reduce_ex____doc__}, + +static PyObject * +array_array___reduce_ex___impl(arrayobject *self, PyTypeObject *cls, + PyObject *value); + +static PyObject * +array_array___reduce_ex__(arrayobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"", NULL}; + static _PyArg_Parser _parser = {"O:__reduce_ex__", _keywords, 0}; + PyObject *value; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &value)) { + goto exit; + } + return_value = array_array___reduce_ex___impl(self, cls, value); + +exit: + return return_value; +} PyDoc_STRVAR(array_arrayiterator___reduce____doc__, "__reduce__($self, /)\n" @@ -553,15 +587,26 @@ PyDoc_STRVAR(array_arrayiterator___reduce____doc__, "Return state information for pickling."); #define ARRAY_ARRAYITERATOR___REDUCE___METHODDEF \ - {"__reduce__", (PyCFunction)array_arrayiterator___reduce__, METH_NOARGS, array_arrayiterator___reduce____doc__}, + {"__reduce__", (PyCFunction)(void(*)(void))array_arrayiterator___reduce__, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, array_arrayiterator___reduce____doc__}, static PyObject * -array_arrayiterator___reduce___impl(arrayiterobject *self); +array_arrayiterator___reduce___impl(arrayiterobject *self, PyTypeObject *cls); static PyObject * -array_arrayiterator___reduce__(arrayiterobject *self, PyObject *Py_UNUSED(ignored)) +array_arrayiterator___reduce__(arrayiterobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return array_arrayiterator___reduce___impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":__reduce__", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = array_arrayiterator___reduce___impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(array_arrayiterator___setstate____doc__, @@ -572,4 +617,4 @@ PyDoc_STRVAR(array_arrayiterator___setstate____doc__, #define ARRAY_ARRAYITERATOR___SETSTATE___METHODDEF \ {"__setstate__", (PyCFunction)array_arrayiterator___setstate__, METH_O, array_arrayiterator___setstate____doc__}, -/*[clinic end generated code: output=f130a994f98f1227 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=1db6decd8492bf91 input=a9049054013a1b77]*/ From webhook-mailer at python.org Wed Feb 16 23:06:33 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 17 Feb 2022 04:06:33 -0000 Subject: [Python-checkins] [3.9] Become a CODEOWNER for typing (GH-31374) (GH-31381) Message-ID: https://github.com/python/cpython/commit/d975a40e33e52cb479defcd2aa22be83f336d81f commit: d975a40e33e52cb479defcd2aa22be83f336d81f branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-16T20:06:25-08:00 summary: [3.9] Become a CODEOWNER for typing (GH-31374) (GH-31381) (cherry picked from commit 6f1efd19a70839d480e4b1fcd9fecd3a8725824b) Co-authored-by: Jelle Zijlstra Automerge-Triggered-By: GH:JelleZijlstra files: M .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 281c9a9979ed0..7fe615a1e869e 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -111,7 +111,7 @@ Include/pytime.h @pganssle @abalkin **/*idlelib* @terryjreedy -**/*typing* @gvanrossum @Fidget-Spinner +**/*typing* @gvanrossum @Fidget-Spinner @JelleZijlstra **/*asyncore @giampaolo **/*asynchat @giampaolo From webhook-mailer at python.org Wed Feb 16 23:09:14 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 17 Feb 2022 04:09:14 -0000 Subject: [Python-checkins] [3.10] Become a CODEOWNER for typing (GH-31374) (GH-31382) Message-ID: https://github.com/python/cpython/commit/a481be0c5f5089ed5e81b9855f71eae12039cad2 commit: a481be0c5f5089ed5e81b9855f71eae12039cad2 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-16T20:09:06-08:00 summary: [3.10] Become a CODEOWNER for typing (GH-31374) (GH-31382) (cherry picked from commit 6f1efd19a70839d480e4b1fcd9fecd3a8725824b) Co-authored-by: Jelle Zijlstra Automerge-Triggered-By: GH:JelleZijlstra files: M .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 8412314680df6..9a5bc63924788 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -128,7 +128,7 @@ Lib/ast.py @isidentical **/*idlelib* @terryjreedy -**/*typing* @gvanrossum @Fidget-Spinner +**/*typing* @gvanrossum @Fidget-Spinner @JelleZijlstra **/*asyncore @giampaolo **/*asynchat @giampaolo From webhook-mailer at python.org Thu Feb 17 00:20:14 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Thu, 17 Feb 2022 05:20:14 -0000 Subject: [Python-checkins] bpo-46745: Fix typo in PositionsIterator (#31322) Message-ID: https://github.com/python/cpython/commit/482a4b6c3f8ef60056e85647a1d84af2e6dfd3ef commit: 482a4b6c3f8ef60056e85647a1d84af2e6dfd3ef branch: main author: Robert-Andr? Mauchin <30413512+eclipseo at users.noreply.github.com> committer: JelleZijlstra date: 2022-02-16T21:20:06-08:00 summary: bpo-46745: Fix typo in PositionsIterator (#31322) Co-authored-by: Jelle Zijlstra files: M Objects/codeobject.c diff --git a/Objects/codeobject.c b/Objects/codeobject.c index efb51464bd87a..9a27f1295b189 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -1020,7 +1020,7 @@ positionsiter_next(positionsiterator* pi) static PyTypeObject PositionsIterator = { PyVarObject_HEAD_INIT(&PyType_Type, 0) - "poisitions_iterator", /* tp_name */ + "positions_iterator", /* tp_name */ sizeof(positionsiterator), /* tp_basicsize */ 0, /* tp_itemsize */ /* methods */ From webhook-mailer at python.org Thu Feb 17 03:20:54 2022 From: webhook-mailer at python.org (ambv) Date: Thu, 17 Feb 2022 08:20:54 -0000 Subject: [Python-checkins] Close stale PRs without signed CLA (GH-30500) Message-ID: https://github.com/python/cpython/commit/5bc4327e2c55c33342a6850c4741e2b63ff96aec commit: 5bc4327e2c55c33342a6850c4741e2b63ff96aec branch: main author: Erlend Egeberg Aasland committer: ambv date: 2022-02-17T09:20:40+01:00 summary: Close stale PRs without signed CLA (GH-30500) Co-authored-by: Hugo van Kemenade files: M .github/workflows/stale.yml diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml index 1532af26f59ff..fc0c15dc0a600 100644 --- a/.github/workflows/stale.yml +++ b/.github/workflows/stale.yml @@ -14,10 +14,23 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/stale at v4 + - name: "Check PRs with 'CLA signed' label" + uses: actions/stale at v4 with: repo-token: ${{ secrets.GITHUB_TOKEN }} + only-pr-labels: 'CLA signed' stale-pr-message: 'This PR is stale because it has been open for 30 days with no activity.' stale-pr-label: 'stale' days-before-stale: 30 days-before-close: -1 + + - name: "Check PRs with 'CLA not signed' label" + uses: actions/stale at v4 + with: + repo-token: ${{ secrets.GITHUB_TOKEN }} + only-pr-labels: 'CLA not signed' + stale-pr-message: 'This PR is stale because it has been open for 30 days with no activity. If the CLA is not signed within 14 days, it will be closed. See also https://devguide.python.org/pullrequest/#licensing' + stale-pr-label: 'stale' + close-pr-message: 'Closing this stale PR because the CLA is still not signed.' + days-before-stale: 30 + days-before-close: 14 From webhook-mailer at python.org Thu Feb 17 03:32:13 2022 From: webhook-mailer at python.org (ambv) Date: Thu, 17 Feb 2022 08:32:13 -0000 Subject: [Python-checkins] bpo-44949: Fix test_readline auto history tests (GH-27813) (GH-31118) Message-ID: https://github.com/python/cpython/commit/b1bc04df817ecb3fb9fdbefa3782efea54488e6f commit: b1bc04df817ecb3fb9fdbefa3782efea54488e6f branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2022-02-17T09:32:03+01:00 summary: bpo-44949: Fix test_readline auto history tests (GH-27813) (GH-31118) (cherry picked from commit 6fb62b42f4db56ed5efe0ca4c1059049276c1083) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Tests/2021-08-18-18-30-12.bpo-44949.VE5ENv.rst M Lib/test/test_readline.py diff --git a/Lib/test/test_readline.py b/Lib/test/test_readline.py index 3574657cf3635..6bde9e3c59806 100644 --- a/Lib/test/test_readline.py +++ b/Lib/test/test_readline.py @@ -147,11 +147,15 @@ def test_init(self): def test_auto_history_enabled(self): output = run_pty(self.auto_history_script.format(True)) - self.assertIn(b"History length: 1\r\n", output) + # bpo-44949: Sometimes, the newline character is not written at the + # end, so don't expect it in the output. + self.assertIn(b"History length: 1", output) def test_auto_history_disabled(self): output = run_pty(self.auto_history_script.format(False)) - self.assertIn(b"History length: 0\r\n", output) + # bpo-44949: Sometimes, the newline character is not written at the + # end, so don't expect it in the output. + self.assertIn(b"History length: 0", output) def test_nonascii(self): loc = locale.setlocale(locale.LC_CTYPE, None) diff --git a/Misc/NEWS.d/next/Tests/2021-08-18-18-30-12.bpo-44949.VE5ENv.rst b/Misc/NEWS.d/next/Tests/2021-08-18-18-30-12.bpo-44949.VE5ENv.rst new file mode 100644 index 0000000000000..7fdf1810b165e --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2021-08-18-18-30-12.bpo-44949.VE5ENv.rst @@ -0,0 +1,2 @@ +Fix auto history tests of test_readline: sometimes, the newline character is +not written at the end, so don't expect it in the output. From webhook-mailer at python.org Thu Feb 17 07:18:14 2022 From: webhook-mailer at python.org (ambv) Date: Thu, 17 Feb 2022 12:18:14 -0000 Subject: [Python-checkins] bpo-41086: Add exception for uninstantiated interpolation (configparser) (GH-21062) Message-ID: https://github.com/python/cpython/commit/fc115c9bde52a58b4fb9be2b80c3d6be8a4d2454 commit: fc115c9bde52a58b4fb9be2b80c3d6be8a4d2454 branch: main author: Brian Faherty committer: ambv date: 2022-02-17T13:17:43+01:00 summary: bpo-41086: Add exception for uninstantiated interpolation (configparser) (GH-21062) * Add exception for uninstantiated interpolation (configparser) The current feedback when users try to pass an uninstantiated interpolation into a ConfigParser is an error message that does not help users solve the problem. This current error of `TypeError: before_set() missing 1 required positional argument: 'value'` does not display until the parser is used, which usually results in the assumption that instantiation of the parser was done correctly. The new exception of InterpolationTypeError, will be raised on the line where the ConfigParser is instantiated. This will result in users see the line that has the error in their backtrace for faster debugging. There have been a number of bugs created in the issue tracker, which could have been addressed by: https://bugs.python.org/issue26831 and https://bugs.python.org/issue26469 * ?? Added by blurb_it. * Replace custom Error with TypeError Per feedback from @iritkatriel, the custom InterpolationTypeError has been dropped in favour of a TypeError with a custom message, and the unittests have been expanded. * More verbose message Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: ?ukasz Langa files: A Misc/NEWS.d/next/Library/2020-06-23-01-50-24.bpo-41086.YnOvpS.rst M Lib/configparser.py M Lib/test/test_configparser.py diff --git a/Lib/configparser.py b/Lib/configparser.py index 3470624e63f61..f5666f518f121 100644 --- a/Lib/configparser.py +++ b/Lib/configparser.py @@ -633,6 +633,11 @@ def __init__(self, defaults=None, dict_type=_default_dict, self._interpolation = self._DEFAULT_INTERPOLATION if self._interpolation is None: self._interpolation = Interpolation() + if not isinstance(self._interpolation, Interpolation): + raise TypeError( + f"interpolation= must be None or an instance of Interpolation;" + f" got an object of type {type(self._interpolation)}" + ) if converters is not _UNSET: self._converters.update(converters) if defaults: diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py index e9b03e6c62ef1..569959c3ed919 100644 --- a/Lib/test/test_configparser.py +++ b/Lib/test/test_configparser.py @@ -1048,6 +1048,14 @@ def test_set_malformatted_interpolation(self): self.assertEqual(cf.get("sect", "option2"), "foo%%bar") +class ConfigParserTestCaseInvalidInterpolationType(unittest.TestCase): + def test_error_on_wrong_type_for_interpolation(self): + for value in [configparser.ExtendedInterpolation, 42, "a string"]: + with self.subTest(value=value): + with self.assertRaises(TypeError): + configparser.ConfigParser(interpolation=value) + + class ConfigParserTestCaseNonStandardDelimiters(ConfigParserTestCase): delimiters = (':=', '$') comment_prefixes = ('//', '"') diff --git a/Misc/NEWS.d/next/Library/2020-06-23-01-50-24.bpo-41086.YnOvpS.rst b/Misc/NEWS.d/next/Library/2020-06-23-01-50-24.bpo-41086.YnOvpS.rst new file mode 100644 index 0000000000000..1041c0490fab7 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-06-23-01-50-24.bpo-41086.YnOvpS.rst @@ -0,0 +1 @@ +Make the :class:`configparser.ConfigParser` constructor raise :exc:`TypeError` if the ``interpolation`` parameter is not of type :class:`configparser.Interpolation` From webhook-mailer at python.org Thu Feb 17 07:20:53 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 17 Feb 2022 12:20:53 -0000 Subject: [Python-checkins] Update html.parser.rst (GH-30678) Message-ID: https://github.com/python/cpython/commit/2945f5a7c51200bfc5c493ccb626fc414e1385b1 commit: 2945f5a7c51200bfc5c493ccb626fc414e1385b1 branch: main author: G?ry Ogam committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-17T04:20:40-08:00 summary: Update html.parser.rst (GH-30678) This PR for the documentation of the [html.parser](https://docs.python.org/3/library/html.parser.html) module will - fix a terminology mistake (?start of a tag? -> ?start tag of an element?); - mention the parameter names of the `HTMLParser.handle_*` methods where missing. files: M Doc/library/html.parser.rst diff --git a/Doc/library/html.parser.rst b/Doc/library/html.parser.rst index ac844a683bf7a..03aff25ce6117 100644 --- a/Doc/library/html.parser.rst +++ b/Doc/library/html.parser.rst @@ -126,7 +126,7 @@ implementations do nothing (except for :meth:`~HTMLParser.handle_startendtag`): .. method:: HTMLParser.handle_starttag(tag, attrs) - This method is called to handle the start of a tag (e.g. ``
``). + This method is called to handle the start tag of an element (e.g. ``
``). The *tag* argument is the name of the tag converted to lower case. The *attrs* argument is a list of ``(name, value)`` pairs containing the attributes found From webhook-mailer at python.org Thu Feb 17 07:41:28 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 17 Feb 2022 12:41:28 -0000 Subject: [Python-checkins] [3.10] Update html.parser.rst (GH-30678) (GH-31385) Message-ID: https://github.com/python/cpython/commit/68e80c4e74fdf549146c2b724588038160d79d26 commit: 68e80c4e74fdf549146c2b724588038160d79d26 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-17T04:41:07-08:00 summary: [3.10] Update html.parser.rst (GH-30678) (GH-31385) This PR for the documentation of the [html.parser](https://docs.python.org/3/library/html.parser.html) module will - fix a terminology mistake (?start of a tag? -> ?start tag of an element?); - mention the parameter names of the `HTMLParser.handle_*` methods where missing. (cherry picked from commit 2945f5a7c51200bfc5c493ccb626fc414e1385b1) Co-authored-by: G?ry Ogam Automerge-Triggered-By: GH:merwok files: M Doc/library/html.parser.rst diff --git a/Doc/library/html.parser.rst b/Doc/library/html.parser.rst index ac844a683bf7a..03aff25ce6117 100644 --- a/Doc/library/html.parser.rst +++ b/Doc/library/html.parser.rst @@ -126,7 +126,7 @@ implementations do nothing (except for :meth:`~HTMLParser.handle_startendtag`): .. method:: HTMLParser.handle_starttag(tag, attrs) - This method is called to handle the start of a tag (e.g. ``
``). + This method is called to handle the start tag of an element (e.g. ``
``). The *tag* argument is the name of the tag converted to lower case. The *attrs* argument is a list of ``(name, value)`` pairs containing the attributes found From webhook-mailer at python.org Thu Feb 17 07:46:34 2022 From: webhook-mailer at python.org (miss-islington) Date: Thu, 17 Feb 2022 12:46:34 -0000 Subject: [Python-checkins] Update html.parser.rst (GH-30678) Message-ID: https://github.com/python/cpython/commit/0da140da71830681f216d2ead5225fc685a50d65 commit: 0da140da71830681f216d2ead5225fc685a50d65 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-17T04:46:25-08:00 summary: Update html.parser.rst (GH-30678) This PR for the documentation of the [html.parser](https://docs.python.org/3/library/html.parser.html) module will - fix a terminology mistake (?start of a tag? -> ?start tag of an element?); - mention the parameter names of the `HTMLParser.handle_*` methods where missing. (cherry picked from commit 2945f5a7c51200bfc5c493ccb626fc414e1385b1) Co-authored-by: G?ry Ogam files: M Doc/library/html.parser.rst diff --git a/Doc/library/html.parser.rst b/Doc/library/html.parser.rst index ac844a683bf7a..03aff25ce6117 100644 --- a/Doc/library/html.parser.rst +++ b/Doc/library/html.parser.rst @@ -126,7 +126,7 @@ implementations do nothing (except for :meth:`~HTMLParser.handle_startendtag`): .. method:: HTMLParser.handle_starttag(tag, attrs) - This method is called to handle the start of a tag (e.g. ``
``). + This method is called to handle the start tag of an element (e.g. ``
``). The *tag* argument is the name of the tag converted to lower case. The *attrs* argument is a list of ``(name, value)`` pairs containing the attributes found From webhook-mailer at python.org Thu Feb 17 13:41:10 2022 From: webhook-mailer at python.org (zooba) Date: Thu, 17 Feb 2022 18:41:10 -0000 Subject: [Python-checkins] bpo-46778: Enable multiprocess compilation for source files when building on Windows (GH-31390) Message-ID: https://github.com/python/cpython/commit/98dd0aec2d0cb8971fb5363532c25041a5ba6fdc commit: 98dd0aec2d0cb8971fb5363532c25041a5ba6fdc branch: main author: Jeremy Kloth committer: zooba date: 2022-02-17T18:40:33Z summary: bpo-46778: Enable multiprocess compilation for source files when building on Windows (GH-31390) files: M PCbuild/pyproject.props diff --git a/PCbuild/pyproject.props b/PCbuild/pyproject.props index 3465adec65b9a..8d24393aa1861 100644 --- a/PCbuild/pyproject.props +++ b/PCbuild/pyproject.props @@ -45,6 +45,7 @@ true $(EnableControlFlowGuard) /utf-8 %(AdditionalOptions) + true OnlyExplicitInline From webhook-mailer at python.org Thu Feb 17 13:55:35 2022 From: webhook-mailer at python.org (ambv) Date: Thu, 17 Feb 2022 18:55:35 -0000 Subject: [Python-checkins] bpo-41028: Doc: Move switchers to docsbuild-scripts. (GH-20969) (GH-30344) Message-ID: https://github.com/python/cpython/commit/4c48cac1c44fbbfe3f5fd0c623b7768e2a709da8 commit: 4c48cac1c44fbbfe3f5fd0c623b7768e2a709da8 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2022-02-17T19:55:23+01:00 summary: bpo-41028: Doc: Move switchers to docsbuild-scripts. (GH-20969) (GH-30344) (cherry picked from commit ee2549c2ba8bae00f2b2fea8a39c6dfbd1d06520) Co-authored-by: Julien Palard files: A Misc/NEWS.d/next/Documentation/2020-06-18-23-37-03.bpo-41028.vM8bC8.rst D Doc/tools/static/switchers.js M Doc/Makefile M Doc/tools/templates/dummy.html M Doc/tools/templates/indexsidebar.html M Doc/tools/templates/layout.html diff --git a/Doc/Makefile b/Doc/Makefile index f653d70674eb1..6bf1f408b56f0 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -215,12 +215,12 @@ serve: # for development releases: always build autobuild-dev: - make dist SPHINXOPTS='$(SPHINXOPTS) -Ea -A daily=1 -A switchers=1' + make dist SPHINXOPTS='$(SPHINXOPTS) -Ea -A daily=1' -make suspicious # for quick rebuilds (HTML only) autobuild-dev-html: - make html SPHINXOPTS='$(SPHINXOPTS) -Ea -A daily=1 -A switchers=1' + make html SPHINXOPTS='$(SPHINXOPTS) -Ea -A daily=1' # for stable releases: only build if not in pre-release stage (alpha, beta) # release candidate downloads are okay, since the stable tree can be in that stage diff --git a/Doc/tools/static/switchers.js b/Doc/tools/static/switchers.js deleted file mode 100644 index 1a1c7d0fa57e2..0000000000000 --- a/Doc/tools/static/switchers.js +++ /dev/null @@ -1,156 +0,0 @@ -(function() { - 'use strict'; - - // Parses versions in URL segments like: - // "3", "dev", "release/2.7" or "3.6rc2" - var version_regexs = [ - '(?:\\d)', - '(?:\\d\\.\\d[\\w\\d\\.]*)', - '(?:dev)', - '(?:release/\\d.\\d[\\x\\d\\.]*)']; - - var all_versions = { - '3.10': 'dev (3.10)', - '3.9': 'pre (3.9)', - '3.8': '3.8', - '3.7': '3.7', - '3.6': '3.6', - '2.7': '2.7', - }; - - var all_languages = { - 'en': 'English', - 'fr': 'French', - 'ja': 'Japanese', - 'ko': 'Korean', - 'pt-br': 'Brazilian Portuguese', - 'zh-cn': 'Simplified Chinese', - }; - - function build_version_select(current_version, current_release) { - var buf = [''); - return buf.join(''); - } - - function build_language_select(current_language) { - var buf = [''); - return buf.join(''); - } - - function navigate_to_first_existing(urls) { - // Navigate to the first existing URL in urls. - var url = urls.shift(); - if (urls.length == 0) { - window.location.href = url; - return; - } - $.ajax({ - url: url, - success: function() { - window.location.href = url; - }, - error: function() { - navigate_to_first_existing(urls); - } - }); - } - - function on_version_switch() { - var selected_version = $(this).children('option:selected').attr('value') + '/'; - var url = window.location.href; - var current_language = language_segment_from_url(url); - var current_version = version_segment_in_url(url); - var new_url = url.replace('.org/' + current_language + current_version, - '.org/' + current_language + selected_version); - if (new_url != url) { - navigate_to_first_existing([ - new_url, - url.replace('.org/' + current_language + current_version, - '.org/' + selected_version), - 'https://docs.python.org/' + current_language + selected_version, - 'https://docs.python.org/' + selected_version, - 'https://docs.python.org/' - ]); - } - } - - function on_language_switch() { - var selected_language = $(this).children('option:selected').attr('value') + '/'; - var url = window.location.href; - var current_language = language_segment_from_url(url); - var current_version = version_segment_in_url(url); - if (selected_language == 'en/') // Special 'default' case for english. - selected_language = ''; - var new_url = url.replace('.org/' + current_language + current_version, - '.org/' + selected_language + current_version); - if (new_url != url) { - navigate_to_first_existing([ - new_url, - 'https://docs.python.org/' - ]); - } - } - - // Returns the path segment of the language as a string, like 'fr/' - // or '' if not found. - function language_segment_from_url(url) { - var language_regexp = '\.org/([a-z]{2}(?:-[a-z]{2})?/)'; - var match = url.match(language_regexp); - if (match !== null) - return match[1]; - return ''; - } - - // Returns the path segment of the version as a string, like '3.6/' - // or '' if not found. - function version_segment_in_url(url) { - var language_segment = '(?:[a-z]{2}(?:-[a-z]{2})?/)'; - var version_segment = '(?:(?:' + version_regexs.join('|') + ')/)'; - var version_regexp = '\\.org/' + language_segment + '?(' + version_segment + ')'; - var match = url.match(version_regexp); - if (match !== null) - return match[1]; - return '' - } - - $(document).ready(function() { - var release = DOCUMENTATION_OPTIONS.VERSION; - var language_segment = language_segment_from_url(window.location.href); - var current_language = language_segment.replace(/\/+$/g, '') || 'en'; - var version = release.substr(0, 3); - var version_select = build_version_select(version, release); - - $('.version_switcher_placeholder').html(version_select); - $('.version_switcher_placeholder select').bind('change', on_version_switch); - - var language_select = build_language_select(current_language); - - $('.language_switcher_placeholder').html(language_select); - $('.language_switcher_placeholder select').bind('change', on_language_switch); - }); -})(); diff --git a/Doc/tools/templates/dummy.html b/Doc/tools/templates/dummy.html index 68ae3ad148ec2..3438b44377fcb 100644 --- a/Doc/tools/templates/dummy.html +++ b/Doc/tools/templates/dummy.html @@ -6,3 +6,12 @@ {% trans %}CPython implementation detail:{% endtrans %} {% trans %}Deprecated since version {deprecated}, will be removed in version {removed}{% endtrans %} {% trans %}Deprecated since version {deprecated}, removed in version {removed}{% endtrans %} + + +In docsbuild-scripts, when rewriting indexsidebar.html with actual versions: + +{% trans %}in development{% endtrans %} +{% trans %}pre-release{% endtrans %} +{% trans %}stable{% endtrans %} +{% trans %}security-fixes{% endtrans %} +{% trans %}EOL{% endtrans %} diff --git a/Doc/tools/templates/indexsidebar.html b/Doc/tools/templates/indexsidebar.html index 1c1cb5484a4f6..f7bf6d8e49117 100644 --- a/Doc/tools/templates/indexsidebar.html +++ b/Doc/tools/templates/indexsidebar.html @@ -2,12 +2,8 @@

{% trans %}Download{% endtrans %}

{% trans %}Download these documents{% endtrans %}

{% trans %}Docs by version{% endtrans %}

diff --git a/Doc/tools/templates/layout.html b/Doc/tools/templates/layout.html index 17592d74a4eb5..98ccf4224804b 100644 --- a/Doc/tools/templates/layout.html +++ b/Doc/tools/templates/layout.html @@ -12,22 +12,14 @@ {% block rootrellink %} {{ super() }} -
  • - {%- if switchers is defined %} - {{ language or 'en' }} - {{ release }} - {% trans %}Documentation {% endtrans %}{{ reldelim1 }} - {%- else %} +
  • {{ shorttitle }}{{ reldelim1 }} - {%- endif %}
  • {% endblock %} {% block extrahead %} {% if builder != "htmlhelp" %} - {% if switchers is defined and not embedded %} - {% endif %} {% if pagename == 'whatsnew/changelog' and not embedded %} {% endif %} {% endif %} diff --git a/Misc/NEWS.d/next/Documentation/2020-06-18-23-37-03.bpo-41028.vM8bC8.rst b/Misc/NEWS.d/next/Documentation/2020-06-18-23-37-03.bpo-41028.vM8bC8.rst new file mode 100644 index 0000000000000..5fc4155b55346 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-06-18-23-37-03.bpo-41028.vM8bC8.rst @@ -0,0 +1,2 @@ +Language and version switchers, previously maintained in every cpython +branches, are now handled by docsbuild-script. From webhook-mailer at python.org Thu Feb 17 14:27:54 2022 From: webhook-mailer at python.org (sweeneyde) Date: Thu, 17 Feb 2022 19:27:54 -0000 Subject: [Python-checkins] bpo-46730: Fix refleak and tighten NULL checks (GH-31389) Message-ID: https://github.com/python/cpython/commit/9e06d03672547041239812efe4901c06da6cbd2f commit: 9e06d03672547041239812efe4901c06da6cbd2f branch: main author: Christian Heimes committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2022-02-17T14:27:42-05:00 summary: bpo-46730: Fix refleak and tighten NULL checks (GH-31389) ``PyType_GetQualName`` returns a new reference. Signed-off-by: Christian Heimes files: M Objects/descrobject.c diff --git a/Objects/descrobject.c b/Objects/descrobject.c index 9379ad65de43e..2d4cfb5b7aeb8 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -1585,18 +1585,22 @@ property_descr_get(PyObject *self, PyObject *obj, PyObject *type) propertyobject *gs = (propertyobject *)self; if (gs->prop_get == NULL) { - if (gs->prop_name != NULL) { + PyObject *qualname = PyType_GetQualName(Py_TYPE(obj)); + if (gs->prop_name != NULL && qualname != NULL) { PyErr_Format(PyExc_AttributeError, "property %R of %R object has no getter", gs->prop_name, - PyType_GetQualName(Py_TYPE(obj))); + qualname); } - else { + else if (qualname != NULL) { PyErr_Format(PyExc_AttributeError, "property of %R object has no getter", - PyType_GetQualName(Py_TYPE(obj))); + qualname); + } else { + PyErr_SetString(PyExc_AttributeError, + "property has no getter"); } - + Py_XDECREF(qualname); return NULL; } @@ -1617,20 +1621,24 @@ property_descr_set(PyObject *self, PyObject *obj, PyObject *value) } if (func == NULL) { - if (gs->prop_name != NULL && obj != NULL) { + PyObject *qualname = NULL; + if (obj != NULL) { + qualname = PyType_GetQualName(Py_TYPE(obj)); + } + if (gs->prop_name != NULL && qualname != NULL) { PyErr_Format(PyExc_AttributeError, value == NULL ? "property %R of %R object has no deleter" : "property %R of %R object has no setter", gs->prop_name, - PyType_GetQualName(Py_TYPE(obj))); + qualname); } - else if (obj != NULL) { + else if (qualname != NULL) { PyErr_Format(PyExc_AttributeError, value == NULL ? "property of %R object has no deleter" : "property of %R object has no setter", - PyType_GetQualName(Py_TYPE(obj))); + qualname); } else { PyErr_SetString(PyExc_AttributeError, @@ -1638,6 +1646,7 @@ property_descr_set(PyObject *self, PyObject *obj, PyObject *value) "property has no deleter" : "property has no setter"); } + Py_XDECREF(qualname); return -1; } From webhook-mailer at python.org Thu Feb 17 14:46:15 2022 From: webhook-mailer at python.org (isidentical) Date: Thu, 17 Feb 2022 19:46:15 -0000 Subject: [Python-checkins] bpo-29418: Add inspect.ismethodwrapper to whatsnew (GH-31377) Message-ID: https://github.com/python/cpython/commit/02815d939362d4093a95da650e7fbddabf147eb0 commit: 02815d939362d4093a95da650e7fbddabf147eb0 branch: main author: Hakan ?elik committer: isidentical date: 2022-02-17T22:45:55+03:00 summary: bpo-29418: Add inspect.ismethodwrapper to whatsnew (GH-31377) files: M Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 510ef86d42cc7..4dd778951f4c6 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -240,6 +240,8 @@ inspect triggering dynamic lookup via the descriptor protocol. (Contributed by Weipeng Hong in :issue:`30533`.) +* Add :func:`inspect.ismethodwrapper` for checking if the type of an object is a + :class:`~types.MethodWrapperType`. (Contributed by Hakan ?elik in :issue:`29418`.) math ---- From webhook-mailer at python.org Thu Feb 17 15:45:36 2022 From: webhook-mailer at python.org (terryjreedy) Date: Thu, 17 Feb 2022 20:45:36 -0000 Subject: [Python-checkins] Fix typo in compileall.py's docstring (GH-29865) Message-ID: https://github.com/python/cpython/commit/aa1ccbc76c2794cc78495c84c8325bf1d18a1cc7 commit: aa1ccbc76c2794cc78495c84c8325bf1d18a1cc7 branch: main author: Alexander M?nch committer: terryjreedy date: 2022-02-17T15:45:27-05:00 summary: Fix typo in compileall.py's docstring (GH-29865) files: From webhook-mailer at python.org Thu Feb 17 16:58:13 2022 From: webhook-mailer at python.org (corona10) Date: Thu, 17 Feb 2022 21:58:13 -0000 Subject: [Python-checkins] bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) (GH-31296) Message-ID: https://github.com/python/cpython/commit/cb7551d5663f35c6993f3c6d8e361bc73f1c43d4 commit: cb7551d5663f35c6993f3c6d8e361bc73f1c43d4 branch: 3.10 author: Dong-hee Na committer: corona10 date: 2022-02-18T06:58:02+09:00 summary: bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) (GH-31296) Co-authored-by: Cyril Jouve files: A Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst M Modules/expat/expat.h M Modules/expat/xmlparse.c M Modules/expat/xmlrole.c M Modules/expat/xmltok.c M Modules/expat/xmltok_ns.c diff --git a/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst new file mode 100644 index 0000000000000..9c1f24c0e5171 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst @@ -0,0 +1 @@ +expat: Update libexpat from 2.4.1 to 2.4.4 diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index b7d6d354801b3..4c5704fd9336b 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -11,7 +11,7 @@ Copyright (c) 2000-2005 Fred L. Drake, Jr. Copyright (c) 2001-2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Cristian Rodr?guez Copyright (c) 2016 Thomas Beutlich Copyright (c) 2017 Rhodri James @@ -1041,7 +1041,7 @@ XML_SetBillionLaughsAttackProtectionActivationThreshold( */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 4 -#define XML_MICRO_VERSION 1 +#define XML_MICRO_VERSION 4 #ifdef __cplusplus } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 034a03c30851a..4b43e61321691 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 8539b9040d9d901366a62560a064af7cb99811335784b363abc039c5b0ebc416 (2.4.1+) +/* 2e2c8ce5f11a473d65ec313ab20ceee6afefb355f5405afc06e7204e2e41c8c0 (2.4.4+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -13,7 +13,7 @@ Copyright (c) 2002-2016 Karl Waclawek Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016 Eric Rahm - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Gaurav Copyright (c) 2016 Thomas Beutlich Copyright (c) 2016 Gustavo Grieco @@ -32,6 +32,8 @@ Copyright (c) 2019 David Loffredo Copyright (c) 2019-2020 Ben Wagner Copyright (c) 2019 Vadim Zeitlin + Copyright (c) 2021 Dong-hee Na + Copyright (c) 2022 Samanta Navarro Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -54,6 +56,10 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#define XML_BUILDING_EXPAT 1 + +#include + #if ! defined(_GNU_SOURCE) # define _GNU_SOURCE 1 /* syscall prototype */ #endif @@ -84,14 +90,10 @@ # include #endif -#define XML_BUILDING_EXPAT 1 - #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "ascii.h" #include "expat.h" #include "siphash.h" @@ -973,7 +975,7 @@ parserCreate(const XML_Char *encodingName, if (memsuite) { XML_Memory_Handling_Suite *mtemp; - parser = (XML_Parser)memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); + parser = memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); if (parser != NULL) { mtemp = (XML_Memory_Handling_Suite *)&(parser->m_mem); mtemp->malloc_fcn = memsuite->malloc_fcn; @@ -2066,6 +2068,11 @@ XML_GetBuffer(XML_Parser parser, int len) { keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; + /* Detect and prevent integer overflow */ + if (keep > INT_MAX - neededSize) { + parser->m_errorCode = XML_ERROR_NO_MEMORY; + return NULL; + } neededSize += keep; #endif /* defined XML_CONTEXT_BYTES */ if (neededSize @@ -3260,13 +3267,38 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, /* get the attributes from the tokenizer */ n = XmlGetAttributes(enc, attStr, parser->m_attsSize, parser->m_atts); + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - nDefaultAtts) { + return XML_ERROR_NO_MEMORY; + } + if (n + nDefaultAtts > parser->m_attsSize) { int oldAttsSize = parser->m_attsSize; ATTRIBUTE *temp; #ifdef XML_ATTR_INFO XML_AttrInfo *temp2; #endif + + /* Detect and prevent integer overflow */ + if ((nDefaultAtts > INT_MAX - INIT_ATTS_SIZE) + || (n > INT_MAX - (nDefaultAtts + INIT_ATTS_SIZE))) { + return XML_ERROR_NO_MEMORY; + } + parser->m_attsSize = n + nDefaultAtts + INIT_ATTS_SIZE; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(ATTRIBUTE)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (ATTRIBUTE *)REALLOC(parser, (void *)parser->m_atts, parser->m_attsSize * sizeof(ATTRIBUTE)); if (temp == NULL) { @@ -3275,6 +3307,17 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, } parser->m_atts = temp; #ifdef XML_ATTR_INFO + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +# if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(XML_AttrInfo)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +# endif + temp2 = (XML_AttrInfo *)REALLOC(parser, (void *)parser->m_attInfo, parser->m_attsSize * sizeof(XML_AttrInfo)); if (temp2 == NULL) { @@ -3413,7 +3456,13 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, if (nPrefixes) { int j; /* hash table index */ unsigned long version = parser->m_nsAttsVersion; - int nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(unsigned int) * 8 /* bits per byte */) { + return XML_ERROR_NO_MEMORY; + } + + unsigned int nsAttsSize = 1u << parser->m_nsAttsPower; unsigned char oldNsAttsPower = parser->m_nsAttsPower; /* size of hash table must be at least 2 * (# of prefixed attributes) */ if ((nPrefixes << 1) @@ -3424,7 +3473,28 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, ; if (parser->m_nsAttsPower < 3) parser->m_nsAttsPower = 3; - nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(nsAttsSize) * 8 /* bits per byte */) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } + + nsAttsSize = 1u << parser->m_nsAttsPower; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (nsAttsSize > (size_t)(-1) / sizeof(NS_ATT)) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (NS_ATT *)REALLOC(parser, parser->m_nsAtts, nsAttsSize * sizeof(NS_ATT)); if (! temp) { @@ -3582,9 +3652,31 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, tagNamePtr->prefixLen = prefixLen; for (i = 0; localPart[i++];) ; /* i includes null terminator */ + + /* Detect and prevent integer overflow */ + if (binding->uriLen > INT_MAX - prefixLen + || i > INT_MAX - (binding->uriLen + prefixLen)) { + return XML_ERROR_NO_MEMORY; + } + n = i + binding->uriLen + prefixLen; if (n > binding->uriAlloc) { TAG *p; + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(n + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + uri = (XML_Char *)MALLOC(parser, (n + EXPAND_SPARE) * sizeof(XML_Char)); if (! uri) return XML_ERROR_NO_MEMORY; @@ -3680,6 +3772,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (parser->m_freeBindingList) { b = parser->m_freeBindingList; if (len > b->uriAlloc) { + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + XML_Char *temp = (XML_Char *)REALLOC( parser, b->uri, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (temp == NULL) @@ -3692,6 +3799,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, b = (BINDING *)MALLOC(parser, sizeof(BINDING)); if (! b) return XML_ERROR_NO_MEMORY; + + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + b->uri = (XML_Char *)MALLOC(parser, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (! b->uri) { @@ -3976,7 +4098,7 @@ initializeEncoding(XML_Parser parser) { const char *s; #ifdef XML_UNICODE char encodingBuf[128]; - /* See comments about `protoclEncodingName` in parserInit() */ + /* See comments about `protocolEncodingName` in parserInit() */ if (! parser->m_protocolEncodingName) s = NULL; else { @@ -5018,6 +5140,11 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (parser->m_prologState.level >= parser->m_groupSize) { if (parser->m_groupSize) { { + /* Detect and prevent integer overflow */ + if (parser->m_groupSize > (unsigned int)(-1) / 2u) { + return XML_ERROR_NO_MEMORY; + } + char *const new_connector = (char *)REALLOC( parser, parser->m_groupConnector, parser->m_groupSize *= 2); if (new_connector == NULL) { @@ -5028,6 +5155,16 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, } if (dtd->scaffIndex) { + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (parser->m_groupSize > (size_t)(-1) / sizeof(int)) { + return XML_ERROR_NO_MEMORY; + } +#endif + int *const new_scaff_index = (int *)REALLOC( parser, dtd->scaffIndex, parser->m_groupSize * sizeof(int)); if (new_scaff_index == NULL) @@ -5236,7 +5373,7 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (dtd->in_eldecl) { ELEMENT_TYPE *el; const XML_Char *name; - int nameLen; + size_t nameLen; const char *nxt = (quant == XML_CQUANT_NONE ? next : next - enc->minBytesPerChar); int myindex = nextScaffoldPart(parser); @@ -5252,7 +5389,13 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, nameLen = 0; for (; name[nameLen++];) ; - dtd->contentStringLen += nameLen; + + /* Detect and prevent integer overflow */ + if (nameLen > UINT_MAX - dtd->contentStringLen) { + return XML_ERROR_NO_MEMORY; + } + + dtd->contentStringLen += (unsigned)nameLen; if (parser->m_elementDeclHandler) handleDefault = XML_FALSE; } @@ -6098,7 +6241,24 @@ defineAttribute(ELEMENT_TYPE *type, ATTRIBUTE_ID *attId, XML_Bool isCdata, } } else { DEFAULT_ATTRIBUTE *temp; + + /* Detect and prevent integer overflow */ + if (type->allocDefaultAtts > INT_MAX / 2) { + return 0; + } + int count = type->allocDefaultAtts * 2; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)count > (size_t)(-1) / sizeof(DEFAULT_ATTRIBUTE)) { + return 0; + } +#endif + temp = (DEFAULT_ATTRIBUTE *)REALLOC(parser, type->defaultAtts, (count * sizeof(DEFAULT_ATTRIBUTE))); if (temp == NULL) @@ -6388,7 +6548,7 @@ normalizePublicId(XML_Char *publicId) { static DTD * dtdCreate(const XML_Memory_Handling_Suite *ms) { - DTD *p = (DTD *)ms->malloc_fcn(sizeof(DTD)); + DTD *p = ms->malloc_fcn(sizeof(DTD)); if (p == NULL) return p; poolInit(&(p->pool), ms); @@ -6561,8 +6721,8 @@ dtdCopy(XML_Parser oldParser, DTD *newDtd, const DTD *oldDtd, if (! newE) return 0; if (oldE->nDefaultAtts) { - newE->defaultAtts = (DEFAULT_ATTRIBUTE *)ms->malloc_fcn( - oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); + newE->defaultAtts + = ms->malloc_fcn(oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); if (! newE->defaultAtts) { return 0; } @@ -6724,7 +6884,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* table->size is a power of 2 */ table->size = (size_t)1 << INIT_POWER; tsize = table->size * sizeof(NAMED *); - table->v = (NAMED **)table->mem->malloc_fcn(tsize); + table->v = table->mem->malloc_fcn(tsize); if (! table->v) { table->size = 0; return NULL; @@ -6749,10 +6909,22 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* check for overflow (table is half full) */ if (table->used >> (table->power - 1)) { unsigned char newPower = table->power + 1; + + /* Detect and prevent invalid shift */ + if (newPower >= sizeof(unsigned long) * 8 /* bits per byte */) { + return NULL; + } + size_t newSize = (size_t)1 << newPower; unsigned long newMask = (unsigned long)newSize - 1; + + /* Detect and prevent integer overflow */ + if (newSize > (size_t)(-1) / sizeof(NAMED *)) { + return NULL; + } + size_t tsize = newSize * sizeof(NAMED *); - NAMED **newV = (NAMED **)table->mem->malloc_fcn(tsize); + NAMED **newV = table->mem->malloc_fcn(tsize); if (! newV) return NULL; memset(newV, 0, tsize); @@ -6781,7 +6953,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { } } } - table->v[i] = (NAMED *)table->mem->malloc_fcn(createSize); + table->v[i] = table->mem->malloc_fcn(createSize); if (! table->v[i]) return NULL; memset(table->v[i], 0, createSize); @@ -7069,7 +7241,7 @@ poolGrow(STRING_POOL *pool) { if (bytesToAllocate == 0) return XML_FALSE; - tem = (BLOCK *)pool->mem->malloc_fcn(bytesToAllocate); + tem = pool->mem->malloc_fcn(bytesToAllocate); if (! tem) return XML_FALSE; tem->size = blockSize; @@ -7100,6 +7272,20 @@ nextScaffoldPart(XML_Parser parser) { if (dtd->scaffCount >= dtd->scaffSize) { CONTENT_SCAFFOLD *temp; if (dtd->scaffold) { + /* Detect and prevent integer overflow */ + if (dtd->scaffSize > UINT_MAX / 2u) { + return -1; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffSize > (size_t)(-1) / 2u / sizeof(CONTENT_SCAFFOLD)) { + return -1; + } +#endif + temp = (CONTENT_SCAFFOLD *)REALLOC( parser, dtd->scaffold, dtd->scaffSize * 2 * sizeof(CONTENT_SCAFFOLD)); if (temp == NULL) @@ -7169,8 +7355,26 @@ build_model(XML_Parser parser) { XML_Content *ret; XML_Content *cpos; XML_Char *str; - int allocsize = (dtd->scaffCount * sizeof(XML_Content) - + (dtd->contentStringLen * sizeof(XML_Char))); + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffCount > (size_t)(-1) / sizeof(XML_Content)) { + return NULL; + } + if (dtd->contentStringLen > (size_t)(-1) / sizeof(XML_Char)) { + return NULL; + } +#endif + if (dtd->scaffCount * sizeof(XML_Content) + > (size_t)(-1) - dtd->contentStringLen * sizeof(XML_Char)) { + return NULL; + } + + const size_t allocsize = (dtd->scaffCount * sizeof(XML_Content) + + (dtd->contentStringLen * sizeof(XML_Char))); ret = (XML_Content *)MALLOC(parser, allocsize); if (! ret) diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 08173b0fd541d..77746ee42d10a 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -15,6 +15,7 @@ Copyright (c) 2016-2021 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -37,14 +38,14 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#include + #include #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "expat_external.h" #include "internal.h" #include "xmlrole.h" diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index 5b935718743ad..502ca1adc33b9 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -20,6 +20,7 @@ Copyright (c) 2017 Benbuck Nason Copyright (c) 2017 Jos? Guti?rrez de la Concha Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -42,16 +43,16 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ -#ifdef _WIN32 -# include "winconfig.h" -#endif - #include #include #include /* memcpy */ #include +#ifdef _WIN32 +# include "winconfig.h" +#endif + #include "expat_external.h" #include "internal.h" #include "xmltok.h" diff --git a/Modules/expat/xmltok_ns.c b/Modules/expat/xmltok_ns.c index 5fd8392235940..fbdd3e3c7b799 100644 --- a/Modules/expat/xmltok_ns.c +++ b/Modules/expat/xmltok_ns.c @@ -11,7 +11,7 @@ Copyright (c) 2002 Greg Stein Copyright (c) 2002 Fred L. Drake, Jr. Copyright (c) 2002-2006 Karl Waclawek - Copyright (c) 2017 Sebastian Pipping + Copyright (c) 2017-2021 Sebastian Pipping Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -93,7 +93,7 @@ NS(XmlInitEncoding)(INIT_ENCODING *p, const ENCODING **encPtr, static const ENCODING * NS(findEncoding)(const ENCODING *enc, const char *ptr, const char *end) { # define ENCODING_MAX 128 - char buf[ENCODING_MAX]; + char buf[ENCODING_MAX] = ""; char *p = buf; int i; XmlUtf8Convert(enc, &ptr, end, &p, p + ENCODING_MAX - 1); From webhook-mailer at python.org Thu Feb 17 16:58:30 2022 From: webhook-mailer at python.org (corona10) Date: Thu, 17 Feb 2022 21:58:30 -0000 Subject: [Python-checkins] bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) (GH-31295) Message-ID: https://github.com/python/cpython/commit/e7828904f39588dad438c5d341a31e72e9cb1775 commit: e7828904f39588dad438c5d341a31e72e9cb1775 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: corona10 date: 2022-02-18T06:58:26+09:00 summary: bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) (GH-31295) files: A Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst M Modules/expat/expat.h M Modules/expat/xmlparse.c M Modules/expat/xmlrole.c M Modules/expat/xmltok.c M Modules/expat/xmltok_ns.c diff --git a/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst new file mode 100644 index 0000000000000..9c1f24c0e5171 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst @@ -0,0 +1 @@ +expat: Update libexpat from 2.4.1 to 2.4.4 diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index b7d6d354801b3..4c5704fd9336b 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -11,7 +11,7 @@ Copyright (c) 2000-2005 Fred L. Drake, Jr. Copyright (c) 2001-2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Cristian Rodr?guez Copyright (c) 2016 Thomas Beutlich Copyright (c) 2017 Rhodri James @@ -1041,7 +1041,7 @@ XML_SetBillionLaughsAttackProtectionActivationThreshold( */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 4 -#define XML_MICRO_VERSION 1 +#define XML_MICRO_VERSION 4 #ifdef __cplusplus } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 034a03c30851a..4b43e61321691 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 8539b9040d9d901366a62560a064af7cb99811335784b363abc039c5b0ebc416 (2.4.1+) +/* 2e2c8ce5f11a473d65ec313ab20ceee6afefb355f5405afc06e7204e2e41c8c0 (2.4.4+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -13,7 +13,7 @@ Copyright (c) 2002-2016 Karl Waclawek Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016 Eric Rahm - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Gaurav Copyright (c) 2016 Thomas Beutlich Copyright (c) 2016 Gustavo Grieco @@ -32,6 +32,8 @@ Copyright (c) 2019 David Loffredo Copyright (c) 2019-2020 Ben Wagner Copyright (c) 2019 Vadim Zeitlin + Copyright (c) 2021 Dong-hee Na + Copyright (c) 2022 Samanta Navarro Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -54,6 +56,10 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#define XML_BUILDING_EXPAT 1 + +#include + #if ! defined(_GNU_SOURCE) # define _GNU_SOURCE 1 /* syscall prototype */ #endif @@ -84,14 +90,10 @@ # include #endif -#define XML_BUILDING_EXPAT 1 - #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "ascii.h" #include "expat.h" #include "siphash.h" @@ -973,7 +975,7 @@ parserCreate(const XML_Char *encodingName, if (memsuite) { XML_Memory_Handling_Suite *mtemp; - parser = (XML_Parser)memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); + parser = memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); if (parser != NULL) { mtemp = (XML_Memory_Handling_Suite *)&(parser->m_mem); mtemp->malloc_fcn = memsuite->malloc_fcn; @@ -2066,6 +2068,11 @@ XML_GetBuffer(XML_Parser parser, int len) { keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; + /* Detect and prevent integer overflow */ + if (keep > INT_MAX - neededSize) { + parser->m_errorCode = XML_ERROR_NO_MEMORY; + return NULL; + } neededSize += keep; #endif /* defined XML_CONTEXT_BYTES */ if (neededSize @@ -3260,13 +3267,38 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, /* get the attributes from the tokenizer */ n = XmlGetAttributes(enc, attStr, parser->m_attsSize, parser->m_atts); + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - nDefaultAtts) { + return XML_ERROR_NO_MEMORY; + } + if (n + nDefaultAtts > parser->m_attsSize) { int oldAttsSize = parser->m_attsSize; ATTRIBUTE *temp; #ifdef XML_ATTR_INFO XML_AttrInfo *temp2; #endif + + /* Detect and prevent integer overflow */ + if ((nDefaultAtts > INT_MAX - INIT_ATTS_SIZE) + || (n > INT_MAX - (nDefaultAtts + INIT_ATTS_SIZE))) { + return XML_ERROR_NO_MEMORY; + } + parser->m_attsSize = n + nDefaultAtts + INIT_ATTS_SIZE; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(ATTRIBUTE)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (ATTRIBUTE *)REALLOC(parser, (void *)parser->m_atts, parser->m_attsSize * sizeof(ATTRIBUTE)); if (temp == NULL) { @@ -3275,6 +3307,17 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, } parser->m_atts = temp; #ifdef XML_ATTR_INFO + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +# if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(XML_AttrInfo)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +# endif + temp2 = (XML_AttrInfo *)REALLOC(parser, (void *)parser->m_attInfo, parser->m_attsSize * sizeof(XML_AttrInfo)); if (temp2 == NULL) { @@ -3413,7 +3456,13 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, if (nPrefixes) { int j; /* hash table index */ unsigned long version = parser->m_nsAttsVersion; - int nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(unsigned int) * 8 /* bits per byte */) { + return XML_ERROR_NO_MEMORY; + } + + unsigned int nsAttsSize = 1u << parser->m_nsAttsPower; unsigned char oldNsAttsPower = parser->m_nsAttsPower; /* size of hash table must be at least 2 * (# of prefixed attributes) */ if ((nPrefixes << 1) @@ -3424,7 +3473,28 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, ; if (parser->m_nsAttsPower < 3) parser->m_nsAttsPower = 3; - nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(nsAttsSize) * 8 /* bits per byte */) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } + + nsAttsSize = 1u << parser->m_nsAttsPower; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (nsAttsSize > (size_t)(-1) / sizeof(NS_ATT)) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (NS_ATT *)REALLOC(parser, parser->m_nsAtts, nsAttsSize * sizeof(NS_ATT)); if (! temp) { @@ -3582,9 +3652,31 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, tagNamePtr->prefixLen = prefixLen; for (i = 0; localPart[i++];) ; /* i includes null terminator */ + + /* Detect and prevent integer overflow */ + if (binding->uriLen > INT_MAX - prefixLen + || i > INT_MAX - (binding->uriLen + prefixLen)) { + return XML_ERROR_NO_MEMORY; + } + n = i + binding->uriLen + prefixLen; if (n > binding->uriAlloc) { TAG *p; + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(n + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + uri = (XML_Char *)MALLOC(parser, (n + EXPAND_SPARE) * sizeof(XML_Char)); if (! uri) return XML_ERROR_NO_MEMORY; @@ -3680,6 +3772,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (parser->m_freeBindingList) { b = parser->m_freeBindingList; if (len > b->uriAlloc) { + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + XML_Char *temp = (XML_Char *)REALLOC( parser, b->uri, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (temp == NULL) @@ -3692,6 +3799,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, b = (BINDING *)MALLOC(parser, sizeof(BINDING)); if (! b) return XML_ERROR_NO_MEMORY; + + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + b->uri = (XML_Char *)MALLOC(parser, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (! b->uri) { @@ -3976,7 +4098,7 @@ initializeEncoding(XML_Parser parser) { const char *s; #ifdef XML_UNICODE char encodingBuf[128]; - /* See comments about `protoclEncodingName` in parserInit() */ + /* See comments about `protocolEncodingName` in parserInit() */ if (! parser->m_protocolEncodingName) s = NULL; else { @@ -5018,6 +5140,11 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (parser->m_prologState.level >= parser->m_groupSize) { if (parser->m_groupSize) { { + /* Detect and prevent integer overflow */ + if (parser->m_groupSize > (unsigned int)(-1) / 2u) { + return XML_ERROR_NO_MEMORY; + } + char *const new_connector = (char *)REALLOC( parser, parser->m_groupConnector, parser->m_groupSize *= 2); if (new_connector == NULL) { @@ -5028,6 +5155,16 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, } if (dtd->scaffIndex) { + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (parser->m_groupSize > (size_t)(-1) / sizeof(int)) { + return XML_ERROR_NO_MEMORY; + } +#endif + int *const new_scaff_index = (int *)REALLOC( parser, dtd->scaffIndex, parser->m_groupSize * sizeof(int)); if (new_scaff_index == NULL) @@ -5236,7 +5373,7 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (dtd->in_eldecl) { ELEMENT_TYPE *el; const XML_Char *name; - int nameLen; + size_t nameLen; const char *nxt = (quant == XML_CQUANT_NONE ? next : next - enc->minBytesPerChar); int myindex = nextScaffoldPart(parser); @@ -5252,7 +5389,13 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, nameLen = 0; for (; name[nameLen++];) ; - dtd->contentStringLen += nameLen; + + /* Detect and prevent integer overflow */ + if (nameLen > UINT_MAX - dtd->contentStringLen) { + return XML_ERROR_NO_MEMORY; + } + + dtd->contentStringLen += (unsigned)nameLen; if (parser->m_elementDeclHandler) handleDefault = XML_FALSE; } @@ -6098,7 +6241,24 @@ defineAttribute(ELEMENT_TYPE *type, ATTRIBUTE_ID *attId, XML_Bool isCdata, } } else { DEFAULT_ATTRIBUTE *temp; + + /* Detect and prevent integer overflow */ + if (type->allocDefaultAtts > INT_MAX / 2) { + return 0; + } + int count = type->allocDefaultAtts * 2; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)count > (size_t)(-1) / sizeof(DEFAULT_ATTRIBUTE)) { + return 0; + } +#endif + temp = (DEFAULT_ATTRIBUTE *)REALLOC(parser, type->defaultAtts, (count * sizeof(DEFAULT_ATTRIBUTE))); if (temp == NULL) @@ -6388,7 +6548,7 @@ normalizePublicId(XML_Char *publicId) { static DTD * dtdCreate(const XML_Memory_Handling_Suite *ms) { - DTD *p = (DTD *)ms->malloc_fcn(sizeof(DTD)); + DTD *p = ms->malloc_fcn(sizeof(DTD)); if (p == NULL) return p; poolInit(&(p->pool), ms); @@ -6561,8 +6721,8 @@ dtdCopy(XML_Parser oldParser, DTD *newDtd, const DTD *oldDtd, if (! newE) return 0; if (oldE->nDefaultAtts) { - newE->defaultAtts = (DEFAULT_ATTRIBUTE *)ms->malloc_fcn( - oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); + newE->defaultAtts + = ms->malloc_fcn(oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); if (! newE->defaultAtts) { return 0; } @@ -6724,7 +6884,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* table->size is a power of 2 */ table->size = (size_t)1 << INIT_POWER; tsize = table->size * sizeof(NAMED *); - table->v = (NAMED **)table->mem->malloc_fcn(tsize); + table->v = table->mem->malloc_fcn(tsize); if (! table->v) { table->size = 0; return NULL; @@ -6749,10 +6909,22 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* check for overflow (table is half full) */ if (table->used >> (table->power - 1)) { unsigned char newPower = table->power + 1; + + /* Detect and prevent invalid shift */ + if (newPower >= sizeof(unsigned long) * 8 /* bits per byte */) { + return NULL; + } + size_t newSize = (size_t)1 << newPower; unsigned long newMask = (unsigned long)newSize - 1; + + /* Detect and prevent integer overflow */ + if (newSize > (size_t)(-1) / sizeof(NAMED *)) { + return NULL; + } + size_t tsize = newSize * sizeof(NAMED *); - NAMED **newV = (NAMED **)table->mem->malloc_fcn(tsize); + NAMED **newV = table->mem->malloc_fcn(tsize); if (! newV) return NULL; memset(newV, 0, tsize); @@ -6781,7 +6953,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { } } } - table->v[i] = (NAMED *)table->mem->malloc_fcn(createSize); + table->v[i] = table->mem->malloc_fcn(createSize); if (! table->v[i]) return NULL; memset(table->v[i], 0, createSize); @@ -7069,7 +7241,7 @@ poolGrow(STRING_POOL *pool) { if (bytesToAllocate == 0) return XML_FALSE; - tem = (BLOCK *)pool->mem->malloc_fcn(bytesToAllocate); + tem = pool->mem->malloc_fcn(bytesToAllocate); if (! tem) return XML_FALSE; tem->size = blockSize; @@ -7100,6 +7272,20 @@ nextScaffoldPart(XML_Parser parser) { if (dtd->scaffCount >= dtd->scaffSize) { CONTENT_SCAFFOLD *temp; if (dtd->scaffold) { + /* Detect and prevent integer overflow */ + if (dtd->scaffSize > UINT_MAX / 2u) { + return -1; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffSize > (size_t)(-1) / 2u / sizeof(CONTENT_SCAFFOLD)) { + return -1; + } +#endif + temp = (CONTENT_SCAFFOLD *)REALLOC( parser, dtd->scaffold, dtd->scaffSize * 2 * sizeof(CONTENT_SCAFFOLD)); if (temp == NULL) @@ -7169,8 +7355,26 @@ build_model(XML_Parser parser) { XML_Content *ret; XML_Content *cpos; XML_Char *str; - int allocsize = (dtd->scaffCount * sizeof(XML_Content) - + (dtd->contentStringLen * sizeof(XML_Char))); + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffCount > (size_t)(-1) / sizeof(XML_Content)) { + return NULL; + } + if (dtd->contentStringLen > (size_t)(-1) / sizeof(XML_Char)) { + return NULL; + } +#endif + if (dtd->scaffCount * sizeof(XML_Content) + > (size_t)(-1) - dtd->contentStringLen * sizeof(XML_Char)) { + return NULL; + } + + const size_t allocsize = (dtd->scaffCount * sizeof(XML_Content) + + (dtd->contentStringLen * sizeof(XML_Char))); ret = (XML_Content *)MALLOC(parser, allocsize); if (! ret) diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 08173b0fd541d..77746ee42d10a 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -15,6 +15,7 @@ Copyright (c) 2016-2021 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -37,14 +38,14 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#include + #include #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "expat_external.h" #include "internal.h" #include "xmlrole.h" diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index 5b935718743ad..502ca1adc33b9 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -20,6 +20,7 @@ Copyright (c) 2017 Benbuck Nason Copyright (c) 2017 Jos? Guti?rrez de la Concha Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -42,16 +43,16 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ -#ifdef _WIN32 -# include "winconfig.h" -#endif - #include #include #include /* memcpy */ #include +#ifdef _WIN32 +# include "winconfig.h" +#endif + #include "expat_external.h" #include "internal.h" #include "xmltok.h" diff --git a/Modules/expat/xmltok_ns.c b/Modules/expat/xmltok_ns.c index 5fd8392235940..fbdd3e3c7b799 100644 --- a/Modules/expat/xmltok_ns.c +++ b/Modules/expat/xmltok_ns.c @@ -11,7 +11,7 @@ Copyright (c) 2002 Greg Stein Copyright (c) 2002 Fred L. Drake, Jr. Copyright (c) 2002-2006 Karl Waclawek - Copyright (c) 2017 Sebastian Pipping + Copyright (c) 2017-2021 Sebastian Pipping Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -93,7 +93,7 @@ NS(XmlInitEncoding)(INIT_ENCODING *p, const ENCODING **encPtr, static const ENCODING * NS(findEncoding)(const ENCODING *enc, const char *ptr, const char *end) { # define ENCODING_MAX 128 - char buf[ENCODING_MAX]; + char buf[ENCODING_MAX] = ""; char *p = buf; int i; XmlUtf8Convert(enc, &ptr, end, &p, p + ENCODING_MAX - 1); From webhook-mailer at python.org Thu Feb 17 19:02:51 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 18 Feb 2022 00:02:51 -0000 Subject: [Python-checkins] docs: Link `match` statement in tutorial (GH-31396) Message-ID: https://github.com/python/cpython/commit/35f55cc5c690a97853f1b544aac53574ab755b54 commit: 35f55cc5c690a97853f1b544aac53574ab755b54 branch: main author: William Andrea committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-17T16:02:42-08:00 summary: docs: Link `match` statement in tutorial (GH-31396) Add a link to the `match` statement from its section in the tutorial. This should be backported to 3.10. [Initially submitted as #31395 but against the wrong branch. Thanks @Mariatta] Automerge-Triggered-By: GH:Mariatta files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index fad8746014646..589263edd3571 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -250,7 +250,7 @@ at a more abstract level. The :keyword:`!pass` is silently ignored:: :keyword:`!match` Statements ============================ -A match statement takes an expression and compares its value to successive +A :keyword:`match` statement takes an expression and compares its value to successive patterns given as one or more case blocks. This is superficially similar to a switch statement in C, Java or JavaScript (and many other languages), but it can also extract components (sequence elements or From webhook-mailer at python.org Thu Feb 17 20:45:44 2022 From: webhook-mailer at python.org (sweeneyde) Date: Fri, 18 Feb 2022 01:45:44 -0000 Subject: [Python-checkins] Fix mistake in barry_as_FLUFL test (GH-31392) Message-ID: https://github.com/python/cpython/commit/5f1c205dcec0adf68b76ba78247a0ec1fa9a7b66 commit: 5f1c205dcec0adf68b76ba78247a0ec1fa9a7b66 branch: main author: Patrick Reader <_ at pxeger.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2022-02-17T20:45:35-05:00 summary: Fix mistake in barry_as_FLUFL test (GH-31392) Use assertEqual(), not assertTrue(lineno, 2) files: M Lib/test/test_flufl.py diff --git a/Lib/test/test_flufl.py b/Lib/test/test_flufl.py index 0ff54aa227e37..a81a4d4c8f0e4 100644 --- a/Lib/test/test_flufl.py +++ b/Lib/test/test_flufl.py @@ -17,7 +17,7 @@ def test_barry_as_bdfl(self): self.assertIn('2 != 3', cm.exception.text) self.assertEqual(cm.exception.filename, '') - self.assertTrue(cm.exception.lineno, 2) + self.assertEqual(cm.exception.lineno, 2) # The old parser reports the end of the token and the new # parser reports the start of the token self.assertEqual(cm.exception.offset, 3) From webhook-mailer at python.org Thu Feb 17 21:08:37 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 18 Feb 2022 02:08:37 -0000 Subject: [Python-checkins] Fix mistake in barry_as_FLUFL test (GH-31392) Message-ID: https://github.com/python/cpython/commit/c11f41b2c989121e34ccf94cb5b7896fbdb859ab commit: c11f41b2c989121e34ccf94cb5b7896fbdb859ab branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-17T18:08:28-08:00 summary: Fix mistake in barry_as_FLUFL test (GH-31392) Use assertEqual(), not assertTrue(lineno, 2) (cherry picked from commit 5f1c205dcec0adf68b76ba78247a0ec1fa9a7b66) Co-authored-by: Patrick Reader <_ at pxeger.com> files: M Lib/test/test_flufl.py diff --git a/Lib/test/test_flufl.py b/Lib/test/test_flufl.py index 22285859a92bb..e9462dfc70c4e 100644 --- a/Lib/test/test_flufl.py +++ b/Lib/test/test_flufl.py @@ -17,7 +17,7 @@ def test_barry_as_bdfl(self): self.assertIn('2 != 3', cm.exception.text) self.assertEqual(cm.exception.filename, '') - self.assertTrue(cm.exception.lineno, 2) + self.assertEqual(cm.exception.lineno, 2) # The old parser reports the end of the token and the new # parser reports the start of the token self.assertEqual(cm.exception.offset, 4 if support.use_old_parser() else 3) From webhook-mailer at python.org Thu Feb 17 21:11:41 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 18 Feb 2022 02:11:41 -0000 Subject: [Python-checkins] Fix mistake in barry_as_FLUFL test (GH-31392) Message-ID: https://github.com/python/cpython/commit/095ffe715e341690fd047917d83bebe0359c9d80 commit: 095ffe715e341690fd047917d83bebe0359c9d80 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-17T18:11:34-08:00 summary: Fix mistake in barry_as_FLUFL test (GH-31392) Use assertEqual(), not assertTrue(lineno, 2) (cherry picked from commit 5f1c205dcec0adf68b76ba78247a0ec1fa9a7b66) Co-authored-by: Patrick Reader <_ at pxeger.com> files: M Lib/test/test_flufl.py diff --git a/Lib/test/test_flufl.py b/Lib/test/test_flufl.py index 0ff54aa227e37..a81a4d4c8f0e4 100644 --- a/Lib/test/test_flufl.py +++ b/Lib/test/test_flufl.py @@ -17,7 +17,7 @@ def test_barry_as_bdfl(self): self.assertIn('2 != 3', cm.exception.text) self.assertEqual(cm.exception.filename, '') - self.assertTrue(cm.exception.lineno, 2) + self.assertEqual(cm.exception.lineno, 2) # The old parser reports the end of the token and the new # parser reports the start of the token self.assertEqual(cm.exception.offset, 3) From webhook-mailer at python.org Thu Feb 17 23:10:38 2022 From: webhook-mailer at python.org (terryjreedy) Date: Fri, 18 Feb 2022 04:10:38 -0000 Subject: [Python-checkins] Fix missing spaces in os.walk example (GH-29918) Message-ID: https://github.com/python/cpython/commit/2a38e1ab65cb28af3babda17f6ad3f8e514a7455 commit: 2a38e1ab65cb28af3babda17f6ad3f8e514a7455 branch: main author: ?? <49506152+BruceLee569 at users.noreply.github.com> committer: terryjreedy date: 2022-02-17T23:10:33-05:00 summary: Fix missing spaces in os.walk example (GH-29918) files: M Lib/os.py diff --git a/Lib/os.py b/Lib/os.py index ab7ef3c17798b..4626d1fc4a9b5 100644 --- a/Lib/os.py +++ b/Lib/os.py @@ -331,8 +331,8 @@ def walk(top, topdown=True, onerror=None, followlinks=False): import os from os.path import join, getsize for root, dirs, files in os.walk('python/Lib/email'): - print(root, "consumes", end="") - print(sum(getsize(join(root, name)) for name in files), end="") + print(root, "consumes ") + print(sum(getsize(join(root, name)) for name in files), end=" ") print("bytes in", len(files), "non-directory files") if 'CVS' in dirs: dirs.remove('CVS') # don't visit CVS directories From webhook-mailer at python.org Thu Feb 17 23:15:09 2022 From: webhook-mailer at python.org (terryjreedy) Date: Fri, 18 Feb 2022 04:15:09 -0000 Subject: [Python-checkins] docs: Link `match` statement in tutorial (GH-31396) Message-ID: https://github.com/python/cpython/commit/feb44550888eb4755efee11bf01daeb285e5b685 commit: feb44550888eb4755efee11bf01daeb285e5b685 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: terryjreedy date: 2022-02-17T23:15:00-05:00 summary: docs: Link `match` statement in tutorial (GH-31396) Add a link to the `match` statement from its section in the tutorial. Automerge-Triggered-By: GH:Mariatta (cherry picked from commit 35f55cc5c690a97853f1b544aac53574ab755b54) Co-authored-by: William Andrea files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index fad8746014646..589263edd3571 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -250,7 +250,7 @@ at a more abstract level. The :keyword:`!pass` is silently ignored:: :keyword:`!match` Statements ============================ -A match statement takes an expression and compares its value to successive +A :keyword:`match` statement takes an expression and compares its value to successive patterns given as one or more case blocks. This is superficially similar to a switch statement in C, Java or JavaScript (and many other languages), but it can also extract components (sequence elements or From webhook-mailer at python.org Fri Feb 18 00:30:58 2022 From: webhook-mailer at python.org (gvanrossum) Date: Fri, 18 Feb 2022 05:30:58 -0000 Subject: [Python-checkins] bpo-46752: Slight improvements to TaskGroup API (GH-31398) Message-ID: https://github.com/python/cpython/commit/d85121660ea50bbe8fbd31797aa6e4afe0850388 commit: d85121660ea50bbe8fbd31797aa6e4afe0850388 branch: main author: Guido van Rossum committer: gvanrossum date: 2022-02-17T21:30:44-08:00 summary: bpo-46752: Slight improvements to TaskGroup API (GH-31398) * Remove task group names (for now) We're not sure that they are needed, and once in the code we would never be able to get rid of them. Yury wrote: > Ideally, there should be a way for someone to build a "trace" > of taskgroups/task leading to the current running task. > We could do that using contextvars, but I'm not sure we should > do that in 3.11. * Pass optional name on to task in create_task() * Remove a bunch of unused stuff files: M Lib/asyncio/taskgroups.py M Lib/test/test_asyncio/test_taskgroups.py diff --git a/Lib/asyncio/taskgroups.py b/Lib/asyncio/taskgroups.py index 718277892c51c..57b0eafefc16f 100644 --- a/Lib/asyncio/taskgroups.py +++ b/Lib/asyncio/taskgroups.py @@ -3,10 +3,6 @@ __all__ = ["TaskGroup"] -import itertools -import textwrap -import traceback -import types import weakref from . import events @@ -15,12 +11,7 @@ class TaskGroup: - def __init__(self, *, name=None): - if name is None: - self._name = f'tg-{_name_counter()}' - else: - self._name = str(name) - + def __init__(self): self._entered = False self._exiting = False self._aborting = False @@ -33,11 +24,8 @@ def __init__(self, *, name=None): self._base_error = None self._on_completed_fut = None - def get_name(self): - return self._name - def __repr__(self): - msg = f' https://github.com/python/cpython/commit/2923d87ca258b9d421e8147b12f0d98295ee3f8e commit: 2923d87ca258b9d421e8147b12f0d98295ee3f8e branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2022-02-18T10:58:48+02:00 summary: bpo-46777: Fix incorrect use of directives in asyncio documentation (GH-31388) files: M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-future.rst M Doc/library/asyncio-queue.rst M Doc/library/asyncio-stream.rst M Doc/library/asyncio-subprocess.rst M Doc/library/asyncio-sync.rst M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 5b48945599f76..0c65d75dbf8e0 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -343,7 +343,7 @@ Creating Futures and Tasks the name of the task using :meth:`Task.set_name`. .. versionchanged:: 3.8 - Added the ``name`` parameter. + Added the *name* parameter. .. method:: loop.set_task_factory(factory) @@ -456,7 +456,20 @@ Opening network connections to wait for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). - .. versionadded:: 3.8 + .. versionchanged:: 3.5 + + Added support for SSL/TLS in :class:`ProactorEventLoop`. + + .. versionchanged:: 3.6 + + The socket option :py:data:`~socket.TCP_NODELAY` is set by default + for all TCP connections. + + .. versionchanged:: 3.7 + + Added the *ssl_handshake_timeout* parameter. + + .. versionchanged:: 3.8 Added the *happy_eyeballs_delay* and *interleave* parameters. @@ -471,19 +484,6 @@ Opening network connections For more information: https://tools.ietf.org/html/rfc6555 - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* parameter. - - .. versionchanged:: 3.6 - - The socket option :py:data:`~socket.TCP_NODELAY` is set by default - for all TCP connections. - - .. versionchanged:: 3.5 - - Added support for SSL/TLS in :class:`ProactorEventLoop`. - .. seealso:: The :func:`open_connection` function is a high-level alternative @@ -588,12 +588,8 @@ Opening network connections .. availability:: Unix. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* parameter. - .. versionchanged:: 3.7 - + Added the *ssl_handshake_timeout* parameter. The *path* parameter can now be a :term:`path-like object`. @@ -672,15 +668,6 @@ Creating network servers :meth:`Server.serve_forever` to make the server to start accepting connections. - .. versionadded:: 3.7 - - Added *ssl_handshake_timeout* and *start_serving* parameters. - - .. versionchanged:: 3.6 - - The socket option :py:data:`~socket.TCP_NODELAY` is set by default - for all TCP connections. - .. versionchanged:: 3.5 Added support for SSL/TLS in :class:`ProactorEventLoop`. @@ -689,6 +676,12 @@ Creating network servers The *host* parameter can be a sequence of strings. + .. versionchanged:: 3.6 + + Added *ssl_handshake_timeout* and *start_serving* parameters. + The socket option :py:data:`~socket.TCP_NODELAY` is set by default + for all TCP connections. + .. seealso:: The :func:`start_server` function is a higher-level alternative API @@ -713,12 +706,9 @@ Creating network servers .. availability:: Unix. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* and *start_serving* parameters. - .. versionchanged:: 3.7 + Added the *ssl_handshake_timeout* and *start_serving* parameters. The *path* parameter can now be a :class:`~pathlib.Path` object. .. coroutinemethod:: loop.connect_accepted_socket(protocol_factory, \ @@ -746,11 +736,11 @@ Creating network servers Returns a ``(transport, protocol)`` pair. - .. versionadded:: 3.7 + .. versionadded:: 3.5.3 - The *ssl_handshake_timeout* parameter. + .. versionchanged:: 3.7 - .. versionadded:: 3.5.3 + Added the *ssl_handshake_timeout* parameter. Transferring files diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index 3496387c178af..7426e8291e142 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -194,7 +194,7 @@ Future Object schedule the callbacks, and return ``True``. .. versionchanged:: 3.9 - Added the ``msg`` parameter. + Added the *msg* parameter. .. method:: exception() diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index e6f26bb959869..d86fbc21351e2 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -36,6 +36,9 @@ Queue the queue is always known and can be returned by calling the :meth:`qsize` method. + .. versionchanged:: 3.10 + Removed the *loop* parameter. + This class is :ref:`not thread safe `. @@ -105,13 +108,6 @@ Queue Raises :exc:`ValueError` if called more times than there were items placed in the queue. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. - Priority Queue ============== diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index ecf178adcd377..cf4a645f5f94e 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -66,16 +66,11 @@ and work with streams: The rest of the arguments are passed directly to :meth:`loop.create_connection`. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* parameter. - - .. deprecated-removed:: 3.8 3.10 + .. versionchanged:: 3.7 + Added the *ssl_handshake_timeout* parameter. - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. coroutinefunction:: start_server(client_connected_cb, host=None, \ @@ -104,16 +99,11 @@ and work with streams: The rest of the arguments are passed directly to :meth:`loop.create_server`. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* and *start_serving* parameters. - - .. deprecated-removed:: 3.8 3.10 + .. versionchanged:: 3.7 + Added the *ssl_handshake_timeout* and *start_serving* parameters. - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. rubric:: Unix Sockets @@ -131,20 +121,12 @@ and work with streams: .. availability:: Unix. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* parameter. - .. versionchanged:: 3.7 - + Added the *ssl_handshake_timeout* parameter. The *path* parameter can now be a :term:`path-like object` - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \ @@ -159,20 +141,12 @@ and work with streams: .. availability:: Unix. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* and *start_serving* parameters. - .. versionchanged:: 3.7 - + Added the *ssl_handshake_timeout* and *start_serving* parameters. The *path* parameter can now be a :term:`path-like object`. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. StreamReader diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index fd1f9c9957879..748b704032554 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -75,12 +75,8 @@ Creating Subprocesses See the documentation of :meth:`loop.subprocess_exec` for other parameters. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \ @@ -106,12 +102,8 @@ Creating Subprocesses escape whitespace and special shell characters in strings that are going to be used to construct shell commands. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. note:: diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index 88e523af0b808..f4063db2ee86e 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -63,11 +63,8 @@ Lock finally: lock.release() - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. coroutinemethod:: acquire() @@ -111,11 +108,8 @@ Event :meth:`clear` method. The :meth:`~Event.wait` method blocks until the flag is set to *true*. The flag is set to *false* initially. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. _asyncio_example_sync_event: @@ -189,11 +183,8 @@ Condition ``None``. In the latter case a new Lock object is created automatically. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. The preferred way to use a Condition is an :keyword:`async with` statement:: @@ -291,11 +282,8 @@ Semaphore internal counter (``1`` by default). If the given value is less than ``0`` a :exc:`ValueError` is raised. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. The preferred way to use a Semaphore is an :keyword:`async with` statement:: @@ -349,12 +337,8 @@ BoundedSemaphore a :exc:`ValueError` in :meth:`~Semaphore.release` if it increases the internal counter above the initial *value*. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. --------- diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index cbc42ac26e48c..731d1559bc09e 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -279,7 +279,7 @@ Creating Tasks .. versionadded:: 3.7 .. versionchanged:: 3.8 - Added the ``name`` parameter. + Added the *name* parameter. Sleeping @@ -319,12 +319,8 @@ Sleeping asyncio.run(display_date()) - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. Running Tasks Concurrently @@ -359,11 +355,8 @@ Running Tasks Concurrently cancellation of one submitted Task/Future to cause other Tasks/Futures to be cancelled. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. _asyncio_example_gather: @@ -416,11 +409,8 @@ Running Tasks Concurrently If the *gather* itself is cancelled, the cancellation is propagated regardless of *return_exceptions*. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. deprecated:: 3.10 Deprecation warning is emitted if no positional arguments are provided @@ -464,11 +454,8 @@ Shielding From Cancellation except CancelledError: res = None - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. deprecated:: 3.10 Deprecation warning is emitted if *aw* is not Future-like object @@ -501,11 +488,8 @@ Timeouts If the wait is cancelled, the future *aw* is also cancelled. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. _asyncio_example_waitfor: @@ -534,11 +518,8 @@ Timeouts for *aw* to be cancelled. Previously, it raised :exc:`TimeoutError` immediately. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. Waiting Primitives @@ -596,11 +577,8 @@ Waiting Primitives ``wait()`` directly is deprecated as it leads to :ref:`confusing behavior `. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. _asyncio_example_wait_coroutine: .. note:: @@ -629,18 +607,14 @@ Waiting Primitives if task in done: # Everything will work as expected now. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. - .. deprecated-removed:: 3.8 3.11 Passing coroutine objects to ``wait()`` directly is deprecated. + .. versionchanged:: 3.10 + Removed the *loop* parameter. + .. function:: as_completed(aws, *, timeout=None) @@ -652,11 +626,8 @@ Waiting Primitives Raises :exc:`TimeoutError` if the timeout occurs before all Futures are done. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. Example:: @@ -664,11 +635,8 @@ Waiting Primitives earliest_result = await coro # ... - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. deprecated:: 3.10 Deprecation warning is emitted if not all awaitable objects in the *aws* @@ -851,10 +819,7 @@ Task Object Added support for the :mod:`contextvars` module. .. versionchanged:: 3.8 - Added the ``name`` parameter. - - .. deprecated-removed:: 3.8 3.10 - The *loop* parameter. + Added the *name* parameter. .. deprecated:: 3.10 Deprecation warning is emitted if *loop* is not specified @@ -876,7 +841,7 @@ Task Object discouraged. .. versionchanged:: 3.9 - Added the ``msg`` parameter. + Added the *msg* parameter. .. _asyncio_example_task_cancel: From webhook-mailer at python.org Fri Feb 18 04:30:54 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 18 Feb 2022 09:30:54 -0000 Subject: [Python-checkins] bpo-46777: Fix incorrect use of directives in asyncio documentation (GH-31388) Message-ID: https://github.com/python/cpython/commit/6eb34772fda2e46ba9a265e038b6e0f303bd790e commit: 6eb34772fda2e46ba9a265e038b6e0f303bd790e branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-18T01:30:36-08:00 summary: bpo-46777: Fix incorrect use of directives in asyncio documentation (GH-31388) (cherry picked from commit 2923d87ca258b9d421e8147b12f0d98295ee3f8e) Co-authored-by: Serhiy Storchaka files: M Doc/library/asyncio-eventloop.rst M Doc/library/asyncio-future.rst M Doc/library/asyncio-queue.rst M Doc/library/asyncio-stream.rst M Doc/library/asyncio-subprocess.rst M Doc/library/asyncio-sync.rst M Doc/library/asyncio-task.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index b1700c996928b..300092d71b1cb 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -343,7 +343,7 @@ Creating Futures and Tasks the name of the task using :meth:`Task.set_name`. .. versionchanged:: 3.8 - Added the ``name`` parameter. + Added the *name* parameter. .. method:: loop.set_task_factory(factory) @@ -456,7 +456,20 @@ Opening network connections to wait for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). - .. versionadded:: 3.8 + .. versionchanged:: 3.5 + + Added support for SSL/TLS in :class:`ProactorEventLoop`. + + .. versionchanged:: 3.6 + + The socket option :py:data:`~socket.TCP_NODELAY` is set by default + for all TCP connections. + + .. versionchanged:: 3.7 + + Added the *ssl_handshake_timeout* parameter. + + .. versionchanged:: 3.8 Added the *happy_eyeballs_delay* and *interleave* parameters. @@ -471,19 +484,6 @@ Opening network connections For more information: https://tools.ietf.org/html/rfc6555 - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* parameter. - - .. versionchanged:: 3.6 - - The socket option :py:data:`~socket.TCP_NODELAY` is set by default - for all TCP connections. - - .. versionchanged:: 3.5 - - Added support for SSL/TLS in :class:`ProactorEventLoop`. - .. seealso:: The :func:`open_connection` function is a high-level alternative @@ -588,12 +588,8 @@ Opening network connections .. availability:: Unix. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* parameter. - .. versionchanged:: 3.7 - + Added the *ssl_handshake_timeout* parameter. The *path* parameter can now be a :term:`path-like object`. @@ -672,15 +668,6 @@ Creating network servers :meth:`Server.serve_forever` to make the server to start accepting connections. - .. versionadded:: 3.7 - - Added *ssl_handshake_timeout* and *start_serving* parameters. - - .. versionchanged:: 3.6 - - The socket option :py:data:`~socket.TCP_NODELAY` is set by default - for all TCP connections. - .. versionchanged:: 3.5 Added support for SSL/TLS in :class:`ProactorEventLoop`. @@ -689,6 +676,12 @@ Creating network servers The *host* parameter can be a sequence of strings. + .. versionchanged:: 3.6 + + Added *ssl_handshake_timeout* and *start_serving* parameters. + The socket option :py:data:`~socket.TCP_NODELAY` is set by default + for all TCP connections. + .. seealso:: The :func:`start_server` function is a higher-level alternative API @@ -713,12 +706,9 @@ Creating network servers .. availability:: Unix. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* and *start_serving* parameters. - .. versionchanged:: 3.7 + Added the *ssl_handshake_timeout* and *start_serving* parameters. The *path* parameter can now be a :class:`~pathlib.Path` object. .. coroutinemethod:: loop.connect_accepted_socket(protocol_factory, \ @@ -746,11 +736,11 @@ Creating network servers Returns a ``(transport, protocol)`` pair. - .. versionadded:: 3.7 + .. versionadded:: 3.5.3 - The *ssl_handshake_timeout* parameter. + .. versionchanged:: 3.7 - .. versionadded:: 3.5.3 + Added the *ssl_handshake_timeout* parameter. Transferring files diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst index 3496387c178af..7426e8291e142 100644 --- a/Doc/library/asyncio-future.rst +++ b/Doc/library/asyncio-future.rst @@ -194,7 +194,7 @@ Future Object schedule the callbacks, and return ``True``. .. versionchanged:: 3.9 - Added the ``msg`` parameter. + Added the *msg* parameter. .. method:: exception() diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst index e6f26bb959869..d86fbc21351e2 100644 --- a/Doc/library/asyncio-queue.rst +++ b/Doc/library/asyncio-queue.rst @@ -36,6 +36,9 @@ Queue the queue is always known and can be returned by calling the :meth:`qsize` method. + .. versionchanged:: 3.10 + Removed the *loop* parameter. + This class is :ref:`not thread safe `. @@ -105,13 +108,6 @@ Queue Raises :exc:`ValueError` if called more times than there were items placed in the queue. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. - Priority Queue ============== diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 95a8e4649beed..269c3dc80c471 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -66,16 +66,11 @@ and work with streams: The rest of the arguments are passed directly to :meth:`loop.create_connection`. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* parameter. - - .. deprecated-removed:: 3.8 3.10 + .. versionchanged:: 3.7 + Added the *ssl_handshake_timeout* parameter. - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. coroutinefunction:: start_server(client_connected_cb, host=None, \ @@ -104,16 +99,11 @@ and work with streams: The rest of the arguments are passed directly to :meth:`loop.create_server`. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* and *start_serving* parameters. - - .. deprecated-removed:: 3.8 3.10 + .. versionchanged:: 3.7 + Added the *ssl_handshake_timeout* and *start_serving* parameters. - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. rubric:: Unix Sockets @@ -131,20 +121,12 @@ and work with streams: .. availability:: Unix. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* parameter. - .. versionchanged:: 3.7 - + Added the *ssl_handshake_timeout* parameter. The *path* parameter can now be a :term:`path-like object` - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \ @@ -159,20 +141,12 @@ and work with streams: .. availability:: Unix. - .. versionadded:: 3.7 - - The *ssl_handshake_timeout* and *start_serving* parameters. - .. versionchanged:: 3.7 - + Added the *ssl_handshake_timeout* and *start_serving* parameters. The *path* parameter can now be a :term:`path-like object`. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. StreamReader diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index fd1f9c9957879..748b704032554 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -75,12 +75,8 @@ Creating Subprocesses See the documentation of :meth:`loop.subprocess_exec` for other parameters. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \ @@ -106,12 +102,8 @@ Creating Subprocesses escape whitespace and special shell characters in strings that are going to be used to construct shell commands. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. note:: diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index 88e523af0b808..f4063db2ee86e 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -63,11 +63,8 @@ Lock finally: lock.release() - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. coroutinemethod:: acquire() @@ -111,11 +108,8 @@ Event :meth:`clear` method. The :meth:`~Event.wait` method blocks until the flag is set to *true*. The flag is set to *false* initially. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. _asyncio_example_sync_event: @@ -189,11 +183,8 @@ Condition ``None``. In the latter case a new Lock object is created automatically. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. The preferred way to use a Condition is an :keyword:`async with` statement:: @@ -291,11 +282,8 @@ Semaphore internal counter (``1`` by default). If the given value is less than ``0`` a :exc:`ValueError` is raised. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. The preferred way to use a Semaphore is an :keyword:`async with` statement:: @@ -349,12 +337,8 @@ BoundedSemaphore a :exc:`ValueError` in :meth:`~Semaphore.release` if it increases the internal counter above the initial *value*. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This class has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. --------- diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 7f4cc70386157..efd4d1bc628ad 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -282,7 +282,7 @@ Creating Tasks .. versionadded:: 3.7 .. versionchanged:: 3.8 - Added the ``name`` parameter. + Added the *name* parameter. Sleeping @@ -328,12 +328,8 @@ Sleeping asyncio.run(display_date()) - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. Running Tasks Concurrently @@ -368,11 +364,8 @@ Running Tasks Concurrently cancellation of one submitted Task/Future to cause other Tasks/Futures to be cancelled. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. _asyncio_example_gather: @@ -425,11 +418,8 @@ Running Tasks Concurrently If the *gather* itself is cancelled, the cancellation is propagated regardless of *return_exceptions*. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. deprecated:: 3.10 Deprecation warning is emitted if no positional arguments are provided @@ -473,11 +463,8 @@ Shielding From Cancellation except CancelledError: res = None - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. deprecated:: 3.10 Deprecation warning is emitted if *aw* is not Future-like object @@ -510,11 +497,8 @@ Timeouts If the wait is cancelled, the future *aw* is also cancelled. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. _asyncio_example_waitfor: @@ -543,11 +527,8 @@ Timeouts for *aw* to be cancelled. Previously, it raised :exc:`asyncio.TimeoutError` immediately. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. Waiting Primitives @@ -605,11 +586,8 @@ Waiting Primitives ``wait()`` directly is deprecated as it leads to :ref:`confusing behavior `. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. _asyncio_example_wait_coroutine: .. note:: @@ -638,18 +616,14 @@ Waiting Primitives if task in done: # Everything will work as expected now. - .. deprecated-removed:: 3.8 3.10 - - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. - .. deprecated-removed:: 3.8 3.11 Passing coroutine objects to ``wait()`` directly is deprecated. + .. versionchanged:: 3.10 + Removed the *loop* parameter. + .. function:: as_completed(aws, *, timeout=None) @@ -661,11 +635,8 @@ Waiting Primitives Raises :exc:`asyncio.TimeoutError` if the timeout occurs before all Futures are done. - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. Example:: @@ -673,11 +644,8 @@ Waiting Primitives earliest_result = await coro # ... - .. deprecated-removed:: 3.8 3.10 - The ``loop`` parameter. This function has been implicitly getting the - current running loop since 3.7. See - :ref:`What's New in 3.10's Removed section ` - for more information. + .. versionchanged:: 3.10 + Removed the *loop* parameter. .. deprecated:: 3.10 Deprecation warning is emitted if not all awaitable objects in the *aws* @@ -860,10 +828,7 @@ Task Object Added support for the :mod:`contextvars` module. .. versionchanged:: 3.8 - Added the ``name`` parameter. - - .. deprecated-removed:: 3.8 3.10 - The *loop* parameter. + Added the *name* parameter. .. deprecated:: 3.10 Deprecation warning is emitted if *loop* is not specified @@ -885,7 +850,7 @@ Task Object discouraged. .. versionchanged:: 3.9 - Added the ``msg`` parameter. + Added the *msg* parameter. .. _asyncio_example_task_cancel: From webhook-mailer at python.org Fri Feb 18 04:56:32 2022 From: webhook-mailer at python.org (markshannon) Date: Fri, 18 Feb 2022 09:56:32 -0000 Subject: [Python-checkins] bpo-46724: Fix dis support for overflow args (GH-31285) Message-ID: https://github.com/python/cpython/commit/c3ce7781e3afe6f2dec5eef8e87fd5a664519ae9 commit: c3ce7781e3afe6f2dec5eef8e87fd5a664519ae9 branch: main author: Saul Shanabrook committer: markshannon date: 2022-02-18T09:56:23Z summary: bpo-46724: Fix dis support for overflow args (GH-31285) files: A Misc/NEWS.d/next/Library/2022-02-11-20-41-17.bpo-46724.eU52_N.rst M Lib/dis.py M Lib/test/test_dis.py diff --git a/Lib/dis.py b/Lib/dis.py index 2462a8434e895..dc3ec169abecf 100644 --- a/Lib/dis.py +++ b/Lib/dis.py @@ -515,6 +515,12 @@ def _disassemble_str(source, **kwargs): disco = disassemble # XXX For backwards compatibility + +# Rely on C `int` being 32 bits for oparg +_INT_BITS = 32 +# Value for c int when it overflows +_INT_OVERFLOW = 2 ** (_INT_BITS - 1) + def _unpack_opargs(code): extended_arg = 0 for i in range(0, len(code), 2): @@ -522,6 +528,11 @@ def _unpack_opargs(code): if op >= HAVE_ARGUMENT: arg = code[i+1] | extended_arg extended_arg = (arg << 8) if op == EXTENDED_ARG else 0 + # The oparg is stored as a signed integer + # If the value exceeds its upper limit, it will overflow and wrap + # to a negative integer + if extended_arg >= _INT_OVERFLOW: + extended_arg -= 2 * _INT_OVERFLOW else: arg = None extended_arg = 0 diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index 4feeb8c5be8a2..488b8dffdedbf 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -1,14 +1,17 @@ # Minimal tests for dis module -from test.support import captured_stdout, requires_debug_ranges -from test.support.bytecode_helper import BytecodeTestCase -import unittest -import sys +import contextlib import dis import io import re +import sys import types -import contextlib +import unittest +from test.support import captured_stdout, requires_debug_ranges +from test.support.bytecode_helper import BytecodeTestCase + +import opcode + def get_tb(): def _error(): @@ -219,6 +222,22 @@ def bug42562(): RETURN_VALUE """ +# [255, 255, 255, 252] is -4 in a 4 byte signed integer +bug46724 = bytes([ + opcode.EXTENDED_ARG, 255, + opcode.EXTENDED_ARG, 255, + opcode.EXTENDED_ARG, 255, + opcode.opmap['JUMP_FORWARD'], 252, +]) + + +dis_bug46724 = """\ + >> EXTENDED_ARG 255 + EXTENDED_ARG 65535 + EXTENDED_ARG 16777215 + JUMP_FORWARD -4 (to 0) +""" + _BIG_LINENO_FORMAT = """\ 1 RESUME 0 @@ -688,6 +707,10 @@ def test_bug_45757(self): # Extended arg followed by NOP self.do_disassembly_test(code_bug_45757, dis_bug_45757) + def test_bug_46724(self): + # Test that negative operargs are handled properly + self.do_disassembly_test(bug46724, dis_bug46724) + def test_big_linenos(self): def func(count): namespace = {} diff --git a/Misc/NEWS.d/next/Library/2022-02-11-20-41-17.bpo-46724.eU52_N.rst b/Misc/NEWS.d/next/Library/2022-02-11-20-41-17.bpo-46724.eU52_N.rst new file mode 100644 index 0000000000000..9ac8c17deb7a2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-11-20-41-17.bpo-46724.eU52_N.rst @@ -0,0 +1 @@ +Fix :mod:`dis` behavior on negative jump offsets. From webhook-mailer at python.org Fri Feb 18 07:56:31 2022 From: webhook-mailer at python.org (markshannon) Date: Fri, 18 Feb 2022 12:56:31 -0000 Subject: [Python-checkins] bpo-46709: check eval breaker in specialized `CALL` opcodes (GH-31404) Message-ID: https://github.com/python/cpython/commit/e2c28616ce6c3cdb1013c415125220a0b86b86a1 commit: e2c28616ce6c3cdb1013c415125220a0b86b86a1 branch: main author: Nikita Sobolev committer: markshannon date: 2022-02-18T12:56:23Z summary: bpo-46709: check eval breaker in specialized `CALL` opcodes (GH-31404) files: M Lib/unittest/test/test_break.py M Python/ceval.c diff --git a/Lib/unittest/test/test_break.py b/Lib/unittest/test/test_break.py index eebd2b610ce11..33cbdd2661c17 100644 --- a/Lib/unittest/test/test_break.py +++ b/Lib/unittest/test/test_break.py @@ -4,14 +4,18 @@ import sys import signal import weakref - import unittest +from test import support + @unittest.skipUnless(hasattr(os, 'kill'), "Test requires os.kill") @unittest.skipIf(sys.platform =="win32", "Test cannot run on Windows") class TestBreak(unittest.TestCase): int_handler = None + # This number was smart-guessed, previously tests were failing + # after 7th run. So, we take `x * 2 + 1` to be sure. + default_repeats = 15 def setUp(self): self._default_handler = signal.getsignal(signal.SIGINT) @@ -24,6 +28,27 @@ def tearDown(self): unittest.signals._interrupt_handler = None + def withRepeats(self, test_function, repeats=None): + if not support.check_impl_detail(cpython=True): + # Override repeats count on non-cpython to execute only once. + # Because this test only makes sense to be repeated on CPython. + repeats = 1 + elif repeats is None: + repeats = self.default_repeats + + for repeat in range(repeats): + with self.subTest(repeat=repeat): + # We don't run `setUp` for the very first repeat + # and we don't run `tearDown` for the very last one, + # because they are handled by the test class itself. + if repeat != 0: + self.setUp() + try: + test_function() + finally: + if repeat != repeats - 1: + self.tearDown() + def testInstallHandler(self): default_handler = signal.getsignal(signal.SIGINT) unittest.installHandler() @@ -48,35 +73,34 @@ def testRegisterResult(self): unittest.removeResult(result) def testInterruptCaught(self): - default_handler = signal.getsignal(signal.SIGINT) - - result = unittest.TestResult() - unittest.installHandler() - unittest.registerResult(result) - - self.assertNotEqual(signal.getsignal(signal.SIGINT), default_handler) - def test(result): pid = os.getpid() os.kill(pid, signal.SIGINT) result.breakCaught = True self.assertTrue(result.shouldStop) - try: - test(result) - except KeyboardInterrupt: - self.fail("KeyboardInterrupt not handled") - self.assertTrue(result.breakCaught) + def test_function(): + result = unittest.TestResult() + unittest.installHandler() + unittest.registerResult(result) + self.assertNotEqual( + signal.getsignal(signal.SIGINT), + self._default_handler, + ) + + try: + test(result) + except KeyboardInterrupt: + self.fail("KeyboardInterrupt not handled") + self.assertTrue(result.breakCaught) + self.withRepeats(test_function) def testSecondInterrupt(self): # Can't use skipIf decorator because the signal handler may have # been changed after defining this method. if signal.getsignal(signal.SIGINT) == signal.SIG_IGN: self.skipTest("test requires SIGINT to not be ignored") - result = unittest.TestResult() - unittest.installHandler() - unittest.registerResult(result) def test(result): pid = os.getpid() @@ -86,40 +110,40 @@ def test(result): os.kill(pid, signal.SIGINT) self.fail("Second KeyboardInterrupt not raised") - try: - test(result) - except KeyboardInterrupt: - pass - else: - self.fail("Second KeyboardInterrupt not raised") - self.assertTrue(result.breakCaught) + def test_function(): + result = unittest.TestResult() + unittest.installHandler() + unittest.registerResult(result) + with self.assertRaises(KeyboardInterrupt): + test(result) + self.assertTrue(result.breakCaught) + self.withRepeats(test_function) - def testTwoResults(self): - unittest.installHandler() - result = unittest.TestResult() - unittest.registerResult(result) - new_handler = signal.getsignal(signal.SIGINT) + def testTwoResults(self): + def test_function(): + unittest.installHandler() - result2 = unittest.TestResult() - unittest.registerResult(result2) - self.assertEqual(signal.getsignal(signal.SIGINT), new_handler) + result = unittest.TestResult() + unittest.registerResult(result) + new_handler = signal.getsignal(signal.SIGINT) - result3 = unittest.TestResult() + result2 = unittest.TestResult() + unittest.registerResult(result2) + self.assertEqual(signal.getsignal(signal.SIGINT), new_handler) - def test(result): - pid = os.getpid() - os.kill(pid, signal.SIGINT) + result3 = unittest.TestResult() - try: - test(result) - except KeyboardInterrupt: - self.fail("KeyboardInterrupt not handled") + try: + os.kill(os.getpid(), signal.SIGINT) + except KeyboardInterrupt: + self.fail("KeyboardInterrupt not handled") - self.assertTrue(result.shouldStop) - self.assertTrue(result2.shouldStop) - self.assertFalse(result3.shouldStop) + self.assertTrue(result.shouldStop) + self.assertTrue(result2.shouldStop) + self.assertFalse(result3.shouldStop) + self.withRepeats(test_function) def testHandlerReplacedButCalled(self): @@ -127,23 +151,25 @@ def testHandlerReplacedButCalled(self): # been changed after defining this method. if signal.getsignal(signal.SIGINT) == signal.SIG_IGN: self.skipTest("test requires SIGINT to not be ignored") - # If our handler has been replaced (is no longer installed) but is - # called by the *new* handler, then it isn't safe to delay the - # SIGINT and we should immediately delegate to the default handler - unittest.installHandler() - - handler = signal.getsignal(signal.SIGINT) - def new_handler(frame, signum): - handler(frame, signum) - signal.signal(signal.SIGINT, new_handler) - try: - pid = os.getpid() - os.kill(pid, signal.SIGINT) - except KeyboardInterrupt: - pass - else: - self.fail("replaced but delegated handler doesn't raise interrupt") + def test_function(): + # If our handler has been replaced (is no longer installed) but is + # called by the *new* handler, then it isn't safe to delay the + # SIGINT and we should immediately delegate to the default handler + unittest.installHandler() + + handler = signal.getsignal(signal.SIGINT) + def new_handler(frame, signum): + handler(frame, signum) + signal.signal(signal.SIGINT, new_handler) + + try: + os.kill(os.getpid(), signal.SIGINT) + except KeyboardInterrupt: + pass + else: + self.fail("replaced but delegated handler doesn't raise interrupt") + self.withRepeats(test_function) def testRunner(self): # Creating a TextTestRunner with the appropriate argument should diff --git a/Python/ceval.c b/Python/ceval.c index b900de53c8c92..5a6de5bce497f 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4742,6 +4742,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (res == NULL) { goto error; } + CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -4761,6 +4762,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (res == NULL) { goto error; } + CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -4785,6 +4787,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (res == NULL) { goto error; } + CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -4816,6 +4819,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (res == NULL) { goto error; } + CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -4854,6 +4858,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr */ goto error; } + CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -4896,6 +4901,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (res == NULL) { goto error; } + CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -5013,6 +5019,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (res == NULL) { goto error; } + CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -5040,6 +5047,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (res == NULL) { goto error; } + CHECK_EVAL_BREAKER(); DISPATCH(); } @@ -5067,6 +5075,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (res == NULL) { goto error; } + CHECK_EVAL_BREAKER(); DISPATCH(); } From webhook-mailer at python.org Fri Feb 18 12:19:22 2022 From: webhook-mailer at python.org (markshannon) Date: Fri, 18 Feb 2022 17:19:22 -0000 Subject: [Python-checkins] bpo-46329: Change calling sequence (again) (GH-31373) Message-ID: https://github.com/python/cpython/commit/cf345e945f48f54785799390c2e92c5310847bd4 commit: cf345e945f48f54785799390c2e92c5310847bd4 branch: main author: Mark Shannon committer: markshannon date: 2022-02-18T17:19:08Z summary: bpo-46329: Change calling sequence (again) (GH-31373) * Change calling sequence: Add PUSH_NULL. Merge PRECALL_FUNCTION and PRECALL_METHOD into PRECALL. files: A Misc/NEWS.d/next/Core and Builtins/2022-02-16-13-15-16.bpo-46329.8aIuz9.rst M Doc/whatsnew/3.11.rst M Include/opcode.h M Lib/importlib/_bootstrap_external.py M Lib/opcode.py M Lib/test/test_compile.py M Lib/test/test_dis.py M Lib/test/test_sys.py M Programs/test_frozenmain.h M Python/ceval.c M Python/compile.c M Python/opcode_targets.h M Python/specialize.c diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 4dd778951f4c6..98fa7a7da466d 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -410,7 +410,7 @@ CPython bytecode changes * Replaced the three call instructions: :opcode:`CALL_FUNCTION`, :opcode:`CALL_FUNCTION_KW` and :opcode:`CALL_METHOD` with - :opcode:`PRECALL_FUNCTION`, :opcode:`PRECALL_METHOD`, :opcode:`CALL`, + :opcode:`PUSH_NULL`, :opcode:`PRECALL`, :opcode:`CALL`, and :opcode:`KW_NAMES`. This decouples the argument shifting for methods from the handling of keyword arguments and allows better specialization of calls. diff --git a/Include/opcode.h b/Include/opcode.h index df45e7b5e5a30..df93a93fbbd98 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -8,6 +8,7 @@ extern "C" { /* Instruction opcodes for compiled code */ #define POP_TOP 1 +#define PUSH_NULL 2 #define NOP 9 #define UNARY_POSITIVE 10 #define UNARY_NEGATIVE 11 @@ -109,75 +110,74 @@ extern "C" { #define SET_UPDATE 163 #define DICT_MERGE 164 #define DICT_UPDATE 165 -#define PRECALL_FUNCTION 167 -#define PRECALL_METHOD 168 +#define PRECALL 166 #define CALL 171 #define KW_NAMES 172 -#define BINARY_OP_ADAPTIVE 2 -#define BINARY_OP_ADD_INT 3 -#define BINARY_OP_ADD_FLOAT 4 -#define BINARY_OP_ADD_UNICODE 5 -#define BINARY_OP_INPLACE_ADD_UNICODE 6 -#define BINARY_OP_MULTIPLY_INT 7 -#define BINARY_OP_MULTIPLY_FLOAT 8 -#define BINARY_OP_SUBTRACT_INT 13 -#define BINARY_OP_SUBTRACT_FLOAT 14 -#define COMPARE_OP_ADAPTIVE 16 -#define COMPARE_OP_FLOAT_JUMP 17 -#define COMPARE_OP_INT_JUMP 18 -#define COMPARE_OP_STR_JUMP 19 -#define BINARY_SUBSCR_ADAPTIVE 20 -#define BINARY_SUBSCR_GETITEM 21 -#define BINARY_SUBSCR_LIST_INT 22 -#define BINARY_SUBSCR_TUPLE_INT 23 -#define BINARY_SUBSCR_DICT 24 -#define STORE_SUBSCR_ADAPTIVE 26 -#define STORE_SUBSCR_LIST_INT 27 -#define STORE_SUBSCR_DICT 28 -#define CALL_ADAPTIVE 29 -#define CALL_BUILTIN_CLASS 34 -#define CALL_NO_KW_BUILTIN_O 36 -#define CALL_NO_KW_BUILTIN_FAST 37 -#define CALL_BUILTIN_FAST_WITH_KEYWORDS 38 -#define CALL_NO_KW_LEN 39 -#define CALL_NO_KW_ISINSTANCE 40 -#define CALL_PY_EXACT_ARGS 41 -#define CALL_PY_WITH_DEFAULTS 42 -#define CALL_NO_KW_LIST_APPEND 43 -#define CALL_NO_KW_METHOD_DESCRIPTOR_O 44 -#define CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 45 -#define CALL_NO_KW_STR_1 46 -#define CALL_NO_KW_TUPLE_1 47 -#define CALL_NO_KW_TYPE_1 48 -#define CALL_NO_KW_METHOD_DESCRIPTOR_FAST 55 -#define JUMP_ABSOLUTE_QUICK 56 -#define LOAD_ATTR_ADAPTIVE 57 -#define LOAD_ATTR_INSTANCE_VALUE 58 -#define LOAD_ATTR_WITH_HINT 59 -#define LOAD_ATTR_SLOT 62 -#define LOAD_ATTR_MODULE 63 -#define LOAD_GLOBAL_ADAPTIVE 64 -#define LOAD_GLOBAL_MODULE 65 -#define LOAD_GLOBAL_BUILTIN 66 -#define LOAD_METHOD_ADAPTIVE 67 -#define LOAD_METHOD_CACHED 72 -#define LOAD_METHOD_CLASS 76 -#define LOAD_METHOD_MODULE 77 -#define LOAD_METHOD_NO_DICT 78 -#define RESUME_QUICK 79 -#define STORE_ATTR_ADAPTIVE 80 -#define STORE_ATTR_INSTANCE_VALUE 81 -#define STORE_ATTR_SLOT 131 -#define STORE_ATTR_WITH_HINT 140 -#define UNPACK_SEQUENCE_ADAPTIVE 141 -#define UNPACK_SEQUENCE_LIST 143 -#define UNPACK_SEQUENCE_TUPLE 150 -#define UNPACK_SEQUENCE_TWO_TUPLE 153 -#define LOAD_FAST__LOAD_FAST 154 -#define STORE_FAST__LOAD_FAST 158 -#define LOAD_FAST__LOAD_CONST 159 -#define LOAD_CONST__LOAD_FAST 161 -#define STORE_FAST__STORE_FAST 166 +#define BINARY_OP_ADAPTIVE 3 +#define BINARY_OP_ADD_INT 4 +#define BINARY_OP_ADD_FLOAT 5 +#define BINARY_OP_ADD_UNICODE 6 +#define BINARY_OP_INPLACE_ADD_UNICODE 7 +#define BINARY_OP_MULTIPLY_INT 8 +#define BINARY_OP_MULTIPLY_FLOAT 13 +#define BINARY_OP_SUBTRACT_INT 14 +#define BINARY_OP_SUBTRACT_FLOAT 16 +#define COMPARE_OP_ADAPTIVE 17 +#define COMPARE_OP_FLOAT_JUMP 18 +#define COMPARE_OP_INT_JUMP 19 +#define COMPARE_OP_STR_JUMP 20 +#define BINARY_SUBSCR_ADAPTIVE 21 +#define BINARY_SUBSCR_GETITEM 22 +#define BINARY_SUBSCR_LIST_INT 23 +#define BINARY_SUBSCR_TUPLE_INT 24 +#define BINARY_SUBSCR_DICT 26 +#define STORE_SUBSCR_ADAPTIVE 27 +#define STORE_SUBSCR_LIST_INT 28 +#define STORE_SUBSCR_DICT 29 +#define CALL_ADAPTIVE 34 +#define CALL_BUILTIN_CLASS 36 +#define CALL_NO_KW_BUILTIN_O 37 +#define CALL_NO_KW_BUILTIN_FAST 38 +#define CALL_BUILTIN_FAST_WITH_KEYWORDS 39 +#define CALL_NO_KW_LEN 40 +#define CALL_NO_KW_ISINSTANCE 41 +#define CALL_PY_EXACT_ARGS 42 +#define CALL_PY_WITH_DEFAULTS 43 +#define CALL_NO_KW_LIST_APPEND 44 +#define CALL_NO_KW_METHOD_DESCRIPTOR_O 45 +#define CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 46 +#define CALL_NO_KW_STR_1 47 +#define CALL_NO_KW_TUPLE_1 48 +#define CALL_NO_KW_TYPE_1 55 +#define CALL_NO_KW_METHOD_DESCRIPTOR_FAST 56 +#define JUMP_ABSOLUTE_QUICK 57 +#define LOAD_ATTR_ADAPTIVE 58 +#define LOAD_ATTR_INSTANCE_VALUE 59 +#define LOAD_ATTR_WITH_HINT 62 +#define LOAD_ATTR_SLOT 63 +#define LOAD_ATTR_MODULE 64 +#define LOAD_GLOBAL_ADAPTIVE 65 +#define LOAD_GLOBAL_MODULE 66 +#define LOAD_GLOBAL_BUILTIN 67 +#define LOAD_METHOD_ADAPTIVE 72 +#define LOAD_METHOD_CACHED 76 +#define LOAD_METHOD_CLASS 77 +#define LOAD_METHOD_MODULE 78 +#define LOAD_METHOD_NO_DICT 79 +#define RESUME_QUICK 80 +#define STORE_ATTR_ADAPTIVE 81 +#define STORE_ATTR_INSTANCE_VALUE 131 +#define STORE_ATTR_SLOT 140 +#define STORE_ATTR_WITH_HINT 141 +#define UNPACK_SEQUENCE_ADAPTIVE 143 +#define UNPACK_SEQUENCE_LIST 150 +#define UNPACK_SEQUENCE_TUPLE 153 +#define UNPACK_SEQUENCE_TWO_TUPLE 154 +#define LOAD_FAST__LOAD_FAST 158 +#define STORE_FAST__LOAD_FAST 159 +#define LOAD_FAST__LOAD_CONST 161 +#define LOAD_CONST__LOAD_FAST 167 +#define STORE_FAST__STORE_FAST 168 #define DO_TRACING 255 #ifdef NEED_OPCODE_JUMP_TABLES static uint32_t _PyOpcode_RelativeJump[8] = { diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py index 7b9803d27fa61..85f50eb958e31 100644 --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -385,6 +385,7 @@ def _write_atomic(path, data, mode=0o666): # Python 3.11a5 3477 (Replace DUP_TOP/DUP_TOP_TWO with COPY and # ROT_TWO/ROT_THREE/ROT_FOUR/ROT_N with SWAP) # Python 3.11a5 3478 (New CALL opcodes) +# Python 3.11a5 3479 (Add PUSH_NULL opcode) # Python 3.12 will start with magic number 3500 @@ -402,7 +403,7 @@ def _write_atomic(path, data, mode=0o666): # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # in PC/launcher.c must also be updated. -MAGIC_NUMBER = (3478).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3479).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Lib/opcode.py b/Lib/opcode.py index 8237aa7fa3d11..95792459c377a 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -55,6 +55,7 @@ def jabs_op(name, op): # Blank lines correspond to available opcodes def_op('POP_TOP', 1) +def_op('PUSH_NULL', 2) def_op('NOP', 9) def_op('UNARY_POSITIVE', 10) @@ -187,9 +188,7 @@ def jabs_op(name, op): def_op('SET_UPDATE', 163) def_op('DICT_MERGE', 164) def_op('DICT_UPDATE', 165) - -def_op('PRECALL_FUNCTION', 167) -def_op('PRECALL_METHOD', 168) +def_op('PRECALL', 166) def_op('CALL', 171) def_op('KW_NAMES', 172) diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py index 7ebe837fce1b5..2e1cb5a67197c 100644 --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -838,7 +838,7 @@ def foo(x): instructions = [opcode.opname for opcode in opcodes] self.assertNotIn('LOAD_METHOD', instructions) self.assertIn('LOAD_ATTR', instructions) - self.assertIn('PRECALL_FUNCTION', instructions) + self.assertIn('PRECALL', instructions) def test_lineno_procedure_call(self): def call(): diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index 488b8dffdedbf..0ab2556da2f4f 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -105,9 +105,10 @@ def _f(a): dis_f = """\ %3d RESUME 0 -%3d LOAD_GLOBAL 0 (print) +%3d PUSH_NULL + LOAD_GLOBAL 0 (print) LOAD_FAST 0 (a) - PRECALL_FUNCTION 1 + PRECALL 1 CALL 0 POP_TOP @@ -120,9 +121,10 @@ def _f(a): dis_f_co_code = """\ RESUME 0 + PUSH_NULL LOAD_GLOBAL 0 LOAD_FAST 0 - PRECALL_FUNCTION 1 + PRECALL 1 CALL 0 POP_TOP LOAD_CONST 1 @@ -138,18 +140,19 @@ def bug708901(): dis_bug708901 = """\ %3d RESUME 0 -%3d LOAD_GLOBAL 0 (range) +%3d PUSH_NULL + LOAD_GLOBAL 0 (range) LOAD_CONST 1 (1) %3d LOAD_CONST 2 (10) -%3d PRECALL_FUNCTION 2 +%3d PRECALL 2 CALL 0 GET_ITER - >> FOR_ITER 2 (to 20) + >> FOR_ITER 2 (to 22) STORE_FAST 0 (res) -%3d JUMP_ABSOLUTE 7 (to 14) +%3d JUMP_ABSOLUTE 8 (to 16) %3d >> LOAD_CONST 0 (None) RETURN_VALUE @@ -174,13 +177,13 @@ def bug1333982(x=[]): MAKE_FUNCTION 0 LOAD_FAST 0 (x) GET_ITER - PRECALL_FUNCTION 1 + PRECALL 0 CALL 0 %3d LOAD_CONST 3 (1) %3d BINARY_OP 0 (+) - PRECALL_FUNCTION 1 + PRECALL 0 CALL 0 RAISE_VARARGS 1 """ % (bug1333982.__code__.co_firstlineno, @@ -312,9 +315,10 @@ def bug42562(): LOAD_CONST 1 ('x') STORE_SUBSCR - 3 LOAD_NAME 3 (fun) + 3 PUSH_NULL + LOAD_NAME 3 (fun) LOAD_CONST 0 (1) - PRECALL_FUNCTION 1 + PRECALL 1 CALL 0 LOAD_NAME 2 (__annotations__) LOAD_CONST 2 ('y') @@ -322,9 +326,10 @@ def bug42562(): 4 LOAD_CONST 0 (1) LOAD_NAME 4 (lst) + PUSH_NULL LOAD_NAME 3 (fun) LOAD_CONST 3 (0) - PRECALL_FUNCTION 1 + PRECALL 1 CALL 0 STORE_SUBSCR LOAD_NAME 1 (int) @@ -444,14 +449,16 @@ def _tryfinallyconst(b): %3d LOAD_FAST 0 (a) -%3d LOAD_FAST 1 (b) - PRECALL_FUNCTION 0 +%3d PUSH_NULL + LOAD_FAST 1 (b) + PRECALL 0 CALL 0 POP_TOP RETURN_VALUE >> PUSH_EXC_INFO + PUSH_NULL LOAD_FAST 1 (b) - PRECALL_FUNCTION 0 + PRECALL 0 CALL 0 POP_TOP RERAISE 0 @@ -472,15 +479,17 @@ def _tryfinallyconst(b): %3d NOP -%3d LOAD_FAST 0 (b) - PRECALL_FUNCTION 0 +%3d PUSH_NULL + LOAD_FAST 0 (b) + PRECALL 0 CALL 0 POP_TOP LOAD_CONST 1 (1) RETURN_VALUE PUSH_EXC_INFO + PUSH_NULL LOAD_FAST 0 (b) - PRECALL_FUNCTION 0 + PRECALL 0 CALL 0 POP_TOP RERAISE 0 @@ -543,7 +552,7 @@ def foo(x): MAKE_FUNCTION 8 (closure) LOAD_DEREF 1 (y) GET_ITER - PRECALL_FUNCTION 1 + PRECALL 0 CALL 0 RETURN_VALUE """ % (dis_nested_0, @@ -882,7 +891,7 @@ def get_disassembly(self, func, lasti=-1, wrapper=True, **kwargs): Positional-only arguments: 0 Kw-only arguments: 0 Number of locals: 1 -Stack size: 3 +Stack size: \\d+ Flags: OPTIMIZED, NEWLOCALS Constants: {code_info_consts} @@ -906,7 +915,7 @@ def f(c=c): Positional-only arguments: 2 Kw-only arguments: 3 Number of locals: 10 -Stack size: 9 +Stack size: \\d+ Flags: OPTIMIZED, NEWLOCALS, VARARGS, VARKEYWORDS, GENERATOR Constants: 0: None @@ -939,7 +948,7 @@ def f(c=c): Positional-only arguments: 0 Kw-only arguments: 0 Number of locals: 1 -Stack size: 10 +Stack size: \\d+ Flags: OPTIMIZED, NEWLOCALS, NESTED Constants: 0: None @@ -962,7 +971,7 @@ def f(c=c): Positional-only arguments: 0 Kw-only arguments: 0 Number of locals: 0 -Stack size: 2 +Stack size: \\d+ Flags: 0x0 Constants: 0: 1 @@ -976,7 +985,7 @@ def f(c=c): Positional-only arguments: 0 Kw-only arguments: 0 Number of locals: 0 -Stack size: 2 +Stack size: \\d+ Flags: 0x0 Constants: 0: 1 @@ -991,7 +1000,7 @@ def f(c=c): Positional-only arguments: 0 Kw-only arguments: 0 Number of locals: 0 -Stack size: 2 +Stack size: \\d+ Flags: 0x0 Constants: 0: 0 @@ -1012,7 +1021,7 @@ async def async_def(): Positional-only arguments: 0 Kw-only arguments: 0 Number of locals: 2 -Stack size: 6 +Stack size: \\d+ Flags: OPTIMIZED, NEWLOCALS, COROUTINE Constants: 0: None @@ -1145,19 +1154,20 @@ def _prepare_test_cases(): Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=code_object_f, argrepr=repr(code_object_f), offset=14, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='MAKE_FUNCTION', opcode=132, arg=9, argval=9, argrepr='defaults, closure', offset=16, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='STORE_FAST', opcode=125, arg=2, argval='f', argrepr='f', offset=18, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='print', argrepr='print', offset=20, starts_line=7, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=0, argval='a', argrepr='a', offset=22, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=1, argval='b', argrepr='b', offset=24, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='', argrepr="''", offset=26, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=28, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='BUILD_LIST', opcode=103, arg=0, argval=0, argrepr='', offset=30, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='BUILD_MAP', opcode=105, arg=0, argval=0, argrepr='', offset=32, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Hello world!', argrepr="'Hello world!'", offset=34, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=7, argval=7, argrepr='', offset=36, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=38, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='f', argrepr='f', offset=42, starts_line=8, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=44, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=20, starts_line=7, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='print', argrepr='print', offset=22, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=0, argval='a', argrepr='a', offset=24, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=1, argval='b', argrepr='b', offset=26, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='', argrepr="''", offset=28, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=30, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='BUILD_LIST', opcode=103, arg=0, argval=0, argrepr='', offset=32, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='BUILD_MAP', opcode=105, arg=0, argval=0, argrepr='', offset=34, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Hello world!', argrepr="'Hello world!'", offset=36, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=7, argval=7, argrepr='', offset=38, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=42, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='f', argrepr='f', offset=44, starts_line=8, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=46, starts_line=None, is_jump_target=False, positions=None), ] expected_opinfo_f = [ @@ -1174,166 +1184,178 @@ def _prepare_test_cases(): Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=code_object_inner, argrepr=repr(code_object_inner), offset=20, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='MAKE_FUNCTION', opcode=132, arg=9, argval=9, argrepr='defaults, closure', offset=22, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='STORE_FAST', opcode=125, arg=2, argval='inner', argrepr='inner', offset=24, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='print', argrepr='print', offset=26, starts_line=5, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=3, argval='a', argrepr='a', offset=28, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=4, argval='b', argrepr='b', offset=30, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=0, argval='c', argrepr='c', offset=32, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=1, argval='d', argrepr='d', offset=34, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=4, argval=4, argrepr='', offset=36, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=38, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='inner', argrepr='inner', offset=42, starts_line=6, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=44, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=26, starts_line=5, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='print', argrepr='print', offset=28, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=3, argval='a', argrepr='a', offset=30, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=4, argval='b', argrepr='b', offset=32, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=0, argval='c', argrepr='c', offset=34, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=1, argval='d', argrepr='d', offset=36, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=4, argval=4, argrepr='', offset=38, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=42, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='inner', argrepr='inner', offset=44, starts_line=6, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=46, starts_line=None, is_jump_target=False, positions=None), ] expected_opinfo_inner = [ Instruction(opname='COPY_FREE_VARS', opcode=149, arg=4, argval=4, argrepr='', offset=0, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='RESUME', opcode=151, arg=0, argval=0, argrepr='', offset=2, starts_line=3, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='print', argrepr='print', offset=4, starts_line=4, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=2, argval='a', argrepr='a', offset=6, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=3, argval='b', argrepr='b', offset=8, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=4, argval='c', argrepr='c', offset=10, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=5, argval='d', argrepr='d', offset=12, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='e', argrepr='e', offset=14, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=1, argval='f', argrepr='f', offset=16, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=6, argval=6, argrepr='', offset=18, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=20, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=22, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=24, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=26, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=4, starts_line=4, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='print', argrepr='print', offset=6, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=2, argval='a', argrepr='a', offset=8, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=3, argval='b', argrepr='b', offset=10, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=4, argval='c', argrepr='c', offset=12, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=5, argval='d', argrepr='d', offset=14, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='e', argrepr='e', offset=16, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=1, argval='f', argrepr='f', offset=18, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=6, argval=6, argrepr='', offset=20, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=22, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=24, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=26, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=28, starts_line=None, is_jump_target=False, positions=None), ] expected_opinfo_jumpy = [ Instruction(opname='RESUME', opcode=151, arg=0, argval=0, argrepr='', offset=0, starts_line=1, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='range', argrepr='range', offset=2, starts_line=3, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval=10, argrepr='10', offset=4, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=6, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=8, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='GET_ITER', opcode=68, arg=None, argval=None, argrepr='', offset=10, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='FOR_ITER', opcode=93, arg=18, argval=50, argrepr='to 50', offset=12, starts_line=None, is_jump_target=True, positions=None), - Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=14, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=16, starts_line=4, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=18, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=20, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=22, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=24, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=26, starts_line=5, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=28, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=30, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=18, argval=36, argrepr='to 36', offset=32, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=6, argval=12, argrepr='to 12', offset=34, starts_line=6, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=36, starts_line=7, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=38, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=40, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=24, argval=48, argrepr='to 48', offset=42, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=44, starts_line=8, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=6, argval=60, argrepr='to 60', offset=46, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=6, argval=12, argrepr='to 12', offset=48, starts_line=7, is_jump_target=True, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=50, starts_line=10, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='I can haz else clause?', argrepr="'I can haz else clause?'", offset=52, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=54, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=56, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=58, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=60, starts_line=11, is_jump_target=True, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=53, argval=106, argrepr='to 106', offset=62, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=64, starts_line=12, is_jump_target=True, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=66, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=68, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=70, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=72, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=74, starts_line=13, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=76, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='BINARY_OP', opcode=122, arg=23, argval=23, argrepr='-=', offset=78, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=80, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=82, starts_line=14, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=84, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=86, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=46, argval=92, argrepr='to 92', offset=88, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=30, argval=60, argrepr='to 60', offset=90, starts_line=15, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=92, starts_line=16, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=94, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=96, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=51, argval=102, argrepr='to 102', offset=98, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=7, argval=116, argrepr='to 116', offset=100, starts_line=17, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=102, starts_line=11, is_jump_target=True, positions=None), - Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=32, argval=64, argrepr='to 64', offset=104, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=106, starts_line=19, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=108, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=110, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=112, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=114, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=116, starts_line=20, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=118, starts_line=21, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=120, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='BINARY_OP', opcode=122, arg=11, argval=11, argrepr='/', offset=122, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=2, starts_line=3, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='range', argrepr='range', offset=4, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval=10, argrepr='10', offset=6, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=8, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=10, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='GET_ITER', opcode=68, arg=None, argval=None, argrepr='', offset=12, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='FOR_ITER', opcode=93, arg=19, argval=54, argrepr='to 54', offset=14, starts_line=None, is_jump_target=True, positions=None), + Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=16, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=18, starts_line=4, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=20, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=22, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=24, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=26, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=28, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=30, starts_line=5, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=32, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=34, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=20, argval=40, argrepr='to 40', offset=36, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=7, argval=14, argrepr='to 14', offset=38, starts_line=6, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=40, starts_line=7, is_jump_target=True, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=42, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=44, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=26, argval=52, argrepr='to 52', offset=46, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=48, starts_line=8, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=7, argval=66, argrepr='to 66', offset=50, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=7, argval=14, argrepr='to 14', offset=52, starts_line=7, is_jump_target=True, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=54, starts_line=10, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=56, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='I can haz else clause?', argrepr="'I can haz else clause?'", offset=58, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=60, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=62, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=64, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=66, starts_line=11, is_jump_target=True, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=57, argval=114, argrepr='to 114', offset=68, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=70, starts_line=12, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=72, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=74, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=76, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=78, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=80, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=82, starts_line=13, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=84, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='BINARY_OP', opcode=122, arg=23, argval=23, argrepr='-=', offset=86, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=88, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=90, starts_line=14, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=92, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=94, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=50, argval=100, argrepr='to 100', offset=96, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=33, argval=66, argrepr='to 66', offset=98, starts_line=15, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=100, starts_line=16, is_jump_target=True, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=102, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=104, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=55, argval=110, argrepr='to 110', offset=106, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=8, argval=126, argrepr='to 126', offset=108, starts_line=17, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=110, starts_line=11, is_jump_target=True, positions=None), + Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=35, argval=70, argrepr='to 70', offset=112, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=114, starts_line=19, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=116, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=118, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=120, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=122, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=124, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=126, starts_line=25, is_jump_target=False, positions=None), - Instruction(opname='BEFORE_WITH', opcode=53, arg=None, argval=None, argrepr='', offset=128, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=130, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=132, starts_line=26, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Never reach this', argrepr="'Never reach this'", offset=134, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=136, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=138, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=140, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=142, starts_line=25, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=144, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=146, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=3, argval=3, argrepr='', offset=148, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=126, starts_line=20, is_jump_target=True, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=128, starts_line=21, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=130, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='BINARY_OP', opcode=122, arg=11, argval=11, argrepr='/', offset=132, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=134, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=136, starts_line=25, is_jump_target=False, positions=None), + Instruction(opname='BEFORE_WITH', opcode=53, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=140, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=142, starts_line=26, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=144, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Never reach this', argrepr="'Never reach this'", offset=146, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=148, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=150, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=152, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=178, argrepr='to 178', offset=154, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=156, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=158, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=85, argval=170, argrepr='to 170', offset=160, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=2, argval=2, argrepr='', offset=162, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=164, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=166, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=168, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=170, starts_line=None, is_jump_target=True, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=174, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=176, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=15, argval=210, argrepr='to 210', offset=178, starts_line=None, is_jump_target=True, positions=None), - Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=180, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=182, starts_line=22, is_jump_target=False, positions=None), - Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=101, argval=202, argrepr='to 202', offset=184, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=154, starts_line=25, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=156, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=158, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=2, argval=2, argrepr='', offset=160, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=162, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=190, argrepr='to 190', offset=166, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=168, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=170, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=91, argval=182, argrepr='to 182', offset=172, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=2, argval=2, argrepr='', offset=174, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=176, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=178, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=180, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=True, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=184, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=188, starts_line=23, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=190, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=192, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=194, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=196, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=198, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=224, argrepr='to 224', offset=200, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=202, starts_line=22, is_jump_target=True, positions=None), - Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=204, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=206, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=208, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=210, starts_line=28, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=212, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=214, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=216, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=218, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=220, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=222, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=224, starts_line=23, is_jump_target=True, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=226, starts_line=28, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=16, argval=224, argrepr='to 224', offset=190, starts_line=None, is_jump_target=True, positions=None), + Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=192, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=194, starts_line=22, is_jump_target=False, positions=None), + Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=108, argval=216, argrepr='to 216', offset=196, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=198, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=200, starts_line=23, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=202, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=204, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=206, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=208, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=210, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=212, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=12, argval=240, argrepr='to 240', offset=214, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=216, starts_line=22, is_jump_target=True, positions=None), + Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=218, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=220, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=222, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=224, starts_line=28, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=226, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=228, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=230, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=230, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=232, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=234, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=236, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=238, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=240, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=242, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=244, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL_FUNCTION', opcode=167, arg=1, argval=1, argrepr='', offset=246, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=248, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=250, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=252, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=254, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=256, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=258, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=240, starts_line=23, is_jump_target=True, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=242, starts_line=28, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=244, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=246, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=248, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=250, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=252, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=254, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=256, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=258, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=260, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=262, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=264, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=266, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=268, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=270, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=272, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=274, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=276, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=278, starts_line=None, is_jump_target=False, positions=None), ] # One last piece of inspect fodder to check the default line number handling @@ -1396,6 +1418,7 @@ def test_co_positions(self): expected = [ (None, None, None, None), (1, 1, 0, 1), + (1, 1, 0, 1), (2, 2, 2, 3), (2, 2, 5, 6), (2, 2, 8, 9), diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index 21d7ccbf70966..f828d1b15d286 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -1393,7 +1393,7 @@ class C(object): pass def func(): return sys._getframe() x = func() - check(x, size('3Pi3c7P2ic??P')) + check(x, size('3Pi3c7P2ic??2P')) # function def func(): pass check(func, size('14Pi')) diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-16-13-15-16.bpo-46329.8aIuz9.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-16-13-15-16.bpo-46329.8aIuz9.rst new file mode 100644 index 0000000000000..7a2dc99c6c79c --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-16-13-15-16.bpo-46329.8aIuz9.rst @@ -0,0 +1,6 @@ +Add ``PUSH_NULL`` instruction. This is used as a prefix when evaluating a +callable, so that the stack has the same shape for methods and other calls. +``PRECALL_FUNCTION`` and ``PRECALL_METHOD`` are merged into a single +``PRECALL`` instruction. + +There is no change in semantics. diff --git a/Programs/test_frozenmain.h b/Programs/test_frozenmain.h index 01e046becd13b..a7f6b6022a49e 100644 --- a/Programs/test_frozenmain.h +++ b/Programs/test_frozenmain.h @@ -1,36 +1,37 @@ // Auto-generated by Programs/freeze_test_frozenmain.py unsigned char M_test_frozenmain[] = { - 227,0,0,0,0,0,0,0,0,0,0,0,0,7,0,0, - 0,0,0,0,0,115,96,0,0,0,151,0,100,0,100,1, - 108,0,90,0,100,0,100,1,108,1,90,1,101,2,100,2, - 167,1,171,0,1,0,101,2,100,3,101,0,106,3,167,2, - 171,0,1,0,101,1,106,4,167,0,171,0,100,4,25,0, - 90,5,100,5,68,0,93,15,90,6,101,2,100,6,101,6, - 155,0,100,7,101,5,101,6,25,0,155,0,157,4,167,1, - 171,0,1,0,113,30,100,1,83,0,41,8,233,0,0,0, - 0,78,122,18,70,114,111,122,101,110,32,72,101,108,108,111, - 32,87,111,114,108,100,122,8,115,121,115,46,97,114,103,118, - 218,6,99,111,110,102,105,103,41,5,90,12,112,114,111,103, - 114,97,109,95,110,97,109,101,218,10,101,120,101,99,117,116, - 97,98,108,101,90,15,117,115,101,95,101,110,118,105,114,111, - 110,109,101,110,116,90,17,99,111,110,102,105,103,117,114,101, - 95,99,95,115,116,100,105,111,90,14,98,117,102,102,101,114, - 101,100,95,115,116,100,105,111,122,7,99,111,110,102,105,103, - 32,122,2,58,32,41,7,218,3,115,121,115,90,17,95,116, - 101,115,116,105,110,116,101,114,110,97,108,99,97,112,105,218, - 5,112,114,105,110,116,218,4,97,114,103,118,90,11,103,101, - 116,95,99,111,110,102,105,103,115,114,2,0,0,0,218,3, - 107,101,121,169,0,243,0,0,0,0,250,18,116,101,115,116, - 95,102,114,111,122,101,110,109,97,105,110,46,112,121,250,8, - 60,109,111,100,117,108,101,62,114,11,0,0,0,1,0,0, - 0,115,18,0,0,0,2,128,8,3,8,1,10,2,14,1, - 14,1,8,1,28,7,4,249,115,20,0,0,0,2,128,8, - 3,8,1,10,2,14,1,14,1,2,7,4,1,2,249,32, - 7,115,96,0,0,0,0,0,1,11,1,11,1,11,1,11, - 1,25,1,25,1,25,1,25,1,6,7,27,1,28,1,28, - 1,28,1,6,7,17,19,22,19,27,1,28,1,28,1,28, - 10,27,10,39,10,41,10,41,42,50,10,51,1,7,12,2, - 1,42,1,42,5,8,5,10,11,41,21,24,11,41,11,41, + 227,0,0,0,0,0,0,0,0,0,0,0,0,8,0,0, + 0,0,0,0,0,115,104,0,0,0,151,0,100,0,100,1, + 108,0,90,0,100,0,100,1,108,1,90,1,2,0,101,2, + 100,2,166,1,171,0,1,0,2,0,101,2,100,3,101,0, + 106,3,166,2,171,0,1,0,2,0,101,1,106,4,166,0, + 171,0,100,4,25,0,90,5,100,5,68,0,93,16,90,6, + 2,0,101,2,100,6,101,6,155,0,100,7,101,5,101,6, + 25,0,155,0,157,4,166,1,171,0,1,0,113,33,100,1, + 83,0,41,8,233,0,0,0,0,78,122,18,70,114,111,122, + 101,110,32,72,101,108,108,111,32,87,111,114,108,100,122,8, + 115,121,115,46,97,114,103,118,218,6,99,111,110,102,105,103, + 41,5,90,12,112,114,111,103,114,97,109,95,110,97,109,101, + 218,10,101,120,101,99,117,116,97,98,108,101,90,15,117,115, + 101,95,101,110,118,105,114,111,110,109,101,110,116,90,17,99, + 111,110,102,105,103,117,114,101,95,99,95,115,116,100,105,111, + 90,14,98,117,102,102,101,114,101,100,95,115,116,100,105,111, + 122,7,99,111,110,102,105,103,32,122,2,58,32,41,7,218, + 3,115,121,115,90,17,95,116,101,115,116,105,110,116,101,114, + 110,97,108,99,97,112,105,218,5,112,114,105,110,116,218,4, + 97,114,103,118,90,11,103,101,116,95,99,111,110,102,105,103, + 115,114,2,0,0,0,218,3,107,101,121,169,0,243,0,0, + 0,0,250,18,116,101,115,116,95,102,114,111,122,101,110,109, + 97,105,110,46,112,121,250,8,60,109,111,100,117,108,101,62, + 114,11,0,0,0,1,0,0,0,115,18,0,0,0,2,128, + 8,3,8,1,12,2,16,1,16,1,8,1,30,7,4,249, + 115,20,0,0,0,2,128,8,3,8,1,12,2,16,1,16, + 1,2,7,4,1,2,249,34,7,115,104,0,0,0,0,0, + 1,11,1,11,1,11,1,11,1,25,1,25,1,25,1,25, + 1,6,1,6,7,27,1,28,1,28,1,28,1,6,1,6, + 7,17,19,22,19,27,1,28,1,28,1,28,10,39,10,27, + 10,39,10,41,10,41,42,50,10,51,1,7,12,2,1,42, + 1,42,5,8,5,10,5,10,11,41,21,24,11,41,11,41, 28,34,35,38,28,39,11,41,11,41,5,42,5,42,5,42, 5,42,1,42,1,42,114,9,0,0,0, }; diff --git a/Python/ceval.c b/Python/ceval.c index 5a6de5bce497f..471bbde46f9db 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1584,7 +1584,7 @@ pop_frame(PyThreadState *tstate, InterpreterFrame *frame) return prev_frame; } -/* It is only between a PRECALL_METHOD/FUNCTION instruction and the following CALL, +/* It is only between the PRECALL instruction and the following CALL, * that these values have any meaning. */ typedef struct { @@ -1872,6 +1872,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } + TARGET(PUSH_NULL) { + /* Use BASIC_PUSH as NULL is not a valid object pointer */ + BASIC_PUSH(NULL); + DISPATCH(); + } + TARGET(UNARY_POSITIVE) { PyObject *value = TOP(); PyObject *res = PyNumber_Positive(value); @@ -4476,25 +4482,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr NOTRACE_DISPATCH(); } - TARGET(PRECALL_FUNCTION) { - /* Move ownership of reference from stack to call_shape */ - call_shape.callable = PEEK(oparg + 1); - call_shape.postcall_shrink = 1; - - call_shape.total_args = oparg; - assert(call_shape.kwnames == NULL); -#ifdef Py_STATS - extern int _PySpecialization_ClassifyCallable(PyObject *); - SpecializationStats *stats = - &_py_stats.opcode_stats[PRECALL_FUNCTION].specialization; - stats->failure++; - int kind = _PySpecialization_ClassifyCallable(call_shape.callable); - stats->failure_kinds[kind]++; -#endif - DISPATCH(); - } - - TARGET(PRECALL_METHOD) { + TARGET(PRECALL) { /* Designed to work in tamdem with LOAD_METHOD. */ /* `meth` is NULL when LOAD_METHOD thinks that it's not a method call. @@ -4533,7 +4521,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr #ifdef Py_STATS extern int _PySpecialization_ClassifyCallable(PyObject *); SpecializationStats *stats = - &_py_stats.opcode_stats[PRECALL_METHOD].specialization; + &_py_stats.opcode_stats[PRECALL].specialization; stats->failure++; int kind = _PySpecialization_ClassifyCallable(call_shape.callable); stats->failure_kinds[kind]++; @@ -5118,6 +5106,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr Py_DECREF(callargs); Py_XDECREF(kwargs); + STACK_SHRINK(1); + assert(TOP() == NULL); SET_TOP(result); if (result == NULL) { goto error; diff --git a/Python/compile.c b/Python/compile.c index 786ef4e8ad3e2..645213b192eae 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -1012,17 +1012,15 @@ stack_effect(int opcode, int oparg, int jump) return -oparg; /* Functions and calls */ - case PRECALL_METHOD: - return -oparg-1; - case PRECALL_FUNCTION: + case PRECALL: return -oparg; case KW_NAMES: return 0; case CALL: - return 0; + return -1; case CALL_FUNCTION_EX: - return -1 - ((oparg & 0x01) != 0); + return -2 - ((oparg & 0x01) != 0); case MAKE_FUNCTION: return 0 - ((oparg & 0x01) != 0) - ((oparg & 0x02) != 0) - ((oparg & 0x04) != 0) - ((oparg & 0x08) != 0); @@ -1084,6 +1082,7 @@ stack_effect(int opcode, int oparg, int jump) case MATCH_KEYS: return 1; case COPY: + case PUSH_NULL: return 1; case BINARY_OP: return -1; @@ -1800,7 +1799,7 @@ compiler_call_exit_with_nones(struct compiler *c) { ADDOP_LOAD_CONST(c, Py_None); ADDOP_LOAD_CONST(c, Py_None); ADDOP_LOAD_CONST(c, Py_None); - ADDOP_I(c, PRECALL_FUNCTION, 3); + ADDOP_I(c, PRECALL, 2); ADDOP_I(c, CALL, 0); return 1; } @@ -2178,7 +2177,7 @@ compiler_apply_decorators(struct compiler *c, asdl_expr_seq* decos) int old_end_col_offset = c->u->u_end_col_offset; for (Py_ssize_t i = asdl_seq_LEN(decos) - 1; i > -1; i--) { SET_LOC(c, (expr_ty)asdl_seq_GET(decos, i)); - ADDOP_I(c, PRECALL_FUNCTION, 1); + ADDOP_I(c, PRECALL, 0); ADDOP_I(c, CALL, 0); } c->u->u_lineno = old_lineno; @@ -2630,6 +2629,7 @@ compiler_class(struct compiler *c, stmt_ty s) return 0; /* 2. load the 'build_class' function */ + ADDOP(c, PUSH_NULL); ADDOP(c, LOAD_BUILD_CLASS); /* 3. load a function (or closure) made from the code object */ @@ -2645,7 +2645,6 @@ compiler_class(struct compiler *c, stmt_ty s) /* 5. generate the rest of the code for the call */ if (!compiler_call_helper(c, 2, s->v.ClassDef.bases, s->v.ClassDef.keywords)) return 0; - /* 6. apply decorators */ if (!compiler_apply_decorators(c, decos)) return 0; @@ -3858,7 +3857,7 @@ compiler_assert(struct compiler *c, stmt_ty s) ADDOP(c, LOAD_ASSERTION_ERROR); if (s->v.Assert.msg) { VISIT(c, expr, s->v.Assert.msg); - ADDOP_I(c, PRECALL_FUNCTION, 1); + ADDOP_I(c, PRECALL, 0); ADDOP_I(c, CALL, 0); } ADDOP_I(c, RAISE_VARARGS, 1); @@ -4680,14 +4679,14 @@ maybe_optimize_method_call(struct compiler *c, expr_ty e) if (kwdsl) { VISIT_SEQ(c, keyword, kwds); - ADDOP_I(c, PRECALL_METHOD, argsl + kwdsl); + ADDOP_I(c, PRECALL, argsl + kwdsl); if (!compiler_call_simple_kw_helper(c, kwds, kwdsl)) { return 0; }; ADDOP_I(c, CALL, kwdsl); } else { - ADDOP_I(c, PRECALL_METHOD, argsl); + ADDOP_I(c, PRECALL, argsl); ADDOP_I(c, CALL, 0); } c->u->u_lineno = old_lineno; @@ -4731,6 +4730,9 @@ compiler_call(struct compiler *c, expr_ty e) if (!check_caller(c, e->v.Call.func)) { return 0; } + SET_LOC(c, e->v.Call.func); + ADDOP(c, PUSH_NULL); + SET_LOC(c, e); VISIT(c, expr, e->v.Call.func); return compiler_call_helper(c, 0, e->v.Call.args, @@ -4755,7 +4757,7 @@ compiler_joined_str(struct compiler *c, expr_ty e) VISIT(c, expr, asdl_seq_GET(e->v.JoinedStr.values, i)); ADDOP_I(c, LIST_APPEND, 1); } - ADDOP_I(c, PRECALL_METHOD, 1); + ADDOP_I(c, PRECALL, 1); ADDOP_I(c, CALL, 0); } else { @@ -4925,7 +4927,7 @@ compiler_call_helper(struct compiler *c, } if (nkwelts) { VISIT_SEQ(c, keyword, keywords); - ADDOP_I(c, PRECALL_FUNCTION, n + nelts + nkwelts); + ADDOP_I(c, PRECALL, n + nelts + nkwelts); if (!compiler_call_simple_kw_helper(c, keywords, nkwelts)) { return 0; }; @@ -4933,7 +4935,7 @@ compiler_call_helper(struct compiler *c, return 1; } else { - ADDOP_I(c, PRECALL_FUNCTION, n + nelts); + ADDOP_I(c, PRECALL, n + nelts); ADDOP_I(c, CALL, 0); return 1; } @@ -5328,7 +5330,7 @@ compiler_comprehension(struct compiler *c, expr_ty e, int type, ADDOP(c, GET_ITER); } - ADDOP_I(c, PRECALL_FUNCTION, 1); + ADDOP_I(c, PRECALL, 0); ADDOP_I(c, CALL, 0); if (is_async_generator && type != COMP_GENEXP) { diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index 1e137f93c7292..f6cbec75089e5 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -1,20 +1,21 @@ static void *opcode_targets[256] = { &&_unknown_opcode, &&TARGET_POP_TOP, + &&TARGET_PUSH_NULL, &&TARGET_BINARY_OP_ADAPTIVE, &&TARGET_BINARY_OP_ADD_INT, &&TARGET_BINARY_OP_ADD_FLOAT, &&TARGET_BINARY_OP_ADD_UNICODE, &&TARGET_BINARY_OP_INPLACE_ADD_UNICODE, &&TARGET_BINARY_OP_MULTIPLY_INT, - &&TARGET_BINARY_OP_MULTIPLY_FLOAT, &&TARGET_NOP, &&TARGET_UNARY_POSITIVE, &&TARGET_UNARY_NEGATIVE, &&TARGET_UNARY_NOT, + &&TARGET_BINARY_OP_MULTIPLY_FLOAT, &&TARGET_BINARY_OP_SUBTRACT_INT, - &&TARGET_BINARY_OP_SUBTRACT_FLOAT, &&TARGET_UNARY_INVERT, + &&TARGET_BINARY_OP_SUBTRACT_FLOAT, &&TARGET_COMPARE_OP_ADAPTIVE, &&TARGET_COMPARE_OP_FLOAT_JUMP, &&TARGET_COMPARE_OP_INT_JUMP, @@ -23,18 +24,18 @@ static void *opcode_targets[256] = { &&TARGET_BINARY_SUBSCR_GETITEM, &&TARGET_BINARY_SUBSCR_LIST_INT, &&TARGET_BINARY_SUBSCR_TUPLE_INT, - &&TARGET_BINARY_SUBSCR_DICT, &&TARGET_BINARY_SUBSCR, + &&TARGET_BINARY_SUBSCR_DICT, &&TARGET_STORE_SUBSCR_ADAPTIVE, &&TARGET_STORE_SUBSCR_LIST_INT, &&TARGET_STORE_SUBSCR_DICT, - &&TARGET_CALL_ADAPTIVE, &&TARGET_GET_LEN, &&TARGET_MATCH_MAPPING, &&TARGET_MATCH_SEQUENCE, &&TARGET_MATCH_KEYS, - &&TARGET_CALL_BUILTIN_CLASS, + &&TARGET_CALL_ADAPTIVE, &&TARGET_PUSH_EXC_INFO, + &&TARGET_CALL_BUILTIN_CLASS, &&TARGET_CALL_NO_KW_BUILTIN_O, &&TARGET_CALL_NO_KW_BUILTIN_FAST, &&TARGET_CALL_BUILTIN_FAST_WITH_KEYWORDS, @@ -47,40 +48,39 @@ static void *opcode_targets[256] = { &&TARGET_CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS, &&TARGET_CALL_NO_KW_STR_1, &&TARGET_CALL_NO_KW_TUPLE_1, - &&TARGET_CALL_NO_KW_TYPE_1, &&TARGET_WITH_EXCEPT_START, &&TARGET_GET_AITER, &&TARGET_GET_ANEXT, &&TARGET_BEFORE_ASYNC_WITH, &&TARGET_BEFORE_WITH, &&TARGET_END_ASYNC_FOR, + &&TARGET_CALL_NO_KW_TYPE_1, &&TARGET_CALL_NO_KW_METHOD_DESCRIPTOR_FAST, &&TARGET_JUMP_ABSOLUTE_QUICK, &&TARGET_LOAD_ATTR_ADAPTIVE, &&TARGET_LOAD_ATTR_INSTANCE_VALUE, - &&TARGET_LOAD_ATTR_WITH_HINT, &&TARGET_STORE_SUBSCR, &&TARGET_DELETE_SUBSCR, + &&TARGET_LOAD_ATTR_WITH_HINT, &&TARGET_LOAD_ATTR_SLOT, &&TARGET_LOAD_ATTR_MODULE, &&TARGET_LOAD_GLOBAL_ADAPTIVE, &&TARGET_LOAD_GLOBAL_MODULE, &&TARGET_LOAD_GLOBAL_BUILTIN, - &&TARGET_LOAD_METHOD_ADAPTIVE, &&TARGET_GET_ITER, &&TARGET_GET_YIELD_FROM_ITER, &&TARGET_PRINT_EXPR, &&TARGET_LOAD_BUILD_CLASS, - &&TARGET_LOAD_METHOD_CACHED, + &&TARGET_LOAD_METHOD_ADAPTIVE, &&TARGET_GET_AWAITABLE, &&TARGET_LOAD_ASSERTION_ERROR, &&TARGET_RETURN_GENERATOR, + &&TARGET_LOAD_METHOD_CACHED, &&TARGET_LOAD_METHOD_CLASS, &&TARGET_LOAD_METHOD_MODULE, &&TARGET_LOAD_METHOD_NO_DICT, &&TARGET_RESUME_QUICK, &&TARGET_STORE_ATTR_ADAPTIVE, - &&TARGET_STORE_ATTR_INSTANCE_VALUE, &&TARGET_LIST_TO_TUPLE, &&TARGET_RETURN_VALUE, &&TARGET_IMPORT_STAR, @@ -130,7 +130,7 @@ static void *opcode_targets[256] = { &&TARGET_POP_JUMP_IF_NOT_NONE, &&TARGET_POP_JUMP_IF_NONE, &&TARGET_RAISE_VARARGS, - &&TARGET_STORE_ATTR_SLOT, + &&TARGET_STORE_ATTR_INSTANCE_VALUE, &&TARGET_MAKE_FUNCTION, &&TARGET_BUILD_SLICE, &&TARGET_JUMP_NO_INTERRUPT, @@ -139,35 +139,35 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_DEREF, &&TARGET_STORE_DEREF, &&TARGET_DELETE_DEREF, + &&TARGET_STORE_ATTR_SLOT, &&TARGET_STORE_ATTR_WITH_HINT, - &&TARGET_UNPACK_SEQUENCE_ADAPTIVE, &&TARGET_CALL_FUNCTION_EX, - &&TARGET_UNPACK_SEQUENCE_LIST, + &&TARGET_UNPACK_SEQUENCE_ADAPTIVE, &&TARGET_EXTENDED_ARG, &&TARGET_LIST_APPEND, &&TARGET_SET_ADD, &&TARGET_MAP_ADD, &&TARGET_LOAD_CLASSDEREF, &&TARGET_COPY_FREE_VARS, - &&TARGET_UNPACK_SEQUENCE_TUPLE, + &&TARGET_UNPACK_SEQUENCE_LIST, &&TARGET_RESUME, &&TARGET_MATCH_CLASS, + &&TARGET_UNPACK_SEQUENCE_TUPLE, &&TARGET_UNPACK_SEQUENCE_TWO_TUPLE, - &&TARGET_LOAD_FAST__LOAD_FAST, &&TARGET_FORMAT_VALUE, &&TARGET_BUILD_CONST_KEY_MAP, &&TARGET_BUILD_STRING, + &&TARGET_LOAD_FAST__LOAD_FAST, &&TARGET_STORE_FAST__LOAD_FAST, - &&TARGET_LOAD_FAST__LOAD_CONST, &&TARGET_LOAD_METHOD, - &&TARGET_LOAD_CONST__LOAD_FAST, + &&TARGET_LOAD_FAST__LOAD_CONST, &&TARGET_LIST_EXTEND, &&TARGET_SET_UPDATE, &&TARGET_DICT_MERGE, &&TARGET_DICT_UPDATE, + &&TARGET_PRECALL, + &&TARGET_LOAD_CONST__LOAD_FAST, &&TARGET_STORE_FAST__STORE_FAST, - &&TARGET_PRECALL_FUNCTION, - &&TARGET_PRECALL_METHOD, &&_unknown_opcode, &&_unknown_opcode, &&TARGET_CALL, diff --git a/Python/specialize.c b/Python/specialize.c index 5fd7d0923a698..7dd12c7de2aeb 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -178,8 +178,7 @@ print_spec_stats(FILE *out, OpcodeStats *stats) /* Mark some opcodes as specializable for stats, * even though we don't specialize them yet. */ fprintf(out, "opcode[%d].specializable : 1\n", FOR_ITER); - fprintf(out, "opcode[%d].specializable : 1\n", PRECALL_FUNCTION); - fprintf(out, "opcode[%d].specializable : 1\n", PRECALL_METHOD); + fprintf(out, "opcode[%d].specializable : 1\n", PRECALL); fprintf(out, "opcode[%d].specializable : 1\n", UNPACK_SEQUENCE); for (int i = 0; i < 256; i++) { if (adaptive_opcodes[i]) { @@ -1528,7 +1527,7 @@ specialize_method_descriptor( } assert(_list_append != NULL); if (nargs == 2 && descr == _list_append) { - assert(_Py_OPCODE(instr[-1]) == PRECALL_METHOD); + assert(_Py_OPCODE(instr[-1]) == PRECALL); cache[-1].obj.obj = (PyObject *)_list_append; *instr = _Py_MAKECODEUNIT(CALL_NO_KW_LIST_APPEND, _Py_OPARG(*instr)); return 0; From webhook-mailer at python.org Fri Feb 18 12:29:15 2022 From: webhook-mailer at python.org (gvanrossum) Date: Fri, 18 Feb 2022 17:29:15 -0000 Subject: [Python-checkins] Reduce flakiness of taskgroups test 13 (GH-31411) Message-ID: https://github.com/python/cpython/commit/80a9ba537f1f1666a9e6c5eceef4683f86967a1f commit: 80a9ba537f1f1666a9e6c5eceef4683f86967a1f branch: main author: Guido van Rossum committer: gvanrossum date: 2022-02-18T09:28:56-08:00 summary: Reduce flakiness of taskgroups test 13 (GH-31411) files: M Lib/test/test_asyncio/test_taskgroups.py diff --git a/Lib/test/test_asyncio/test_taskgroups.py b/Lib/test/test_asyncio/test_taskgroups.py index aab1fd1ebb38d..40774a8993e3f 100644 --- a/Lib/test/test_asyncio/test_taskgroups.py +++ b/Lib/test/test_asyncio/test_taskgroups.py @@ -372,7 +372,7 @@ async def runner(): g1.create_task(crash_after(0.1)) async with taskgroups.TaskGroup() as g2: - g2.create_task(crash_after(0.2)) + g2.create_task(crash_after(10)) r = asyncio.create_task(runner()) with self.assertRaises(ExceptionGroup) as cm: From webhook-mailer at python.org Fri Feb 18 13:57:11 2022 From: webhook-mailer at python.org (ambv) Date: Fri, 18 Feb 2022 18:57:11 -0000 Subject: [Python-checkins] "Own" pattern matching and peepholing (GH-31412) Message-ID: https://github.com/python/cpython/commit/91cc5fd14cb8fc0f9d4e18d7f8997e6e3027a516 commit: 91cc5fd14cb8fc0f9d4e18d7f8997e6e3027a516 branch: main author: Brandt Bucher committer: ambv date: 2022-02-18T19:57:02+01:00 summary: "Own" pattern matching and peepholing (GH-31412) files: M .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index a884713ec500f..d2097d32557e1 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -20,6 +20,8 @@ Objects/call.c @markshannon Python/ceval.c @markshannon Python/compile.c @markshannon Python/ast_opt.c @isidentical +Lib/test/test_patma.py @brandtbucher +Lib/test/test_peepholer.py @brandtbucher # Exceptions Lib/traceback.py @iritkatriel From webhook-mailer at python.org Fri Feb 18 14:28:41 2022 From: webhook-mailer at python.org (Mariatta) Date: Fri, 18 Feb 2022 19:28:41 -0000 Subject: [Python-checkins] Process more stale PRs, oldest first (GH-31407) Message-ID: https://github.com/python/cpython/commit/892ff93d7f47f8edae5c8a0f5d5ffd4b0a8f8d77 commit: 892ff93d7f47f8edae5c8a0f5d5ffd4b0a8f8d77 branch: main author: Hugo van Kemenade committer: Mariatta date: 2022-02-18T11:28:12-08:00 summary: Process more stale PRs, oldest first (GH-31407) files: M .github/workflows/stale.yml diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml index fc0c15dc0a600..25e29e83dff2b 100644 --- a/.github/workflows/stale.yml +++ b/.github/workflows/stale.yml @@ -23,6 +23,8 @@ jobs: stale-pr-label: 'stale' days-before-stale: 30 days-before-close: -1 + ascending: true + operations-per-run: 120 - name: "Check PRs with 'CLA not signed' label" uses: actions/stale at v4 @@ -34,3 +36,5 @@ jobs: close-pr-message: 'Closing this stale PR because the CLA is still not signed.' days-before-stale: 30 days-before-close: 14 + ascending: true + operations-per-run: 120 From webhook-mailer at python.org Fri Feb 18 17:33:16 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 18 Feb 2022 22:33:16 -0000 Subject: [Python-checkins] bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397) Message-ID: https://github.com/python/cpython/commit/6312c1052c0186b4596fc45c42fd3ade9f8f5911 commit: 6312c1052c0186b4596fc45c42fd3ade9f8f5911 branch: main author: Yilei "Dolee" Yang committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-18T14:33:06-08:00 summary: bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397) The libexpat 2.4.1 upgrade from introduced the following new exported symbols: * `testingAccountingGetCountBytesDirect` * `testingAccountingGetCountBytesIndirect` * `unsignedCharToPrintable` * `XML_SetBillionLaughsAttackProtectionActivationThreshold` * `XML_SetBillionLaughsAttackProtectionMaximumAmplification` We need to adjust [Modules/expat/pyexpatns.h](https://github.com/python/cpython/blob/master/Modules/expat/pyexpatns.h) (The newer libexpat upgrade has no new symbols). Automerge-Triggered-By: GH:gpshead files: A Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst M Modules/expat/pyexpatns.h diff --git a/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst b/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst new file mode 100644 index 0000000000000..d190816637ae8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst @@ -0,0 +1 @@ +Fix libexpat symbols collisions with user dynamically loaded or statically linked libexpat in embedded Python. diff --git a/Modules/expat/pyexpatns.h b/Modules/expat/pyexpatns.h index cfb742ee000b0..d45d9b6c45715 100644 --- a/Modules/expat/pyexpatns.h +++ b/Modules/expat/pyexpatns.h @@ -38,6 +38,9 @@ #ifndef PYEXPATNS_H #define PYEXPATNS_H +#define testingAccountingGetCountBytesDirect PyExpat_testingAccountingGetCountBytesDirect +#define testingAccountingGetCountBytesIndirect PyExpat_testingAccountingGetCountBytesIndirect +#define unsignedCharToPrintable PyExpat_unsignedCharToPrintable #define XML_DefaultCurrent PyExpat_XML_DefaultCurrent #define XML_ErrorString PyExpat_XML_ErrorString #define XML_ExpatVersion PyExpat_XML_ExpatVersion @@ -81,6 +84,8 @@ #define XML_ResumeParser PyExpat_XML_ResumeParser #define XML_SetAttlistDeclHandler PyExpat_XML_SetAttlistDeclHandler #define XML_SetBase PyExpat_XML_SetBase +#define XML_SetBillionLaughsAttackProtectionActivationThreshold PyExpat_XML_SetBillionLaughsAttackProtectionActivationThreshold +#define XML_SetBillionLaughsAttackProtectionMaximumAmplification PyExpat_XML_SetBillionLaughsAttackProtectionMaximumAmplification #define XML_SetCdataSectionHandler PyExpat_XML_SetCdataSectionHandler #define XML_SetCharacterDataHandler PyExpat_XML_SetCharacterDataHandler #define XML_SetCommentHandler PyExpat_XML_SetCommentHandler From webhook-mailer at python.org Fri Feb 18 18:13:20 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 18 Feb 2022 23:13:20 -0000 Subject: [Python-checkins] bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397) Message-ID: https://github.com/python/cpython/commit/9aca412db8343702d9199c31ba73519e6f8823b5 commit: 9aca412db8343702d9199c31ba73519e6f8823b5 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-18T15:13:16-08:00 summary: bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397) The libexpat 2.4.1 upgrade from introduced the following new exported symbols: * `testingAccountingGetCountBytesDirect` * `testingAccountingGetCountBytesIndirect` * `unsignedCharToPrintable` * `XML_SetBillionLaughsAttackProtectionActivationThreshold` * `XML_SetBillionLaughsAttackProtectionMaximumAmplification` We need to adjust [Modules/expat/pyexpatns.h](https://github.com/python/cpython/blob/master/Modules/expat/pyexpatns.h) (The newer libexpat upgrade has no new symbols). Automerge-Triggered-By: GH:gpshead (cherry picked from commit 6312c1052c0186b4596fc45c42fd3ade9f8f5911) Co-authored-by: Yilei "Dolee" Yang files: A Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst M Modules/expat/pyexpatns.h diff --git a/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst b/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst new file mode 100644 index 0000000000000..d190816637ae8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst @@ -0,0 +1 @@ +Fix libexpat symbols collisions with user dynamically loaded or statically linked libexpat in embedded Python. diff --git a/Modules/expat/pyexpatns.h b/Modules/expat/pyexpatns.h index cfb742ee000b0..d45d9b6c45715 100644 --- a/Modules/expat/pyexpatns.h +++ b/Modules/expat/pyexpatns.h @@ -38,6 +38,9 @@ #ifndef PYEXPATNS_H #define PYEXPATNS_H +#define testingAccountingGetCountBytesDirect PyExpat_testingAccountingGetCountBytesDirect +#define testingAccountingGetCountBytesIndirect PyExpat_testingAccountingGetCountBytesIndirect +#define unsignedCharToPrintable PyExpat_unsignedCharToPrintable #define XML_DefaultCurrent PyExpat_XML_DefaultCurrent #define XML_ErrorString PyExpat_XML_ErrorString #define XML_ExpatVersion PyExpat_XML_ExpatVersion @@ -81,6 +84,8 @@ #define XML_ResumeParser PyExpat_XML_ResumeParser #define XML_SetAttlistDeclHandler PyExpat_XML_SetAttlistDeclHandler #define XML_SetBase PyExpat_XML_SetBase +#define XML_SetBillionLaughsAttackProtectionActivationThreshold PyExpat_XML_SetBillionLaughsAttackProtectionActivationThreshold +#define XML_SetBillionLaughsAttackProtectionMaximumAmplification PyExpat_XML_SetBillionLaughsAttackProtectionMaximumAmplification #define XML_SetCdataSectionHandler PyExpat_XML_SetCdataSectionHandler #define XML_SetCharacterDataHandler PyExpat_XML_SetCharacterDataHandler #define XML_SetCommentHandler PyExpat_XML_SetCommentHandler From webhook-mailer at python.org Fri Feb 18 19:53:59 2022 From: webhook-mailer at python.org (gpshead) Date: Sat, 19 Feb 2022 00:53:59 -0000 Subject: [Python-checkins] [3.10] bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397) (GH-31420) Message-ID: https://github.com/python/cpython/commit/e05e3d20d309603010f2c1194e612f894ad8a985 commit: e05e3d20d309603010f2c1194e612f894ad8a985 branch: 3.10 author: Gregory P. Smith committer: gpshead date: 2022-02-18T16:53:49-08:00 summary: [3.10] bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397) (GH-31420) The libexpat 2.4.1 upgrade from introduced the following new exported symbols: * `testingAccountingGetCountBytesDirect` * `testingAccountingGetCountBytesIndirect` * `unsignedCharToPrintable` * `XML_SetBillionLaughsAttackProtectionActivationThreshold` * `XML_SetBillionLaughsAttackProtectionMaximumAmplification` We need to adjust [Modules/expat/pyexpatns.h](https://github.com/python/cpython/blob/master/Modules/expat/pyexpatns.h) (The newer libexpat upgrade has no new symbols). Automerge-Triggered-By: GH:gpshead (cherry picked from commit 6312c1052c0186b4596fc45c42fd3ade9f8f5911) Co-authored-by: Yilei "Dolee" Yang Co-authored-by: Yilei "Dolee" Yang files: A Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst M Modules/expat/pyexpatns.h diff --git a/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst b/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst new file mode 100644 index 0000000000000..d190816637ae8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst @@ -0,0 +1 @@ +Fix libexpat symbols collisions with user dynamically loaded or statically linked libexpat in embedded Python. diff --git a/Modules/expat/pyexpatns.h b/Modules/expat/pyexpatns.h index cfb742ee000b0..d45d9b6c45715 100644 --- a/Modules/expat/pyexpatns.h +++ b/Modules/expat/pyexpatns.h @@ -38,6 +38,9 @@ #ifndef PYEXPATNS_H #define PYEXPATNS_H +#define testingAccountingGetCountBytesDirect PyExpat_testingAccountingGetCountBytesDirect +#define testingAccountingGetCountBytesIndirect PyExpat_testingAccountingGetCountBytesIndirect +#define unsignedCharToPrintable PyExpat_unsignedCharToPrintable #define XML_DefaultCurrent PyExpat_XML_DefaultCurrent #define XML_ErrorString PyExpat_XML_ErrorString #define XML_ExpatVersion PyExpat_XML_ExpatVersion @@ -81,6 +84,8 @@ #define XML_ResumeParser PyExpat_XML_ResumeParser #define XML_SetAttlistDeclHandler PyExpat_XML_SetAttlistDeclHandler #define XML_SetBase PyExpat_XML_SetBase +#define XML_SetBillionLaughsAttackProtectionActivationThreshold PyExpat_XML_SetBillionLaughsAttackProtectionActivationThreshold +#define XML_SetBillionLaughsAttackProtectionMaximumAmplification PyExpat_XML_SetBillionLaughsAttackProtectionMaximumAmplification #define XML_SetCdataSectionHandler PyExpat_XML_SetCdataSectionHandler #define XML_SetCharacterDataHandler PyExpat_XML_SetCharacterDataHandler #define XML_SetCommentHandler PyExpat_XML_SetCommentHandler From webhook-mailer at python.org Fri Feb 18 20:49:15 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Sat, 19 Feb 2022 01:49:15 -0000 Subject: [Python-checkins] [docs] Correct typos in SSLContext.sni_callback (GH-30623) Message-ID: https://github.com/python/cpython/commit/f80a97b492f41afd3c42bb2bd6da7b2828dca215 commit: f80a97b492f41afd3c42bb2bd6da7b2828dca215 branch: main author: J?rn Heissler committer: JelleZijlstra date: 2022-02-18T17:49:04-08:00 summary: [docs] Correct typos in SSLContext.sni_callback (GH-30623) Co-authored-by: J?rn Heissler files: M Doc/library/ssl.rst diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 151f2546eeb60..8e5554ee4b842 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1753,10 +1753,10 @@ to speed up repeated connections from the same clients. Due to the early negotiation phase of the TLS connection, only limited methods and attributes are usable like :meth:`SSLSocket.selected_alpn_protocol` and :attr:`SSLSocket.context`. - :meth:`SSLSocket.getpeercert`, :meth:`SSLSocket.getpeercert`, - :meth:`SSLSocket.cipher` and :meth:`SSLSocket.compress` methods require that + The :meth:`SSLSocket.getpeercert`, + :meth:`SSLSocket.cipher` and :meth:`SSLSocket.compression` methods require that the TLS connection has progressed beyond the TLS Client Hello and therefore - will not contain return meaningful values nor can they be called safely. + will not return meaningful values nor can they be called safely. The *sni_callback* function must return ``None`` to allow the TLS negotiation to continue. If a TLS failure is required, a constant From webhook-mailer at python.org Fri Feb 18 20:53:44 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Sat, 19 Feb 2022 01:53:44 -0000 Subject: [Python-checkins] bpo-46571: improve `typing.no_type_check` to skip foreign objects (GH-31042) Message-ID: https://github.com/python/cpython/commit/395029b0bd343648b4da8044c7509672ea768775 commit: 395029b0bd343648b4da8044c7509672ea768775 branch: main author: Nikita Sobolev committer: JelleZijlstra date: 2022-02-18T17:53:29-08:00 summary: bpo-46571: improve `typing.no_type_check` to skip foreign objects (GH-31042) There are several changes: 1. We now don't explicitly check for any base / sub types, because new name check covers it 2. I've also checked that `no_type_check` do not modify foreign functions. It was the same as with `type`s 3. I've also covered `except TypeError` in `no_type_check` with a simple test case, it was not covered at all 4. I also felt like adding `lambda` test is a good idea: because `lambda` is a bit of both in class bodies: a function and an assignment https://bugs.python.org/issue46571 files: A Lib/test/ann_module8.py A Misc/NEWS.d/next/Library/2022-02-01-11-21-34.bpo-46571.L40xUJ.rst M Doc/library/typing.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 8240c912c6497..a02ad244d9fbb 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -2145,8 +2145,8 @@ Functions and decorators Decorator to indicate that annotations are not type hints. This works as class or function :term:`decorator`. With a class, it - applies recursively to all methods defined in that class (but not - to methods defined in its superclasses or subclasses). + applies recursively to all methods and classes defined in that class + (but not to methods defined in its superclasses or subclasses). This mutates the function(s) in place. diff --git a/Lib/test/ann_module8.py b/Lib/test/ann_module8.py new file mode 100644 index 0000000000000..bd03148137841 --- /dev/null +++ b/Lib/test/ann_module8.py @@ -0,0 +1,10 @@ +# Test `@no_type_check`, +# see https://bugs.python.org/issue46571 + +class NoTypeCheck_Outer: + class Inner: + x: int + + +def NoTypeCheck_function(arg: int) -> int: + ... diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 1b3eb0667668f..d24a357134361 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -2744,6 +2744,18 @@ def test_errors(self): cast('hello', 42) +# We need this to make sure that `@no_type_check` respects `__module__` attr: +from test import ann_module8 + + at no_type_check +class NoTypeCheck_Outer: + Inner = ann_module8.NoTypeCheck_Outer.Inner + + at no_type_check +class NoTypeCheck_WithFunction: + NoTypeCheck_function = ann_module8.NoTypeCheck_function + + class ForwardRefTests(BaseTestCase): def test_basics(self): @@ -3058,9 +3070,98 @@ def meth(self, x: int): ... @no_type_check class D(C): c = C + # verify that @no_type_check never affects bases self.assertEqual(get_type_hints(C.meth), {'x': int}) + # and never child classes: + class Child(D): + def foo(self, x: int): ... + + self.assertEqual(get_type_hints(Child.foo), {'x': int}) + + def test_no_type_check_nested_types(self): + # See https://bugs.python.org/issue46571 + class Other: + o: int + class B: # Has the same `__name__`` as `A.B` and different `__qualname__` + o: int + @no_type_check + class A: + a: int + class B: + b: int + class C: + c: int + class D: + d: int + + Other = Other + + for klass in [A, A.B, A.B.C, A.D]: + with self.subTest(klass=klass): + self.assertTrue(klass.__no_type_check__) + self.assertEqual(get_type_hints(klass), {}) + + for not_modified in [Other, B]: + with self.subTest(not_modified=not_modified): + with self.assertRaises(AttributeError): + not_modified.__no_type_check__ + self.assertNotEqual(get_type_hints(not_modified), {}) + + def test_no_type_check_class_and_static_methods(self): + @no_type_check + class Some: + @staticmethod + def st(x: int) -> int: ... + @classmethod + def cl(cls, y: int) -> int: ... + + self.assertTrue(Some.st.__no_type_check__) + self.assertEqual(get_type_hints(Some.st), {}) + self.assertTrue(Some.cl.__no_type_check__) + self.assertEqual(get_type_hints(Some.cl), {}) + + def test_no_type_check_other_module(self): + self.assertTrue(NoTypeCheck_Outer.__no_type_check__) + with self.assertRaises(AttributeError): + ann_module8.NoTypeCheck_Outer.__no_type_check__ + with self.assertRaises(AttributeError): + ann_module8.NoTypeCheck_Outer.Inner.__no_type_check__ + + self.assertTrue(NoTypeCheck_WithFunction.__no_type_check__) + with self.assertRaises(AttributeError): + ann_module8.NoTypeCheck_function.__no_type_check__ + + def test_no_type_check_foreign_functions(self): + # We should not modify this function: + def some(*args: int) -> int: + ... + + @no_type_check + class A: + some_alias = some + some_class = classmethod(some) + some_static = staticmethod(some) + + with self.assertRaises(AttributeError): + some.__no_type_check__ + self.assertEqual(get_type_hints(some), {'args': int, 'return': int}) + + def test_no_type_check_lambda(self): + @no_type_check + class A: + # Corner case: `lambda` is both an assignment and a function: + bar: Callable[[int], int] = lambda arg: arg + + self.assertTrue(A.bar.__no_type_check__) + self.assertEqual(get_type_hints(A.bar), {}) + + def test_no_type_check_TypeError(self): + # This simply should not fail with + # `TypeError: can't set attributes of built-in/extension type 'dict'` + no_type_check(dict) + def test_no_type_check_forward_ref_as_string(self): class C: foo: typing.ClassVar[int] = 7 diff --git a/Lib/typing.py b/Lib/typing.py index 8f923fa20f534..ad1435ed23d27 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -2131,13 +2131,23 @@ def no_type_check(arg): This mutates the function(s) or class(es) in place. """ if isinstance(arg, type): - arg_attrs = arg.__dict__.copy() - for attr, val in arg.__dict__.items(): - if val in arg.__bases__ + (arg,): - arg_attrs.pop(attr) - for obj in arg_attrs.values(): + for key in dir(arg): + obj = getattr(arg, key) + if ( + not hasattr(obj, '__qualname__') + or obj.__qualname__ != f'{arg.__qualname__}.{obj.__name__}' + or getattr(obj, '__module__', None) != arg.__module__ + ): + # We only modify objects that are defined in this type directly. + # If classes / methods are nested in multiple layers, + # we will modify them when processing their direct holders. + continue + # Instance, class, and static methods: if isinstance(obj, types.FunctionType): obj.__no_type_check__ = True + if isinstance(obj, types.MethodType): + obj.__func__.__no_type_check__ = True + # Nested types: if isinstance(obj, type): no_type_check(obj) try: diff --git a/Misc/NEWS.d/next/Library/2022-02-01-11-21-34.bpo-46571.L40xUJ.rst b/Misc/NEWS.d/next/Library/2022-02-01-11-21-34.bpo-46571.L40xUJ.rst new file mode 100644 index 0000000000000..f56c9e4fd76d3 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-01-11-21-34.bpo-46571.L40xUJ.rst @@ -0,0 +1,4 @@ +Improve :func:`typing.no_type_check`. + +Now it does not modify external classes and functions. +We also now correctly mark classmethods as not to be type checked. From webhook-mailer at python.org Fri Feb 18 20:54:05 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Sat, 19 Feb 2022 01:54:05 -0000 Subject: [Python-checkins] bpo-46603: improve coverage of `typing._strip_annotations` (GH-31063) Message-ID: https://github.com/python/cpython/commit/25c0b9d243b64ccd2eeab483089eaf7e4b4d5834 commit: 25c0b9d243b64ccd2eeab483089eaf7e4b4d5834 branch: main author: Nikita Sobolev committer: JelleZijlstra date: 2022-02-18T17:54:01-08:00 summary: bpo-46603: improve coverage of `typing._strip_annotations` (GH-31063) files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index d24a357134361..3dc9497c3a02e 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -3549,6 +3549,15 @@ def barfoo4(x: BA3): ... {"x": typing.Annotated[int | float, "const"]} ) + def test_get_type_hints_annotated_in_union(self): # bpo-46603 + def with_union(x: int | list[Annotated[str, 'meta']]): ... + + self.assertEqual(get_type_hints(with_union), {'x': int | list[str]}) + self.assertEqual( + get_type_hints(with_union, include_extras=True), + {'x': int | list[Annotated[str, 'meta']]}, + ) + def test_get_type_hints_annotated_refs(self): Const = Annotated[T, "Const"] From webhook-mailer at python.org Fri Feb 18 20:56:35 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Sat, 19 Feb 2022 01:56:35 -0000 Subject: [Python-checkins] bpo-46685: improve test coverage of `Self` and `Never` in `typing` (GH-31222) Message-ID: https://github.com/python/cpython/commit/32e3e0bea613711a8f19003445eebcb05fb75acc commit: 32e3e0bea613711a8f19003445eebcb05fb75acc branch: main author: Nikita Sobolev committer: JelleZijlstra date: 2022-02-18T17:56:30-08:00 summary: bpo-46685: improve test coverage of `Self` and `Never` in `typing` (GH-31222) files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 3dc9497c3a02e..b38e27c5f9047 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -127,6 +127,14 @@ def test_any_works_with_alias(self): class BottomTypeTestsMixin: bottom_type: ClassVar[Any] + def test_equality(self): + self.assertEqual(self.bottom_type, self.bottom_type) + self.assertIs(self.bottom_type, self.bottom_type) + self.assertNotEqual(self.bottom_type, None) + + def test_get_origin(self): + self.assertIs(get_origin(self.bottom_type), None) + def test_instance_type_error(self): with self.assertRaises(TypeError): isinstance(42, self.bottom_type) @@ -162,6 +170,19 @@ class NoReturnTests(BottomTypeTestsMixin, BaseTestCase): def test_repr(self): self.assertEqual(repr(NoReturn), 'typing.NoReturn') + def test_get_type_hints(self): + def some(arg: NoReturn) -> NoReturn: ... + def some_str(arg: 'NoReturn') -> 'typing.NoReturn': ... + + expected = {'arg': NoReturn, 'return': NoReturn} + for target in [some, some_str]: + with self.subTest(target=target): + self.assertEqual(gth(target), expected) + + def test_not_equality(self): + self.assertNotEqual(NoReturn, Never) + self.assertNotEqual(Never, NoReturn) + class NeverTests(BottomTypeTestsMixin, BaseTestCase): bottom_type = Never @@ -169,6 +190,15 @@ class NeverTests(BottomTypeTestsMixin, BaseTestCase): def test_repr(self): self.assertEqual(repr(Never), 'typing.Never') + def test_get_type_hints(self): + def some(arg: Never) -> Never: ... + def some_str(arg: 'Never') -> 'typing.Never': ... + + expected = {'arg': Never, 'return': Never} + for target in [some, some_str]: + with self.subTest(target=target): + self.assertEqual(gth(target), expected) + class AssertNeverTests(BaseTestCase): def test_exception(self): @@ -177,11 +207,23 @@ def test_exception(self): class SelfTests(BaseTestCase): + def test_equality(self): + self.assertEqual(Self, Self) + self.assertIs(Self, Self) + self.assertNotEqual(Self, None) + def test_basics(self): class Foo: def bar(self) -> Self: ... + class FooStr: + def bar(self) -> 'Self': ... + class FooStrTyping: + def bar(self) -> 'typing.Self': ... - self.assertEqual(gth(Foo.bar), {'return': Self}) + for target in [Foo, FooStr, FooStrTyping]: + with self.subTest(target=target): + self.assertEqual(gth(target.bar), {'return': Self}) + self.assertIs(get_origin(Self), None) def test_repr(self): self.assertEqual(repr(Self), 'typing.Self') @@ -194,6 +236,9 @@ def test_cannot_subclass(self): with self.assertRaises(TypeError): class C(type(Self)): pass + with self.assertRaises(TypeError): + class C(Self): + pass def test_cannot_init(self): with self.assertRaises(TypeError): @@ -5425,11 +5470,13 @@ def test_special_attrs(self): typing.Literal: 'Literal', typing.NewType: 'NewType', typing.NoReturn: 'NoReturn', + typing.Never: 'Never', typing.Optional: 'Optional', typing.TypeAlias: 'TypeAlias', typing.TypeGuard: 'TypeGuard', typing.TypeVar: 'TypeVar', typing.Union: 'Union', + typing.Self: 'Self', # Subscribed special forms typing.Annotated[Any, "Annotation"]: 'Annotated', typing.ClassVar[Any]: 'ClassVar', From webhook-mailer at python.org Fri Feb 18 21:13:45 2022 From: webhook-mailer at python.org (miss-islington) Date: Sat, 19 Feb 2022 02:13:45 -0000 Subject: [Python-checkins] [docs] Correct typos in SSLContext.sni_callback (GH-30623) Message-ID: https://github.com/python/cpython/commit/9399dc45cb4b5d5f1c47de9dd183d3abac4ed496 commit: 9399dc45cb4b5d5f1c47de9dd183d3abac4ed496 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-18T18:13:33-08:00 summary: [docs] Correct typos in SSLContext.sni_callback (GH-30623) Co-authored-by: J?rn Heissler (cherry picked from commit f80a97b492f41afd3c42bb2bd6da7b2828dca215) Co-authored-by: J?rn Heissler files: M Doc/library/ssl.rst diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 0b231f170aa05..c3acf61d2b527 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1719,10 +1719,10 @@ to speed up repeated connections from the same clients. Due to the early negotiation phase of the TLS connection, only limited methods and attributes are usable like :meth:`SSLSocket.selected_alpn_protocol` and :attr:`SSLSocket.context`. - :meth:`SSLSocket.getpeercert`, :meth:`SSLSocket.getpeercert`, - :meth:`SSLSocket.cipher` and :meth:`SSLSocket.compress` methods require that + The :meth:`SSLSocket.getpeercert`, + :meth:`SSLSocket.cipher` and :meth:`SSLSocket.compression` methods require that the TLS connection has progressed beyond the TLS Client Hello and therefore - will not contain return meaningful values nor can they be called safely. + will not return meaningful values nor can they be called safely. The *sni_callback* function must return ``None`` to allow the TLS negotiation to continue. If a TLS failure is required, a constant From webhook-mailer at python.org Fri Feb 18 21:16:09 2022 From: webhook-mailer at python.org (miss-islington) Date: Sat, 19 Feb 2022 02:16:09 -0000 Subject: [Python-checkins] bpo-46603: improve coverage of `typing._strip_annotations` (GH-31063) Message-ID: https://github.com/python/cpython/commit/103f3ca80616958b4e608e9176b9c12cfc80f68b commit: 103f3ca80616958b4e608e9176b9c12cfc80f68b branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-18T18:15:56-08:00 summary: bpo-46603: improve coverage of `typing._strip_annotations` (GH-31063) (cherry picked from commit 25c0b9d243b64ccd2eeab483089eaf7e4b4d5834) Co-authored-by: Nikita Sobolev files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index c4c06008d91fa..8ced27824205d 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -3305,6 +3305,15 @@ def barfoo4(x: BA3): ... {"x": typing.Annotated[int | float, "const"]} ) + def test_get_type_hints_annotated_in_union(self): # bpo-46603 + def with_union(x: int | list[Annotated[str, 'meta']]): ... + + self.assertEqual(get_type_hints(with_union), {'x': int | list[str]}) + self.assertEqual( + get_type_hints(with_union, include_extras=True), + {'x': int | list[Annotated[str, 'meta']]}, + ) + def test_get_type_hints_annotated_refs(self): Const = Annotated[T, "Const"] From webhook-mailer at python.org Fri Feb 18 21:24:14 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Sat, 19 Feb 2022 02:24:14 -0000 Subject: [Python-checkins] Counter doc mentions three methods, but lists four (GH-30706) Message-ID: https://github.com/python/cpython/commit/99331fcf175b03b6b82451ac9aa953b476ab48e0 commit: 99331fcf175b03b6b82451ac9aa953b476ab48e0 branch: main author: Charles Brunet committer: JelleZijlstra date: 2022-02-18T18:23:53-08:00 summary: Counter doc mentions three methods, but lists four (GH-30706) Was probably caused by the addition of the `total()` method files: M Doc/library/collections.rst diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index b97bc425de9eb..c45d90cbbc1c7 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -276,7 +276,7 @@ For example:: according to when an element is first encountered in the left operand and then by the order encountered in the right operand. - Counter objects support three methods beyond those available for all + Counter objects support additional methods beyond those available for all dictionaries: .. method:: elements() From webhook-mailer at python.org Fri Feb 18 21:46:23 2022 From: webhook-mailer at python.org (miss-islington) Date: Sat, 19 Feb 2022 02:46:23 -0000 Subject: [Python-checkins] Counter doc mentions three methods, but lists four (GH-30706) Message-ID: https://github.com/python/cpython/commit/75c5dbc27eb6d02677543b06e9001523e03a169a commit: 75c5dbc27eb6d02677543b06e9001523e03a169a branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-18T18:45:56-08:00 summary: Counter doc mentions three methods, but lists four (GH-30706) Was probably caused by the addition of the `total()` method (cherry picked from commit 99331fcf175b03b6b82451ac9aa953b476ab48e0) Co-authored-by: Charles Brunet files: M Doc/library/collections.rst diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index b97bc425de9eb..c45d90cbbc1c7 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -276,7 +276,7 @@ For example:: according to when an element is first encountered in the left operand and then by the order encountered in the right operand. - Counter objects support three methods beyond those available for all + Counter objects support additional methods beyond those available for all dictionaries: .. method:: elements() From webhook-mailer at python.org Fri Feb 18 21:50:03 2022 From: webhook-mailer at python.org (miss-islington) Date: Sat, 19 Feb 2022 02:50:03 -0000 Subject: [Python-checkins] Counter doc mentions three methods, but lists four (GH-30706) Message-ID: https://github.com/python/cpython/commit/cf1993210fdb5b9140c7710ca1a93ca661400f19 commit: cf1993210fdb5b9140c7710ca1a93ca661400f19 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-18T18:49:48-08:00 summary: Counter doc mentions three methods, but lists four (GH-30706) Was probably caused by the addition of the `total()` method (cherry picked from commit 99331fcf175b03b6b82451ac9aa953b476ab48e0) Co-authored-by: Charles Brunet files: M Doc/library/collections.rst diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index e00b75fa9be19..f3b8e2abe5e5d 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -277,7 +277,7 @@ For example:: according to when an element is first encountered in the left operand and then by the order encountered in the right operand. - Counter objects support three methods beyond those available for all + Counter objects support additional methods beyond those available for all dictionaries: .. method:: elements() From webhook-mailer at python.org Fri Feb 18 23:15:34 2022 From: webhook-mailer at python.org (methane) Date: Sat, 19 Feb 2022 04:15:34 -0000 Subject: [Python-checkins] dict: Use DK_LOG_SIZE in hot loop. (GH-31405) Message-ID: https://github.com/python/cpython/commit/5543d9c5590c47b7734569d95c3a478f092b525d commit: 5543d9c5590c47b7734569d95c3a478f092b525d branch: main author: Inada Naoki committer: methane date: 2022-02-19T13:15:20+09:00 summary: dict: Use DK_LOG_SIZE in hot loop. (GH-31405) DK_LOG_SIZE(key) < 8 is faster than DK_SIZE(key) <= 0xff, at least on GCC. files: M Objects/dictobject.c diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 18de42cba6903..63e3eda49881a 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -320,19 +320,19 @@ dictkeys_decref(PyDictKeysObject *dk) static inline Py_ssize_t dictkeys_get_index(const PyDictKeysObject *keys, Py_ssize_t i) { - Py_ssize_t s = DK_SIZE(keys); + int log2size = DK_LOG_SIZE(keys); Py_ssize_t ix; - if (s <= 0xff) { + if (log2size < 8) { const int8_t *indices = (const int8_t*)(keys->dk_indices); ix = indices[i]; } - else if (s <= 0xffff) { + else if (log2size < 16) { const int16_t *indices = (const int16_t*)(keys->dk_indices); ix = indices[i]; } #if SIZEOF_VOID_P > 4 - else if (s > 0xffffffff) { + else if (log2size >= 32) { const int64_t *indices = (const int64_t*)(keys->dk_indices); ix = indices[i]; } @@ -349,23 +349,23 @@ dictkeys_get_index(const PyDictKeysObject *keys, Py_ssize_t i) static inline void dictkeys_set_index(PyDictKeysObject *keys, Py_ssize_t i, Py_ssize_t ix) { - Py_ssize_t s = DK_SIZE(keys); + int log2size = DK_LOG_SIZE(keys); assert(ix >= DKIX_DUMMY); assert(keys->dk_version == 0); - if (s <= 0xff) { + if (log2size < 8) { int8_t *indices = (int8_t*)(keys->dk_indices); assert(ix <= 0x7f); indices[i] = (char)ix; } - else if (s <= 0xffff) { + else if (log2size < 16) { int16_t *indices = (int16_t*)(keys->dk_indices); assert(ix <= 0x7fff); indices[i] = (int16_t)ix; } #if SIZEOF_VOID_P > 4 - else if (s > 0xffffffff) { + else if (log2size >= 32) { int64_t *indices = (int64_t*)(keys->dk_indices); indices[i] = ix; } @@ -631,7 +631,7 @@ free_keys_object(PyDictKeysObject *keys) // free_keys_object() must not be called after _PyDict_Fini() assert(state->keys_numfree != -1); #endif - if (DK_SIZE(keys) == PyDict_MINSIZE && state->keys_numfree < PyDict_MAXFREELIST) { + if (DK_LOG_SIZE(keys) == PyDict_LOG_MINSIZE && state->keys_numfree < PyDict_MAXFREELIST) { state->keys_free_list[state->keys_numfree++] = keys; return; } @@ -1196,7 +1196,7 @@ Internal routine used by dictresize() to build a hashtable of entries. static void build_indices(PyDictKeysObject *keys, PyDictKeyEntry *ep, Py_ssize_t n) { - size_t mask = (size_t)DK_SIZE(keys) - 1; + size_t mask = DK_MASK(keys); for (Py_ssize_t ix = 0; ix != n; ix++, ep++) { Py_hash_t hash = ep->me_hash; size_t i = hash & mask; @@ -1296,7 +1296,7 @@ dictresize(PyDictObject *mp, uint8_t log2_newsize) // dictresize() must not be called after _PyDict_Fini() assert(state->keys_numfree != -1); #endif - if (DK_SIZE(oldkeys) == PyDict_MINSIZE && + if (DK_LOG_SIZE(oldkeys) == PyDict_LOG_MINSIZE && state->keys_numfree < PyDict_MAXFREELIST) { state->keys_free_list[state->keys_numfree++] = oldkeys; @@ -2555,7 +2555,7 @@ dict_merge(PyObject *a, PyObject *b, int override) // If other is clean, combined, and just allocated, just clone it. if (other->ma_values == NULL && other->ma_used == okeys->dk_nentries && - (DK_SIZE(okeys) == PyDict_MINSIZE || + (DK_LOG_SIZE(okeys) == PyDict_LOG_MINSIZE || USABLE_FRACTION(DK_SIZE(okeys)/2) < other->ma_used)) { PyDictKeysObject *keys = clone_combined_dict_keys(other); if (keys == NULL) { From webhook-mailer at python.org Sat Feb 19 00:57:40 2022 From: webhook-mailer at python.org (sweeneyde) Date: Sat, 19 Feb 2022 05:57:40 -0000 Subject: [Python-checkins] Use raw string to avoid deprecation warning (GH-31427) Message-ID: https://github.com/python/cpython/commit/7a4791e03613bfbdc0d3ddfabfc0b59e6a6f7358 commit: 7a4791e03613bfbdc0d3ddfabfc0b59e6a6f7358 branch: main author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: sweeneyde <36520290+sweeneyde at users.noreply.github.com> date: 2022-02-19T00:57:36-05:00 summary: Use raw string to avoid deprecation warning (GH-31427) files: M Lib/test/test_property.py diff --git a/Lib/test/test_property.py b/Lib/test/test_property.py index 7d1c4a1e12880..d91ad1c191275 100644 --- a/Lib/test/test_property.py +++ b/Lib/test/test_property.py @@ -342,7 +342,7 @@ class cls: class PropertyUnreachableAttributeNoName(_PropertyUnreachableAttribute, unittest.TestCase): - msg_format = "^property of 'PropertyUnreachableAttributeNoName\.cls' object {}$" + msg_format = r"^property of 'PropertyUnreachableAttributeNoName\.cls' object {}$" class cls: pass From webhook-mailer at python.org Sat Feb 19 20:44:59 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Sun, 20 Feb 2022 01:44:59 -0000 Subject: [Python-checkins] bpo-46066: Check DeprecationWarning in test_typing (GH-31428) Message-ID: https://github.com/python/cpython/commit/0a8a8e7454c6565cf1554d5f23314e4c70960bcd commit: 0a8a8e7454c6565cf1554d5f23314e4c70960bcd branch: main author: Jelle Zijlstra committer: JelleZijlstra date: 2022-02-19T17:44:51-08:00 summary: bpo-46066: Check DeprecationWarning in test_typing (GH-31428) files: M Lib/test/test_typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index b38e27c5f9047..dc1514d63b777 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -4585,8 +4585,6 @@ def test_typeddict_create_errors(self): with self.assertRaises(TypeError): TypedDict(_typename='Emp', name=str, id=int) - with self.assertRaises(TypeError): - TypedDict('Emp', _fields={'name': str, 'id': int}) def test_typeddict_errors(self): Emp = TypedDict('Emp', {'name': str, 'id': int}) @@ -4598,8 +4596,11 @@ def test_typeddict_errors(self): isinstance(jim, Emp) with self.assertRaises(TypeError): issubclass(dict, Emp) - with self.assertRaises(TypeError): - TypedDict('Hi', x=1) + # We raise a DeprecationWarning for the keyword syntax + # before the TypeError. + with self.assertWarns(DeprecationWarning): + with self.assertRaises(TypeError): + TypedDict('Hi', x=1) with self.assertRaises(TypeError): TypedDict('Hi', [('x', int), ('y', 1)]) with self.assertRaises(TypeError): From webhook-mailer at python.org Sun Feb 20 00:02:38 2022 From: webhook-mailer at python.org (rhettinger) Date: Sun, 20 Feb 2022 05:02:38 -0000 Subject: [Python-checkins] Improve discussion about how __getattr__ is invoked. (GH-31435) Message-ID: https://github.com/python/cpython/commit/12a2e41e8a276ec3a68f3e5f94b02752185c66fb commit: 12a2e41e8a276ec3a68f3e5f94b02752185c66fb branch: main author: Raymond Hettinger committer: rhettinger date: 2022-02-19T23:02:30-06:00 summary: Improve discussion about how __getattr__ is invoked. (GH-31435) files: M Doc/howto/descriptor.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 4f6389e3b2d48..825e9d0347f5d 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -696,10 +696,14 @@ a pure Python equivalent: >>> b.g == b['g'] == ('getattr_hook', b, 'g') True +Note, there is no :meth:`__getattr__` hook in the :meth:`__getattribute__` +code. That is why calling :meth:`__getattribute__` directly or with +``super().__getattribute__`` will bypass :meth:`__getattr__` entirely. -Interestingly, attribute lookup doesn't call :meth:`object.__getattribute__` -directly. Instead, both the dot operator and the :func:`getattr` function -perform attribute lookup by way of a helper function: +Instead, it is the dot operator and the :func:`getattr` function that are +responsible for invoking :meth:`__getattr__` whenever :meth:`__getattribute__` +raises an :exc:`AttributeError`. Their logic is encapsulated in a helper +function: .. testcode:: @@ -744,12 +748,6 @@ perform attribute lookup by way of a helper function: ... AttributeError: 'ClassWithoutGetAttr' object has no attribute 'z' -So if :meth:`__getattr__` exists, it is called whenever :meth:`__getattribute__` -raises :exc:`AttributeError` (either directly or in one of the descriptor calls). - -Also, if a user calls :meth:`object.__getattribute__` directly, the -:meth:`__getattr__` hook is bypassed entirely. - Invocation from a class ----------------------- From webhook-mailer at python.org Sun Feb 20 00:26:07 2022 From: webhook-mailer at python.org (rhettinger) Date: Sun, 20 Feb 2022 05:26:07 -0000 Subject: [Python-checkins] Improve discussion about how __getattr__ is invoked. (GH-31435) (GH-31437) Message-ID: https://github.com/python/cpython/commit/ea3e0421b04a6081d604786a40e7f27ff854b428 commit: ea3e0421b04a6081d604786a40e7f27ff854b428 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: rhettinger date: 2022-02-19T23:25:57-06:00 summary: Improve discussion about how __getattr__ is invoked. (GH-31435) (GH-31437) files: M Doc/howto/descriptor.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index f8b1e00d96fad..f2e2f7ee68c2e 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -696,10 +696,14 @@ a pure Python equivalent: >>> b.g == b['g'] == ('getattr_hook', b, 'g') True +Note, there is no :meth:`__getattr__` hook in the :meth:`__getattribute__` +code. That is why calling :meth:`__getattribute__` directly or with +``super().__getattribute__`` will bypass :meth:`__getattr__` entirely. -Interestingly, attribute lookup doesn't call :meth:`object.__getattribute__` -directly. Instead, both the dot operator and the :func:`getattr` function -perform attribute lookup by way of a helper function: +Instead, it is the dot operator and the :func:`getattr` function that are +responsible for invoking :meth:`__getattr__` whenever :meth:`__getattribute__` +raises an :exc:`AttributeError`. Their logic is encapsulated in a helper +function: .. testcode:: @@ -744,12 +748,6 @@ perform attribute lookup by way of a helper function: ... AttributeError: 'ClassWithoutGetAttr' object has no attribute 'z' -So if :meth:`__getattr__` exists, it is called whenever :meth:`__getattribute__` -raises :exc:`AttributeError` (either directly or in one of the descriptor calls). - -Also, if a user calls :meth:`object.__getattribute__` directly, the -:meth:`__getattr__` hook is bypassed entirely. - Invocation from a class ----------------------- From webhook-mailer at python.org Sun Feb 20 05:07:16 2022 From: webhook-mailer at python.org (asvetlov) Date: Sun, 20 Feb 2022 10:07:16 -0000 Subject: [Python-checkins] bpo-46752: Uniform TaskGroup.__repr__ (GH-31409) Message-ID: https://github.com/python/cpython/commit/e7130c2e8c6abfaf04b209bd5b239059eda024b9 commit: e7130c2e8c6abfaf04b209bd5b239059eda024b9 branch: main author: Andrew Svetlov committer: asvetlov date: 2022-02-20T12:07:00+02:00 summary: bpo-46752: Uniform TaskGroup.__repr__ (GH-31409) files: M Lib/asyncio/taskgroups.py diff --git a/Lib/asyncio/taskgroups.py b/Lib/asyncio/taskgroups.py index 57b0eafefc16f..756fc551e013d 100644 --- a/Lib/asyncio/taskgroups.py +++ b/Lib/asyncio/taskgroups.py @@ -9,6 +9,7 @@ from . import exceptions from . import tasks + class TaskGroup: def __init__(self): @@ -25,19 +26,20 @@ def __init__(self): self._on_completed_fut = None def __repr__(self): - msg = f'' async def __aenter__(self): if self._entered: From webhook-mailer at python.org Sun Feb 20 05:24:18 2022 From: webhook-mailer at python.org (asvetlov) Date: Sun, 20 Feb 2022 10:24:18 -0000 Subject: [Python-checkins] bpo-46672: fix `NameError` in `asyncio.gather` if type check fails (GH-31187) Message-ID: https://github.com/python/cpython/commit/4ab8167b9c60d1a04b2e3116d0c52db254b68cda commit: 4ab8167b9c60d1a04b2e3116d0c52db254b68cda branch: main author: Nikita Sobolev committer: asvetlov date: 2022-02-20T12:24:00+02:00 summary: bpo-46672: fix `NameError` in `asyncio.gather` if type check fails (GH-31187) Co-authored-by: Alex Waygood files: A Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst M Lib/asyncio/tasks.py M Lib/test/test_asyncio/test_tasks.py diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py index c11d0daaefea7..25a650ffbb744 100644 --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -735,7 +735,7 @@ def _done_callback(fut): nonlocal nfinished nfinished += 1 - if outer.done(): + if outer is None or outer.done(): if not fut.cancelled(): # Mark exception retrieved. fut.exception() @@ -791,6 +791,7 @@ def _done_callback(fut): nfuts = 0 nfinished = 0 loop = None + outer = None # bpo-46672 for arg in coros_or_futures: if arg not in arg_to_fut: fut = _ensure_future(arg, loop=loop) diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index fe6bfb363f1c6..40b33deb17ee0 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -3235,6 +3235,20 @@ async def outer(): test_utils.run_briefly(self.one_loop) self.assertIsInstance(f.exception(), RuntimeError) + def test_issue46672(self): + with mock.patch( + 'asyncio.base_events.BaseEventLoop.call_exception_handler', + ): + async def coro(s): + return s + c = coro('abc') + + with self.assertRaises(TypeError): + self._gather(c, {}) + self._run_loop(self.one_loop) + # NameError should not happen: + self.one_loop.call_exception_handler.assert_not_called() + class RunCoroutineThreadsafeTests(test_utils.TestCase): """Test case for asyncio.run_coroutine_threadsafe.""" diff --git a/Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst b/Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst new file mode 100644 index 0000000000000..9a76c29a334d8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst @@ -0,0 +1 @@ +Fix ``NameError`` in :func:`asyncio.gather` when initial type check fails. From webhook-mailer at python.org Sun Feb 20 07:17:30 2022 From: webhook-mailer at python.org (asvetlov) Date: Sun, 20 Feb 2022 12:17:30 -0000 Subject: [Python-checkins] Raise TypeError if SSLSocket is passed to asyncio transport-based methods (GH-31442) Message-ID: https://github.com/python/cpython/commit/1f9d4c93af380d00bf2e24bc5f5ce662d41504aa commit: 1f9d4c93af380d00bf2e24bc5f5ce662d41504aa branch: main author: Andrew Svetlov committer: asvetlov date: 2022-02-20T14:17:15+02:00 summary: Raise TypeError if SSLSocket is passed to asyncio transport-based methods (GH-31442) files: A Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst M Lib/asyncio/base_events.py M Lib/asyncio/selector_events.py diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index 703c8a4ce2408..f9215c5e0182a 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -198,6 +198,11 @@ def _set_nodelay(sock): pass +def _check_ssl_socket(sock): + if ssl is not None and isinstance(sock, ssl.SSLSocket): + raise TypeError("Socket cannot be of type SSLSocket") + + class _SendfileFallbackProtocol(protocols.Protocol): def __init__(self, transp): if not isinstance(transp, transports._FlowControlMixin): @@ -862,6 +867,7 @@ async def sock_sendfile(self, sock, file, offset=0, count=None, *, fallback=True): if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") + _check_ssl_socket(sock) self._check_sendfile_params(sock, file, offset, count) try: return await self._sock_sendfile_native(sock, file, @@ -1008,6 +1014,9 @@ async def create_connection( raise ValueError( 'ssl_shutdown_timeout is only meaningful with ssl') + if sock is not None: + _check_ssl_socket(sock) + if happy_eyeballs_delay is not None and interleave is None: # If using happy eyeballs, default to interleave addresses by family interleave = 1 @@ -1438,6 +1447,9 @@ async def create_server( raise ValueError( 'ssl_shutdown_timeout is only meaningful with ssl') + if sock is not None: + _check_ssl_socket(sock) + if host is not None or port is not None: if sock is not None: raise ValueError( @@ -1538,6 +1550,9 @@ async def connect_accepted_socket( raise ValueError( 'ssl_shutdown_timeout is only meaningful with ssl') + if sock is not None: + _check_ssl_socket(sock) + transport, protocol = await self._create_connection_transport( sock, protocol_factory, ssl, '', server_side=True, ssl_handshake_timeout=ssl_handshake_timeout, diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index 63ab15f30fb5d..ebb5cbec03e4b 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -40,11 +40,6 @@ def _test_selector_event(selector, fd, event): return bool(key.events & event) -def _check_ssl_socket(sock): - if ssl is not None and isinstance(sock, ssl.SSLSocket): - raise TypeError("Socket cannot be of type SSLSocket") - - class BaseSelectorEventLoop(base_events.BaseEventLoop): """Selector event loop. @@ -366,7 +361,7 @@ async def sock_recv(self, sock, n): The maximum amount of data to be received at once is specified by nbytes. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") try: @@ -407,7 +402,7 @@ async def sock_recv_into(self, sock, buf): The received data is written into *buf* (a writable buffer). The return value is the number of bytes written. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") try: @@ -448,7 +443,7 @@ async def sock_sendall(self, sock, data): raised, and there is no way to determine how much data, if any, was successfully processed by the receiving end of the connection. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") try: @@ -497,7 +492,7 @@ async def sock_connect(self, sock, address): This method is a coroutine. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") @@ -562,7 +557,7 @@ async def sock_accept(self, sock): object usable to send and receive data on the connection, and address is the address bound to the socket on the other end of the connection. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") fut = self.create_future() diff --git a/Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst b/Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst new file mode 100644 index 0000000000000..a15e7aaaa3389 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst @@ -0,0 +1,2 @@ +Raise :exc:`TypeError` if :class:`ssl.SSLSocket` is passed to +transport-based APIs. From webhook-mailer at python.org Sun Feb 20 07:31:35 2022 From: webhook-mailer at python.org (asvetlov) Date: Sun, 20 Feb 2022 12:31:35 -0000 Subject: [Python-checkins] replace `self` param with more appropriate `cls` in classmethods (GH-31402) Message-ID: https://github.com/python/cpython/commit/a3fcca4af1cb418dc802feb75100ecc1a286afaa commit: a3fcca4af1cb418dc802feb75100ecc1a286afaa branch: main author: Josh Smith committer: asvetlov date: 2022-02-20T14:31:09+02:00 summary: replace `self` param with more appropriate `cls` in classmethods (GH-31402) files: M Lib/_collections_abc.py diff --git a/Lib/_collections_abc.py b/Lib/_collections_abc.py index 97913c77721da..40417dc1d3133 100644 --- a/Lib/_collections_abc.py +++ b/Lib/_collections_abc.py @@ -871,7 +871,7 @@ class KeysView(MappingView, Set): __slots__ = () @classmethod - def _from_iterable(self, it): + def _from_iterable(cls, it): return set(it) def __contains__(self, key): @@ -889,7 +889,7 @@ class ItemsView(MappingView, Set): __slots__ = () @classmethod - def _from_iterable(self, it): + def _from_iterable(cls, it): return set(it) def __contains__(self, item): From webhook-mailer at python.org Sun Feb 20 07:39:36 2022 From: webhook-mailer at python.org (asvetlov) Date: Sun, 20 Feb 2022 12:39:36 -0000 Subject: [Python-checkins] [3.9] bpo-46672: fix `NameError` in `asyncio.gather` if type check fails (GH-31187) (GH-31441) Message-ID: https://github.com/python/cpython/commit/a6116a980c9eae91c2f9af7cbd0a9727e9b887ba commit: a6116a980c9eae91c2f9af7cbd0a9727e9b887ba branch: 3.9 author: Andrew Svetlov committer: asvetlov date: 2022-02-20T14:39:21+02:00 summary: [3.9] bpo-46672: fix `NameError` in `asyncio.gather` if type check fails (GH-31187) (GH-31441) Co-authored-by: Nikita Sobolev files: A Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst M Lib/asyncio/tasks.py M Lib/test/test_asyncio/test_tasks.py diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py index 39bd068535668..53252f2079d7c 100644 --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -768,7 +768,7 @@ def _done_callback(fut): nonlocal nfinished nfinished += 1 - if outer.done(): + if outer is None or outer.done(): if not fut.cancelled(): # Mark exception retrieved. fut.exception() @@ -823,6 +823,7 @@ def _done_callback(fut): children = [] nfuts = 0 nfinished = 0 + outer = None # bpo-46672 for arg in coros_or_futures: if arg not in arg_to_fut: fut = ensure_future(arg, loop=loop) diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index 2f016740b9e76..0e538edcd9678 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -3407,6 +3407,11 @@ async def coro(fut=fut): coros.append(coro()) return coros + def _gather(self, *args, **kwargs): + async def coro(): + return asyncio.gather(*args, **kwargs) + return self.one_loop.run_until_complete(coro()) + def test_constructor_loop_selection(self): async def coro(): return 'abc' @@ -3488,6 +3493,20 @@ async def outer(): test_utils.run_briefly(self.one_loop) self.assertIsInstance(f.exception(), RuntimeError) + def test_issue46672(self): + with mock.patch( + 'asyncio.base_events.BaseEventLoop.call_exception_handler', + ): + async def coro(s): + return s + c = coro('abc') + + with self.assertRaises(TypeError): + self._gather(c, {}) + self._run_loop(self.one_loop) + # NameError should not happen: + self.one_loop.call_exception_handler.assert_not_called() + class RunCoroutineThreadsafeTests(test_utils.TestCase): """Test case for asyncio.run_coroutine_threadsafe.""" diff --git a/Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst b/Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst new file mode 100644 index 0000000000000..9a76c29a334d8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst @@ -0,0 +1 @@ +Fix ``NameError`` in :func:`asyncio.gather` when initial type check fails. From webhook-mailer at python.org Sun Feb 20 07:45:11 2022 From: webhook-mailer at python.org (asvetlov) Date: Sun, 20 Feb 2022 12:45:11 -0000 Subject: [Python-checkins] [3.10] Raise TypeError if SSLSocket is passed to asyncio transport-based methods (GH-31442). (GH-31443) Message-ID: https://github.com/python/cpython/commit/dde048819f3975bdf98b3e50ea9ef8ea25ecdb8e commit: dde048819f3975bdf98b3e50ea9ef8ea25ecdb8e branch: 3.10 author: Andrew Svetlov committer: asvetlov date: 2022-02-20T14:45:02+02:00 summary: [3.10] Raise TypeError if SSLSocket is passed to asyncio transport-based methods (GH-31442). (GH-31443) (cherry picked from commit 1f9d4c93af380d00bf2e24bc5f5ce662d41504aa) Co-authored-by: Andrew Svetlov files: A Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst M Lib/asyncio/base_events.py M Lib/asyncio/selector_events.py diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index f726e91e889b7..f51613d3b7dfa 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -202,6 +202,11 @@ def _set_nodelay(sock): pass +def _check_ssl_socket(sock): + if ssl is not None and isinstance(sock, ssl.SSLSocket): + raise TypeError("Socket cannot be of type SSLSocket") + + class _SendfileFallbackProtocol(protocols.Protocol): def __init__(self, transp): if not isinstance(transp, transports._FlowControlMixin): @@ -863,6 +868,7 @@ async def sock_sendfile(self, sock, file, offset=0, count=None, *, fallback=True): if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") + _check_ssl_socket(sock) self._check_sendfile_params(sock, file, offset, count) try: return await self._sock_sendfile_native(sock, file, @@ -1004,6 +1010,9 @@ async def create_connection( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if sock is not None: + _check_ssl_socket(sock) + if happy_eyeballs_delay is not None and interleave is None: # If using happy eyeballs, default to interleave addresses by family interleave = 1 @@ -1437,6 +1446,9 @@ async def create_server( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if sock is not None: + _check_ssl_socket(sock) + if host is not None or port is not None: if sock is not None: raise ValueError( @@ -1531,6 +1543,9 @@ async def connect_accepted_socket( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if sock is not None: + _check_ssl_socket(sock) + transport, protocol = await self._create_connection_transport( sock, protocol_factory, ssl, '', server_side=True, ssl_handshake_timeout=ssl_handshake_timeout) diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index 59cb6b1babec5..6f764450ae3be 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -40,11 +40,6 @@ def _test_selector_event(selector, fd, event): return bool(key.events & event) -def _check_ssl_socket(sock): - if ssl is not None and isinstance(sock, ssl.SSLSocket): - raise TypeError("Socket cannot be of type SSLSocket") - - class BaseSelectorEventLoop(base_events.BaseEventLoop): """Selector event loop. @@ -357,7 +352,7 @@ async def sock_recv(self, sock, n): The maximum amount of data to be received at once is specified by nbytes. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") try: @@ -398,7 +393,7 @@ async def sock_recv_into(self, sock, buf): The received data is written into *buf* (a writable buffer). The return value is the number of bytes written. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") try: @@ -439,7 +434,7 @@ async def sock_sendall(self, sock, data): raised, and there is no way to determine how much data, if any, was successfully processed by the receiving end of the connection. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") try: @@ -488,7 +483,7 @@ async def sock_connect(self, sock, address): This method is a coroutine. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") @@ -553,7 +548,7 @@ async def sock_accept(self, sock): object usable to send and receive data on the connection, and address is the address bound to the socket on the other end of the connection. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") fut = self.create_future() diff --git a/Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst b/Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst new file mode 100644 index 0000000000000..a15e7aaaa3389 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst @@ -0,0 +1,2 @@ +Raise :exc:`TypeError` if :class:`ssl.SSLSocket` is passed to +transport-based APIs. From webhook-mailer at python.org Sun Feb 20 07:49:31 2022 From: webhook-mailer at python.org (asvetlov) Date: Sun, 20 Feb 2022 12:49:31 -0000 Subject: [Python-checkins] [3.9] Raise TypeError if SSLSocket is passed to asyncio transport-based methods (GH-31442) (GH-31444) Message-ID: https://github.com/python/cpython/commit/64705e6a94a28abeeafeddb8e9a68a8500e1d91b commit: 64705e6a94a28abeeafeddb8e9a68a8500e1d91b branch: 3.9 author: Andrew Svetlov committer: asvetlov date: 2022-02-20T14:49:27+02:00 summary: [3.9] Raise TypeError if SSLSocket is passed to asyncio transport-based methods (GH-31442) (GH-31444) (cherry picked from commit 1f9d4c93af380d00bf2e24bc5f5ce662d41504aa) Co-authored-by: Andrew Svetlov files: A Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst M Lib/asyncio/base_events.py M Lib/asyncio/selector_events.py diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index 8c1fb49694875..8a380bb80ea06 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -202,6 +202,11 @@ def _set_nodelay(sock): pass +def _check_ssl_socket(sock): + if ssl is not None and isinstance(sock, ssl.SSLSocket): + raise TypeError("Socket cannot be of type SSLSocket") + + class _SendfileFallbackProtocol(protocols.Protocol): def __init__(self, transp): if not isinstance(transp, transports._FlowControlMixin): @@ -864,6 +869,7 @@ async def sock_sendfile(self, sock, file, offset=0, count=None, *, fallback=True): if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") + _check_ssl_socket(sock) self._check_sendfile_params(sock, file, offset, count) try: return await self._sock_sendfile_native(sock, file, @@ -1005,6 +1011,9 @@ async def create_connection( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if sock is not None: + _check_ssl_socket(sock) + if happy_eyeballs_delay is not None and interleave is None: # If using happy eyeballs, default to interleave addresses by family interleave = 1 @@ -1438,6 +1447,9 @@ async def create_server( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if sock is not None: + _check_ssl_socket(sock) + if host is not None or port is not None: if sock is not None: raise ValueError( @@ -1540,6 +1552,9 @@ async def connect_accepted_socket( raise ValueError( 'ssl_handshake_timeout is only meaningful with ssl') + if sock is not None: + _check_ssl_socket(sock) + transport, protocol = await self._create_connection_transport( sock, protocol_factory, ssl, '', server_side=True, ssl_handshake_timeout=ssl_handshake_timeout) diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index 59cb6b1babec5..6f764450ae3be 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -40,11 +40,6 @@ def _test_selector_event(selector, fd, event): return bool(key.events & event) -def _check_ssl_socket(sock): - if ssl is not None and isinstance(sock, ssl.SSLSocket): - raise TypeError("Socket cannot be of type SSLSocket") - - class BaseSelectorEventLoop(base_events.BaseEventLoop): """Selector event loop. @@ -357,7 +352,7 @@ async def sock_recv(self, sock, n): The maximum amount of data to be received at once is specified by nbytes. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") try: @@ -398,7 +393,7 @@ async def sock_recv_into(self, sock, buf): The received data is written into *buf* (a writable buffer). The return value is the number of bytes written. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") try: @@ -439,7 +434,7 @@ async def sock_sendall(self, sock, data): raised, and there is no way to determine how much data, if any, was successfully processed by the receiving end of the connection. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") try: @@ -488,7 +483,7 @@ async def sock_connect(self, sock, address): This method is a coroutine. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") @@ -553,7 +548,7 @@ async def sock_accept(self, sock): object usable to send and receive data on the connection, and address is the address bound to the socket on the other end of the connection. """ - _check_ssl_socket(sock) + base_events._check_ssl_socket(sock) if self._debug and sock.gettimeout() != 0: raise ValueError("the socket must be non-blocking") fut = self.create_future() diff --git a/Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst b/Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst new file mode 100644 index 0000000000000..a15e7aaaa3389 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-20-12-59-46.bpo-46252.KG1SqA.rst @@ -0,0 +1,2 @@ +Raise :exc:`TypeError` if :class:`ssl.SSLSocket` is passed to +transport-based APIs. From webhook-mailer at python.org Sun Feb 20 08:37:24 2022 From: webhook-mailer at python.org (rhettinger) Date: Sun, 20 Feb 2022 13:37:24 -0000 Subject: [Python-checkins] Improve discussion about how __getattr__ is invoked. (GH-31435) (GH-31438) Message-ID: https://github.com/python/cpython/commit/83eabc6c8f6757f277d26064a83a626c4779424e commit: 83eabc6c8f6757f277d26064a83a626c4779424e branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: rhettinger date: 2022-02-20T07:37:00-06:00 summary: Improve discussion about how __getattr__ is invoked. (GH-31435) (GH-31438) files: M Doc/howto/descriptor.rst diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index 9f0dd2f1f1550..4b76e061c35a5 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -696,10 +696,14 @@ a pure Python equivalent: >>> b.g == b['g'] == ('getattr_hook', b, 'g') True +Note, there is no :meth:`__getattr__` hook in the :meth:`__getattribute__` +code. That is why calling :meth:`__getattribute__` directly or with +``super().__getattribute__`` will bypass :meth:`__getattr__` entirely. -Interestingly, attribute lookup doesn't call :meth:`object.__getattribute__` -directly. Instead, both the dot operator and the :func:`getattr` function -perform attribute lookup by way of a helper function: +Instead, it is the dot operator and the :func:`getattr` function that are +responsible for invoking :meth:`__getattr__` whenever :meth:`__getattribute__` +raises an :exc:`AttributeError`. Their logic is encapsulated in a helper +function: .. testcode:: @@ -744,12 +748,6 @@ perform attribute lookup by way of a helper function: ... AttributeError: 'ClassWithoutGetAttr' object has no attribute 'z' -So if :meth:`__getattr__` exists, it is called whenever :meth:`__getattribute__` -raises :exc:`AttributeError` (either directly or in one of the descriptor calls). - -Also, if a user calls :meth:`object.__getattribute__` directly, the -:meth:`__getattr__` hook is bypassed entirely. - Invocation from a class ----------------------- From webhook-mailer at python.org Sun Feb 20 08:45:24 2022 From: webhook-mailer at python.org (asvetlov) Date: Sun, 20 Feb 2022 13:45:24 -0000 Subject: [Python-checkins] [3.10] replace `self` param with more appropriate `cls` in classmethods (GH-31402) (GH-31446) Message-ID: https://github.com/python/cpython/commit/fa621a738875dc8a644a6149e7ffd4be0a40f1d9 commit: fa621a738875dc8a644a6149e7ffd4be0a40f1d9 branch: 3.10 author: Andrew Svetlov committer: asvetlov date: 2022-02-20T15:45:13+02:00 summary: [3.10] replace `self` param with more appropriate `cls` in classmethods (GH-31402) (GH-31446) (cherry picked from commit a3fcca4af1cb418dc802feb75100ecc1a286afaa) Co-authored-by: Josh Smith Co-authored-by: Josh Smith files: M Lib/_collections_abc.py diff --git a/Lib/_collections_abc.py b/Lib/_collections_abc.py index 97913c77721da..40417dc1d3133 100644 --- a/Lib/_collections_abc.py +++ b/Lib/_collections_abc.py @@ -871,7 +871,7 @@ class KeysView(MappingView, Set): __slots__ = () @classmethod - def _from_iterable(self, it): + def _from_iterable(cls, it): return set(it) def __contains__(self, key): @@ -889,7 +889,7 @@ class ItemsView(MappingView, Set): __slots__ = () @classmethod - def _from_iterable(self, it): + def _from_iterable(cls, it): return set(it) def __contains__(self, item): From webhook-mailer at python.org Sun Feb 20 10:29:09 2022 From: webhook-mailer at python.org (asvetlov) Date: Sun, 20 Feb 2022 15:29:09 -0000 Subject: [Python-checkins] bpo-46672: fix `NameError` in `asyncio.gather` if type check fails (GH-31187) (GH-31440) Message-ID: https://github.com/python/cpython/commit/f1916cde24053f4c8b6799730666d19474f8dd09 commit: f1916cde24053f4c8b6799730666d19474f8dd09 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: asvetlov date: 2022-02-20T17:28:42+02:00 summary: bpo-46672: fix `NameError` in `asyncio.gather` if type check fails (GH-31187) (GH-31440) Co-authored-by: Alex Waygood (cherry picked from commit 4ab8167b9c60d1a04b2e3116d0c52db254b68cda) Co-authored-by: Nikita Sobolev Co-authored-by: Nikita Sobolev files: A Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst M Lib/asyncio/tasks.py M Lib/test/test_asyncio/test_tasks.py diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py index 2bee5c050ded7..c4bedb5c72b0e 100644 --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -721,7 +721,7 @@ def _done_callback(fut): nonlocal nfinished nfinished += 1 - if outer.done(): + if outer is None or outer.done(): if not fut.cancelled(): # Mark exception retrieved. fut.exception() @@ -777,6 +777,7 @@ def _done_callback(fut): nfuts = 0 nfinished = 0 loop = None + outer = None # bpo-46672 for arg in coros_or_futures: if arg not in arg_to_fut: fut = _ensure_future(arg, loop=loop) diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index 4782c92a7c155..398b143d2d0c6 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -3593,6 +3593,20 @@ async def outer(): test_utils.run_briefly(self.one_loop) self.assertIsInstance(f.exception(), RuntimeError) + def test_issue46672(self): + with mock.patch( + 'asyncio.base_events.BaseEventLoop.call_exception_handler', + ): + async def coro(s): + return s + c = coro('abc') + + with self.assertRaises(TypeError): + self._gather(c, {}) + self._run_loop(self.one_loop) + # NameError should not happen: + self.one_loop.call_exception_handler.assert_not_called() + class RunCoroutineThreadsafeTests(test_utils.TestCase): """Test case for asyncio.run_coroutine_threadsafe.""" diff --git a/Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst b/Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst new file mode 100644 index 0000000000000..9a76c29a334d8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-07-13-15-16.bpo-46672.4swIjx.rst @@ -0,0 +1 @@ +Fix ``NameError`` in :func:`asyncio.gather` when initial type check fails. From webhook-mailer at python.org Sun Feb 20 11:02:25 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sun, 20 Feb 2022 16:02:25 -0000 Subject: [Python-checkins] bpo-39327: Close file descriptors as soon as possible in shutil.rmtree (GH-31384) Message-ID: https://github.com/python/cpython/commit/b77158b4da449ec5b8f682816a79d004fd65ed07 commit: b77158b4da449ec5b8f682816a79d004fd65ed07 branch: main author: Lital Natan committer: serhiy-storchaka date: 2022-02-20T18:02:10+02:00 summary: bpo-39327: Close file descriptors as soon as possible in shutil.rmtree (GH-31384) It fixes the "Text File Busy" OSError when using 'rmtree' on a windows-managed filesystem in via the VirtualBox shared folder (and possible other scenarios like a windows-managed network file system). files: A Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst M Lib/shutil.py M Misc/ACKS diff --git a/Lib/shutil.py b/Lib/shutil.py index 949e024853c1d..eb768f9e03b7d 100644 --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -648,6 +648,7 @@ def _rmtree_safe_fd(topfd, path, onerror): if is_dir: try: dirfd = os.open(entry.name, os.O_RDONLY, dir_fd=topfd) + dirfd_closed = False except OSError: onerror(os.open, fullname, sys.exc_info()) else: @@ -655,6 +656,8 @@ def _rmtree_safe_fd(topfd, path, onerror): if os.path.samestat(orig_st, os.fstat(dirfd)): _rmtree_safe_fd(dirfd, fullname, onerror) try: + os.close(dirfd) + dirfd_closed = True os.rmdir(entry.name, dir_fd=topfd) except OSError: onerror(os.rmdir, fullname, sys.exc_info()) @@ -668,7 +671,8 @@ def _rmtree_safe_fd(topfd, path, onerror): except OSError: onerror(os.path.islink, fullname, sys.exc_info()) finally: - os.close(dirfd) + if not dirfd_closed: + os.close(dirfd) else: try: os.unlink(entry.name, dir_fd=topfd) @@ -711,6 +715,7 @@ def onerror(*args): return try: fd = os.open(path, os.O_RDONLY) + fd_closed = False except Exception: onerror(os.open, path, sys.exc_info()) return @@ -718,6 +723,8 @@ def onerror(*args): if os.path.samestat(orig_st, os.fstat(fd)): _rmtree_safe_fd(fd, path, onerror) try: + os.close(fd) + fd_closed = True os.rmdir(path) except OSError: onerror(os.rmdir, path, sys.exc_info()) @@ -728,7 +735,8 @@ def onerror(*args): except OSError: onerror(os.path.islink, path, sys.exc_info()) finally: - os.close(fd) + if not fd_closed: + os.close(fd) else: try: if _rmtree_islink(path): diff --git a/Misc/ACKS b/Misc/ACKS index dceb2b628eb20..bab04b4613646 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -429,6 +429,7 @@ Caleb Deveraux Catherine Devlin Scott Dial Alon Diamant +Lital Natan Toby Dickenson Mark Dickinson Jack Diederich diff --git a/Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst b/Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst new file mode 100644 index 0000000000000..fc6e8250922ff --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst @@ -0,0 +1,2 @@ +:func:`shutil.rmtree` can now work with VirtualBox shared folders when +running from the guest operating-system. From webhook-mailer at python.org Sun Feb 20 15:42:43 2022 From: webhook-mailer at python.org (tiran) Date: Sun, 20 Feb 2022 20:42:43 -0000 Subject: [Python-checkins] bpo-46232: Fix parsing of certs with bit string in DN (GH-30351) Message-ID: https://github.com/python/cpython/commit/be095f6c32188bba02079d086ac8639ea37cec3c commit: be095f6c32188bba02079d086ac8639ea37cec3c branch: main author: Christian Heimes committer: tiran date: 2022-02-20T21:42:31+01:00 summary: bpo-46232: Fix parsing of certs with bit string in DN (GH-30351) files: A Misc/NEWS.d/next/Library/2022-01-03-09-46-44.bpo-46232.s0KlyI.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Library/2022-01-03-09-46-44.bpo-46232.s0KlyI.rst b/Misc/NEWS.d/next/Library/2022-01-03-09-46-44.bpo-46232.s0KlyI.rst new file mode 100644 index 0000000000000..e252449199a05 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-03-09-46-44.bpo-46232.s0KlyI.rst @@ -0,0 +1,2 @@ +The :mod:`ssl` module now handles certificates with bit strings in DN +correctly. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index d7e041fed3cb3..312b2eabe3db5 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1053,17 +1053,29 @@ _create_tuple_for_attribute(_sslmodulestate *state, ASN1_OBJECT *name, ASN1_STRING *value) { Py_ssize_t buflen; - unsigned char *valuebuf = NULL; - PyObject *attr; + PyObject *pyattr; + PyObject *pyname = _asn1obj2py(state, name, 0); - buflen = ASN1_STRING_to_UTF8(&valuebuf, value); - if (buflen < 0) { + if (pyname == NULL) { _setSSLError(state, NULL, 0, __FILE__, __LINE__); return NULL; } - attr = Py_BuildValue("Ns#", _asn1obj2py(state, name, 0), valuebuf, buflen); - OPENSSL_free(valuebuf); - return attr; + + if (ASN1_STRING_type(value) == V_ASN1_BIT_STRING) { + buflen = ASN1_STRING_length(value); + pyattr = Py_BuildValue("Ny#", pyname, ASN1_STRING_get0_data(value), buflen); + } else { + unsigned char *valuebuf = NULL; + buflen = ASN1_STRING_to_UTF8(&valuebuf, value); + if (buflen < 0) { + _setSSLError(state, NULL, 0, __FILE__, __LINE__); + Py_DECREF(pyname); + return NULL; + } + pyattr = Py_BuildValue("Ns#", pyname, valuebuf, buflen); + OPENSSL_free(valuebuf); + } + return pyattr; } static PyObject * From webhook-mailer at python.org Sun Feb 20 19:10:40 2022 From: webhook-mailer at python.org (asvetlov) Date: Mon, 21 Feb 2022 00:10:40 -0000 Subject: [Python-checkins] replace `self` param with more appropriate `cls` in classmethods (GH-31402) (#31445) Message-ID: https://github.com/python/cpython/commit/e7115d53632bebf137ca7245a9ced7107c082e99 commit: e7115d53632bebf137ca7245a9ced7107c082e99 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: asvetlov date: 2022-02-21T02:10:35+02:00 summary: replace `self` param with more appropriate `cls` in classmethods (GH-31402) (#31445) (cherry picked from commit a3fcca4af1cb418dc802feb75100ecc1a286afaa) Co-authored-by: Josh Smith Co-authored-by: Josh Smith files: M Lib/_collections_abc.py diff --git a/Lib/_collections_abc.py b/Lib/_collections_abc.py index acfaff802f506..023ac7cf03695 100644 --- a/Lib/_collections_abc.py +++ b/Lib/_collections_abc.py @@ -816,7 +816,7 @@ class KeysView(MappingView, Set): __slots__ = () @classmethod - def _from_iterable(self, it): + def _from_iterable(cls, it): return set(it) def __contains__(self, key): @@ -834,7 +834,7 @@ class ItemsView(MappingView, Set): __slots__ = () @classmethod - def _from_iterable(self, it): + def _from_iterable(cls, it): return set(it) def __contains__(self, item): From webhook-mailer at python.org Sun Feb 20 19:13:14 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Mon, 21 Feb 2022 00:13:14 -0000 Subject: [Python-checkins] [3.10] bpo-43853: Expand test suite for SQLite UDF's (GH-27642) (GH-31030) Message-ID: https://github.com/python/cpython/commit/ba457fe6f8e50ad3ef3ceffb75dee96629a42ad7 commit: ba457fe6f8e50ad3ef3ceffb75dee96629a42ad7 branch: 3.10 author: Erlend Egeberg Aasland committer: JelleZijlstra date: 2022-02-20T16:13:04-08:00 summary: [3.10] bpo-43853: Expand test suite for SQLite UDF's (GH-27642) (GH-31030) * [3.10] bpo-43853: Expand test suite for SQLite UDF's (GH-27642). (cherry picked from commit 3eb3b4f270757f66c7fb6dcf5afa416ee1582a4b) Co-authored-by: Erlend Egeberg Aasland * Fix test_func_return_too_large_int GH-27613 (bpo 44839) was not backported, so exceptions differ between main (3.11) and older versions. files: M Lib/sqlite3/test/userfunctions.py M Modules/_sqlite/connection.c M Modules/_sqlite/statement.c diff --git a/Lib/sqlite3/test/userfunctions.py b/Lib/sqlite3/test/userfunctions.py index 75e803582e06b..539fd4bcac695 100644 --- a/Lib/sqlite3/test/userfunctions.py +++ b/Lib/sqlite3/test/userfunctions.py @@ -23,9 +23,11 @@ import unittest import unittest.mock -import gc import sqlite3 as sqlite +from test.support import gc_collect + + def func_returntext(): return "foo" def func_returntextwithnull(): @@ -45,22 +47,6 @@ def func_returnlonglong(): def func_raiseexception(): 5/0 -def func_isstring(v): - return type(v) is str -def func_isint(v): - return type(v) is int -def func_isfloat(v): - return type(v) is float -def func_isnone(v): - return type(v) is type(None) -def func_isblob(v): - return isinstance(v, (bytes, memoryview)) -def func_islonglong(v): - return isinstance(v, int) and v >= 1<<31 - -def func(*args): - return len(args) - class AggrNoStep: def __init__(self): pass @@ -161,15 +147,13 @@ def setUp(self): self.con.create_function("returnnull", 0, func_returnnull) self.con.create_function("returnblob", 0, func_returnblob) self.con.create_function("returnlonglong", 0, func_returnlonglong) + self.con.create_function("returnnan", 0, lambda: float("nan")) + self.con.create_function("returntoolargeint", 0, lambda: 1 << 65) self.con.create_function("raiseexception", 0, func_raiseexception) - self.con.create_function("isstring", 1, func_isstring) - self.con.create_function("isint", 1, func_isint) - self.con.create_function("isfloat", 1, func_isfloat) - self.con.create_function("isnone", 1, func_isnone) - self.con.create_function("isblob", 1, func_isblob) - self.con.create_function("islonglong", 1, func_islonglong) - self.con.create_function("spam", -1, func) + self.con.create_function("isblob", 1, lambda x: isinstance(x, bytes)) + self.con.create_function("isnone", 1, lambda x: x is None) + self.con.create_function("spam", -1, lambda *x: len(x)) self.con.execute("create table test(t text)") def tearDown(self): @@ -246,6 +230,16 @@ def test_func_return_long_long(self): val = cur.fetchone()[0] self.assertEqual(val, 1<<31) + def test_func_return_nan(self): + cur = self.con.cursor() + cur.execute("select returnnan()") + self.assertIsNone(cur.fetchone()[0]) + + def test_func_return_too_large_int(self): + cur = self.con.cursor() + with self.assertRaises(sqlite.OperationalError): + self.con.execute("select returntoolargeint()") + def test_func_exception(self): cur = self.con.cursor() with self.assertRaises(sqlite.OperationalError) as cm: @@ -253,44 +247,6 @@ def test_func_exception(self): cur.fetchone() self.assertEqual(str(cm.exception), 'user-defined function raised exception') - def test_param_string(self): - cur = self.con.cursor() - for text in ["foo", str()]: - with self.subTest(text=text): - cur.execute("select isstring(?)", (text,)) - val = cur.fetchone()[0] - self.assertEqual(val, 1) - - def test_param_int(self): - cur = self.con.cursor() - cur.execute("select isint(?)", (42,)) - val = cur.fetchone()[0] - self.assertEqual(val, 1) - - def test_param_float(self): - cur = self.con.cursor() - cur.execute("select isfloat(?)", (3.14,)) - val = cur.fetchone()[0] - self.assertEqual(val, 1) - - def test_param_none(self): - cur = self.con.cursor() - cur.execute("select isnone(?)", (None,)) - val = cur.fetchone()[0] - self.assertEqual(val, 1) - - def test_param_blob(self): - cur = self.con.cursor() - cur.execute("select isblob(?)", (memoryview(b"blob"),)) - val = cur.fetchone()[0] - self.assertEqual(val, 1) - - def test_param_long_long(self): - cur = self.con.cursor() - cur.execute("select islonglong(?)", (1<<42,)) - val = cur.fetchone()[0] - self.assertEqual(val, 1) - def test_any_arguments(self): cur = self.con.cursor() cur.execute("select spam(?, ?)", (1, 2)) @@ -301,6 +257,52 @@ def test_empty_blob(self): cur = self.con.execute("select isblob(x'')") self.assertTrue(cur.fetchone()[0]) + def test_nan_float(self): + cur = self.con.execute("select isnone(?)", (float("nan"),)) + # SQLite has no concept of nan; it is converted to NULL + self.assertTrue(cur.fetchone()[0]) + + def test_too_large_int(self): + err = "Python int too large to convert to SQLite INTEGER" + self.assertRaisesRegex(OverflowError, err, self.con.execute, + "select spam(?)", (1 << 65,)) + + def test_non_contiguous_blob(self): + self.assertRaisesRegex(ValueError, "could not convert BLOB to buffer", + self.con.execute, "select spam(?)", + (memoryview(b"blob")[::2],)) + + def test_param_surrogates(self): + self.assertRaisesRegex(UnicodeEncodeError, "surrogates not allowed", + self.con.execute, "select spam(?)", + ("\ud803\ude6d",)) + + def test_func_params(self): + results = [] + def append_result(arg): + results.append((arg, type(arg))) + self.con.create_function("test_params", 1, append_result) + + dataset = [ + (42, int), + (-1, int), + (1234567890123456789, int), + (4611686018427387905, int), # 63-bit int with non-zero low bits + (3.14, float), + (float('inf'), float), + ("text", str), + ("1\x002", str), + ("\u02e2q\u02e1\u2071\u1d57\u1d49", str), + (b"blob", bytes), + (bytearray(range(2)), bytes), + (memoryview(b"blob"), bytes), + (None, type(None)), + ] + for val, _ in dataset: + cur = self.con.execute("select test_params(?)", (val,)) + cur.fetchone() + self.assertEqual(dataset, results) + # Regarding deterministic functions: # # Between 3.8.3 and 3.15.0, deterministic functions were only used to @@ -356,7 +358,7 @@ def md5sum(t): y.append(y) del x,y - gc.collect() + gc_collect() class AggregateTests(unittest.TestCase): def setUp(self): diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index 64610393ec1c3..b669160128691 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -552,7 +552,11 @@ _pysqlite_set_result(sqlite3_context* context, PyObject* py_val) return -1; sqlite3_result_int64(context, value); } else if (PyFloat_Check(py_val)) { - sqlite3_result_double(context, PyFloat_AsDouble(py_val)); + double value = PyFloat_AsDouble(py_val); + if (value == -1 && PyErr_Occurred()) { + return -1; + } + sqlite3_result_double(context, value); } else if (PyUnicode_Check(py_val)) { Py_ssize_t sz; const char *str = PyUnicode_AsUTF8AndSize(py_val, &sz); diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c index c875eb0cd74a4..3bc86420aa0ae 100644 --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -172,9 +172,16 @@ int pysqlite_statement_bind_parameter(pysqlite_Statement* self, int pos, PyObjec rc = sqlite3_bind_int64(self->st, pos, value); break; } - case TYPE_FLOAT: - rc = sqlite3_bind_double(self->st, pos, PyFloat_AsDouble(parameter)); + case TYPE_FLOAT: { + double value = PyFloat_AsDouble(parameter); + if (value == -1 && PyErr_Occurred()) { + rc = -1; + } + else { + rc = sqlite3_bind_double(self->st, pos, value); + } break; + } case TYPE_UNICODE: string = PyUnicode_AsUTF8AndSize(parameter, &buflen); if (string == NULL) From webhook-mailer at python.org Sun Feb 20 19:18:04 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Mon, 21 Feb 2022 00:18:04 -0000 Subject: [Python-checkins] [docs] Correct typos in SSLContext.sni_callback (GH-30623) (GH-31421) Message-ID: https://github.com/python/cpython/commit/a2e13ec0ce050a9050ec195ec0cc2a540912480d commit: a2e13ec0ce050a9050ec195ec0cc2a540912480d branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: JelleZijlstra date: 2022-02-20T16:17:56-08:00 summary: [docs] Correct typos in SSLContext.sni_callback (GH-30623) (GH-31421) Co-authored-by: J?rn Heissler (cherry picked from commit f80a97b492f41afd3c42bb2bd6da7b2828dca215) Co-authored-by: J?rn Heissler files: M Doc/library/ssl.rst diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index a20f97a26535e..3531cc3413afa 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1753,10 +1753,10 @@ to speed up repeated connections from the same clients. Due to the early negotiation phase of the TLS connection, only limited methods and attributes are usable like :meth:`SSLSocket.selected_alpn_protocol` and :attr:`SSLSocket.context`. - :meth:`SSLSocket.getpeercert`, :meth:`SSLSocket.getpeercert`, - :meth:`SSLSocket.cipher` and :meth:`SSLSocket.compress` methods require that + The :meth:`SSLSocket.getpeercert`, + :meth:`SSLSocket.cipher` and :meth:`SSLSocket.compression` methods require that the TLS connection has progressed beyond the TLS Client Hello and therefore - will not contain return meaningful values nor can they be called safely. + will not return meaningful values nor can they be called safely. The *sni_callback* function must return ``None`` to allow the TLS negotiation to continue. If a TLS failure is required, a constant From webhook-mailer at python.org Mon Feb 21 03:08:00 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 21 Feb 2022 08:08:00 -0000 Subject: [Python-checkins] bpo-39327: Close file descriptors as soon as possible in shutil.rmtree (GH-31384) Message-ID: https://github.com/python/cpython/commit/959846be52b03da57f50ebc07f99ec262a86e860 commit: 959846be52b03da57f50ebc07f99ec262a86e860 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-21T00:07:54-08:00 summary: bpo-39327: Close file descriptors as soon as possible in shutil.rmtree (GH-31384) It fixes the "Text File Busy" OSError when using 'rmtree' on a windows-managed filesystem in via the VirtualBox shared folder (and possible other scenarios like a windows-managed network file system). (cherry picked from commit b77158b4da449ec5b8f682816a79d004fd65ed07) Co-authored-by: Lital Natan files: A Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst M Lib/shutil.py M Misc/ACKS diff --git a/Lib/shutil.py b/Lib/shutil.py index 752900c58890c..c048cdf9b2cbd 100644 --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -656,6 +656,7 @@ def _rmtree_safe_fd(topfd, path, onerror): if is_dir: try: dirfd = os.open(entry.name, os.O_RDONLY, dir_fd=topfd) + dirfd_closed = False except OSError: onerror(os.open, fullname, sys.exc_info()) else: @@ -663,6 +664,8 @@ def _rmtree_safe_fd(topfd, path, onerror): if os.path.samestat(orig_st, os.fstat(dirfd)): _rmtree_safe_fd(dirfd, fullname, onerror) try: + os.close(dirfd) + dirfd_closed = True os.rmdir(entry.name, dir_fd=topfd) except OSError: onerror(os.rmdir, fullname, sys.exc_info()) @@ -676,7 +679,8 @@ def _rmtree_safe_fd(topfd, path, onerror): except OSError: onerror(os.path.islink, fullname, sys.exc_info()) finally: - os.close(dirfd) + if not dirfd_closed: + os.close(dirfd) else: try: os.unlink(entry.name, dir_fd=topfd) @@ -719,6 +723,7 @@ def onerror(*args): return try: fd = os.open(path, os.O_RDONLY) + fd_closed = False except Exception: onerror(os.open, path, sys.exc_info()) return @@ -726,6 +731,8 @@ def onerror(*args): if os.path.samestat(orig_st, os.fstat(fd)): _rmtree_safe_fd(fd, path, onerror) try: + os.close(fd) + fd_closed = True os.rmdir(path) except OSError: onerror(os.rmdir, path, sys.exc_info()) @@ -736,7 +743,8 @@ def onerror(*args): except OSError: onerror(os.path.islink, path, sys.exc_info()) finally: - os.close(fd) + if not fd_closed: + os.close(fd) else: try: if _rmtree_islink(path): diff --git a/Misc/ACKS b/Misc/ACKS index fe7e692e04967..c8d0be6999b78 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -418,6 +418,7 @@ Caleb Deveraux Catherine Devlin Scott Dial Alon Diamant +Lital Natan Toby Dickenson Mark Dickinson Jack Diederich diff --git a/Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst b/Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst new file mode 100644 index 0000000000000..fc6e8250922ff --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst @@ -0,0 +1,2 @@ +:func:`shutil.rmtree` can now work with VirtualBox shared folders when +running from the guest operating-system. From webhook-mailer at python.org Mon Feb 21 03:08:00 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 21 Feb 2022 08:08:00 -0000 Subject: [Python-checkins] bpo-39327: Close file descriptors as soon as possible in shutil.rmtree (GH-31384) Message-ID: https://github.com/python/cpython/commit/95d6271f1997580f8cb8a2985d0cd48af912763f commit: 95d6271f1997580f8cb8a2985d0cd48af912763f branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-21T00:07:46-08:00 summary: bpo-39327: Close file descriptors as soon as possible in shutil.rmtree (GH-31384) It fixes the "Text File Busy" OSError when using 'rmtree' on a windows-managed filesystem in via the VirtualBox shared folder (and possible other scenarios like a windows-managed network file system). (cherry picked from commit b77158b4da449ec5b8f682816a79d004fd65ed07) Co-authored-by: Lital Natan files: A Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst M Lib/shutil.py M Misc/ACKS diff --git a/Lib/shutil.py b/Lib/shutil.py index 0056a1bc77ad8..37bf98df79672 100644 --- a/Lib/shutil.py +++ b/Lib/shutil.py @@ -646,6 +646,7 @@ def _rmtree_safe_fd(topfd, path, onerror): if is_dir: try: dirfd = os.open(entry.name, os.O_RDONLY, dir_fd=topfd) + dirfd_closed = False except OSError: onerror(os.open, fullname, sys.exc_info()) else: @@ -653,6 +654,8 @@ def _rmtree_safe_fd(topfd, path, onerror): if os.path.samestat(orig_st, os.fstat(dirfd)): _rmtree_safe_fd(dirfd, fullname, onerror) try: + os.close(dirfd) + dirfd_closed = True os.rmdir(entry.name, dir_fd=topfd) except OSError: onerror(os.rmdir, fullname, sys.exc_info()) @@ -666,7 +669,8 @@ def _rmtree_safe_fd(topfd, path, onerror): except OSError: onerror(os.path.islink, fullname, sys.exc_info()) finally: - os.close(dirfd) + if not dirfd_closed: + os.close(dirfd) else: try: os.unlink(entry.name, dir_fd=topfd) @@ -709,6 +713,7 @@ def onerror(*args): return try: fd = os.open(path, os.O_RDONLY) + fd_closed = False except Exception: onerror(os.open, path, sys.exc_info()) return @@ -716,6 +721,8 @@ def onerror(*args): if os.path.samestat(orig_st, os.fstat(fd)): _rmtree_safe_fd(fd, path, onerror) try: + os.close(fd) + fd_closed = True os.rmdir(path) except OSError: onerror(os.rmdir, path, sys.exc_info()) @@ -726,7 +733,8 @@ def onerror(*args): except OSError: onerror(os.path.islink, path, sys.exc_info()) finally: - os.close(fd) + if not fd_closed: + os.close(fd) else: try: if _rmtree_islink(path): diff --git a/Misc/ACKS b/Misc/ACKS index 4bd05b59a9982..e107945600988 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -427,6 +427,7 @@ Caleb Deveraux Catherine Devlin Scott Dial Alon Diamant +Lital Natan Toby Dickenson Mark Dickinson Jack Diederich diff --git a/Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst b/Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst new file mode 100644 index 0000000000000..fc6e8250922ff --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-17-13-10-50.bpo-39327.ytIT7Z.rst @@ -0,0 +1,2 @@ +:func:`shutil.rmtree` can now work with VirtualBox shared folders when +running from the guest operating-system. From webhook-mailer at python.org Mon Feb 21 04:37:47 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 21 Feb 2022 09:37:47 -0000 Subject: [Python-checkins] bpo-46232: Fix parsing of certs with bit string in DN (GH-30351) Message-ID: https://github.com/python/cpython/commit/633d0f90f933515a9fca21a38cf87a8baf8ddc7d commit: 633d0f90f933515a9fca21a38cf87a8baf8ddc7d branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-21T01:37:26-08:00 summary: bpo-46232: Fix parsing of certs with bit string in DN (GH-30351) (cherry picked from commit be095f6c32188bba02079d086ac8639ea37cec3c) Co-authored-by: Christian Heimes files: A Misc/NEWS.d/next/Library/2022-01-03-09-46-44.bpo-46232.s0KlyI.rst M Modules/_ssl.c diff --git a/Misc/NEWS.d/next/Library/2022-01-03-09-46-44.bpo-46232.s0KlyI.rst b/Misc/NEWS.d/next/Library/2022-01-03-09-46-44.bpo-46232.s0KlyI.rst new file mode 100644 index 0000000000000..e252449199a05 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-03-09-46-44.bpo-46232.s0KlyI.rst @@ -0,0 +1,2 @@ +The :mod:`ssl` module now handles certificates with bit strings in DN +correctly. diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 6c63301b2a7d8..af2520432a64e 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1053,17 +1053,29 @@ _create_tuple_for_attribute(_sslmodulestate *state, ASN1_OBJECT *name, ASN1_STRING *value) { Py_ssize_t buflen; - unsigned char *valuebuf = NULL; - PyObject *attr; + PyObject *pyattr; + PyObject *pyname = _asn1obj2py(state, name, 0); - buflen = ASN1_STRING_to_UTF8(&valuebuf, value); - if (buflen < 0) { + if (pyname == NULL) { _setSSLError(state, NULL, 0, __FILE__, __LINE__); return NULL; } - attr = Py_BuildValue("Ns#", _asn1obj2py(state, name, 0), valuebuf, buflen); - OPENSSL_free(valuebuf); - return attr; + + if (ASN1_STRING_type(value) == V_ASN1_BIT_STRING) { + buflen = ASN1_STRING_length(value); + pyattr = Py_BuildValue("Ny#", pyname, ASN1_STRING_get0_data(value), buflen); + } else { + unsigned char *valuebuf = NULL; + buflen = ASN1_STRING_to_UTF8(&valuebuf, value); + if (buflen < 0) { + _setSSLError(state, NULL, 0, __FILE__, __LINE__); + Py_DECREF(pyname); + return NULL; + } + pyattr = Py_BuildValue("Ns#", pyname, valuebuf, buflen); + OPENSSL_free(valuebuf); + } + return pyattr; } static PyObject * From webhook-mailer at python.org Mon Feb 21 06:26:03 2022 From: webhook-mailer at python.org (asvetlov) Date: Mon, 21 Feb 2022 11:26:03 -0000 Subject: [Python-checkins] bpo-46796: Simplify handling of removed parameter "loop" in asyncio (GH-31431) Message-ID: https://github.com/python/cpython/commit/195a46d6ffd4cec6c5fb69c5890f8b1758ac91ca commit: 195a46d6ffd4cec6c5fb69c5890f8b1758ac91ca branch: main author: Serhiy Storchaka committer: asvetlov date: 2022-02-21T13:25:52+02:00 summary: bpo-46796: Simplify handling of removed parameter "loop" in asyncio (GH-31431) files: M Lib/asyncio/locks.py M Lib/asyncio/mixins.py M Lib/asyncio/queues.py M Lib/test/test_asyncio/test_locks.py diff --git a/Lib/asyncio/locks.py b/Lib/asyncio/locks.py index 4fef64e3921e1..0fbccfab7604f 100644 --- a/Lib/asyncio/locks.py +++ b/Lib/asyncio/locks.py @@ -73,8 +73,7 @@ class Lock(_ContextManagerMixin, mixins._LoopBoundMixin): """ - def __init__(self, *, loop=mixins._marker): - super().__init__(loop=loop) + def __init__(self): self._waiters = None self._locked = False @@ -163,8 +162,7 @@ class Event(mixins._LoopBoundMixin): false. """ - def __init__(self, *, loop=mixins._marker): - super().__init__(loop=loop) + def __init__(self): self._waiters = collections.deque() self._value = False @@ -226,8 +224,7 @@ class Condition(_ContextManagerMixin, mixins._LoopBoundMixin): A new Lock object is created and used as the underlying lock. """ - def __init__(self, lock=None, *, loop=mixins._marker): - super().__init__(loop=loop) + def __init__(self, lock=None): if lock is None: lock = Lock() @@ -344,8 +341,7 @@ class Semaphore(_ContextManagerMixin, mixins._LoopBoundMixin): ValueError is raised. """ - def __init__(self, value=1, *, loop=mixins._marker): - super().__init__(loop=loop) + def __init__(self, value=1): if value < 0: raise ValueError("Semaphore initial value must be >= 0") self._value = value @@ -408,9 +404,9 @@ class BoundedSemaphore(Semaphore): above the initial value. """ - def __init__(self, value=1, *, loop=mixins._marker): + def __init__(self, value=1): self._bound_value = value - super().__init__(value, loop=loop) + super().__init__(value) def release(self): if self._value >= self._bound_value: diff --git a/Lib/asyncio/mixins.py b/Lib/asyncio/mixins.py index 650df05ccc93e..c6bf97329e924 100644 --- a/Lib/asyncio/mixins.py +++ b/Lib/asyncio/mixins.py @@ -5,20 +5,10 @@ _global_lock = threading.Lock() -# Used as a sentinel for loop parameter -_marker = object() - class _LoopBoundMixin: _loop = None - def __init__(self, *, loop=_marker): - if loop is not _marker: - raise TypeError( - f'As of 3.10, the *loop* parameter was removed from ' - f'{type(self).__name__}() since it is no longer necessary' - ) - def _get_loop(self): loop = events._get_running_loop() diff --git a/Lib/asyncio/queues.py b/Lib/asyncio/queues.py index 10dd689bbb2ef..a9656a6df561b 100644 --- a/Lib/asyncio/queues.py +++ b/Lib/asyncio/queues.py @@ -30,8 +30,7 @@ class Queue(mixins._LoopBoundMixin): interrupted between calling qsize() and doing an operation on the Queue. """ - def __init__(self, maxsize=0, *, loop=mixins._marker): - super().__init__(loop=loop) + def __init__(self, maxsize=0): self._maxsize = maxsize # Futures. diff --git a/Lib/test/test_asyncio/test_locks.py b/Lib/test/test_asyncio/test_locks.py index 4ce338774f748..d8b164a20ead4 100644 --- a/Lib/test/test_asyncio/test_locks.py +++ b/Lib/test/test_asyncio/test_locks.py @@ -55,8 +55,8 @@ async def test_lock_doesnt_accept_loop_parameter(self): for cls in primitives_cls: with self.assertRaisesRegex( TypeError, - rf'As of 3.10, the \*loop\* parameter was removed from ' - rf'{cls.__name__}\(\) since it is no longer necessary' + rf"{cls.__name__}\.__init__\(\) got an unexpected " + rf"keyword argument 'loop'" ): cls(loop=loop) From webhook-mailer at python.org Mon Feb 21 09:47:08 2022 From: webhook-mailer at python.org (corona10) Date: Mon, 21 Feb 2022 14:47:08 -0000 Subject: [Python-checkins] bpo-46541: Remove usage of _Py_IDENTIFIER from pyexpat (GH-31468) Message-ID: https://github.com/python/cpython/commit/2b86616456629e11de33629da1bb732f033c436e commit: 2b86616456629e11de33629da1bb732f033c436e branch: main author: Dong-hee Na committer: corona10 date: 2022-02-21T23:46:52+09:00 summary: bpo-46541: Remove usage of _Py_IDENTIFIER from pyexpat (GH-31468) files: M Modules/pyexpat.c diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index f224f91f38fc3..7a26fe24e7592 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -1,5 +1,3 @@ -#define NEEDS_PY_IDENTIFIER - #include "Python.h" #include @@ -52,6 +50,7 @@ enum HandlerTypes { typedef struct { PyTypeObject *xml_parse_type; PyObject *error; + PyObject *str_read; } pyexpat_state; static inline pyexpat_state* @@ -824,11 +823,10 @@ pyexpat_xmlparser_ParseFile_impl(xmlparseobject *self, PyTypeObject *cls, { int rv = 1; PyObject *readmethod = NULL; - _Py_IDENTIFIER(read); pyexpat_state *state = PyType_GetModuleState(cls); - if (_PyObject_LookupAttrId(file, &PyId_read, &readmethod) < 0) { + if (_PyObject_LookupAttr(file, state->str_read, &readmethod) < 0) { return NULL; } if (readmethod == NULL) { @@ -1898,6 +1896,10 @@ static int pyexpat_exec(PyObject *mod) { pyexpat_state *state = pyexpat_get_state(mod); + state->str_read = PyUnicode_InternFromString("read"); + if (state->str_read == NULL) { + return -1; + } state->xml_parse_type = (PyTypeObject *)PyType_FromModuleAndSpec( mod, &_xml_parse_type_spec, NULL); @@ -2034,6 +2036,7 @@ pyexpat_traverse(PyObject *module, visitproc visit, void *arg) pyexpat_state *state = pyexpat_get_state(module); Py_VISIT(state->xml_parse_type); Py_VISIT(state->error); + Py_VISIT(state->str_read); return 0; } @@ -2043,6 +2046,7 @@ pyexpat_clear(PyObject *module) pyexpat_state *state = pyexpat_get_state(module); Py_CLEAR(state->xml_parse_type); Py_CLEAR(state->error); + Py_CLEAR(state->str_read); return 0; } From webhook-mailer at python.org Mon Feb 21 09:48:49 2022 From: webhook-mailer at python.org (ambv) Date: Mon, 21 Feb 2022 14:48:49 -0000 Subject: [Python-checkins] bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) Message-ID: https://github.com/python/cpython/commit/2cae93832f46b245847bdc252456ddf7742ef45e commit: 2cae93832f46b245847bdc252456ddf7742ef45e branch: main author: Sebastian Pipping committer: ambv date: 2022-02-21T15:48:32+01:00 summary: bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) Curly brackets were never allowed in namespace URIs according to RFC 3986, and so-called namespace-validating XML parsers have the right to reject them a invalid URIs. libexpat >=2.4.5 has become strcter in that regard due to related security issues; with ET.XML instantiating a namespace-aware parser under the hood, this test has no future in CPython. References: - https://datatracker.ietf.org/doc/html/rfc3968 - https://www.w3.org/TR/xml-names/ Also, test_minidom.py: Support Expat >=2.4.5 files: A Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst M Lib/test/test_minidom.py M Lib/test/test_xml_etree.py diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py index 1663b1f1143dd..97620258d82f6 100644 --- a/Lib/test/test_minidom.py +++ b/Lib/test/test_minidom.py @@ -6,10 +6,12 @@ from test import support import unittest +import pyexpat import xml.dom.minidom from xml.dom.minidom import parse, Node, Document, parseString from xml.dom.minidom import getDOMImplementation +from xml.parsers.expat import ExpatError tstfile = support.findfile("test.xml", subdir="xmltestdata") @@ -1147,7 +1149,13 @@ def testEncodings(self): # Verify that character decoding errors raise exceptions instead # of crashing - self.assertRaises(UnicodeDecodeError, parseString, + if pyexpat.version_info >= (2, 4, 5): + self.assertRaises(ExpatError, parseString, + b'') + self.assertRaises(ExpatError, parseString, + b'Comment \xe7a va ? Tr\xe8s bien ?') + else: + self.assertRaises(UnicodeDecodeError, parseString, b'Comment \xe7a va ? Tr\xe8s bien ?') doc.unlink() @@ -1609,7 +1617,12 @@ def testEmptyXMLNSValue(self): self.confirm(doc2.namespaceURI == xml.dom.EMPTY_NAMESPACE) def testExceptionOnSpacesInXMLNSValue(self): - with self.assertRaisesRegex(ValueError, 'Unsupported syntax'): + if pyexpat.version_info >= (2, 4, 5): + context = self.assertRaisesRegex(ExpatError, 'syntax error') + else: + context = self.assertRaisesRegex(ValueError, 'Unsupported syntax') + + with context: parseString('') def testDocRemoveChild(self): diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index a25f536134c7b..c5292b5e9ef68 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -2192,12 +2192,6 @@ def test_issue6233(self): b"\n" b'tãg') - def test_issue3151(self): - e = ET.XML('') - self.assertEqual(e.tag, '{${stuff}}localname') - t = ET.ElementTree(e) - self.assertEqual(ET.tostring(e), b'') - def test_issue6565(self): elem = ET.XML("") self.assertEqual(summarize_list(elem), ['tag']) diff --git a/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst new file mode 100644 index 0000000000000..6969bd1898f65 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst @@ -0,0 +1 @@ +Make test suite support Expat >=2.4.5 From webhook-mailer at python.org Mon Feb 21 10:02:43 2022 From: webhook-mailer at python.org (ambv) Date: Mon, 21 Feb 2022 15:02:43 -0000 Subject: [Python-checkins] bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) (GH-31297) Message-ID: https://github.com/python/cpython/commit/c60414de7cefd092643ba200c2c045da1569c391 commit: c60414de7cefd092643ba200c2c045da1569c391 branch: 3.8 author: Dong-hee Na committer: ambv date: 2022-02-21T16:02:38+01:00 summary: bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) (GH-31297) Co-authored-by: Cyril Jouve files: A Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst M Modules/expat/expat.h M Modules/expat/xmlparse.c M Modules/expat/xmlrole.c M Modules/expat/xmltok.c M Modules/expat/xmltok_ns.c diff --git a/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst new file mode 100644 index 0000000000000..9c1f24c0e5171 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst @@ -0,0 +1 @@ +expat: Update libexpat from 2.4.1 to 2.4.4 diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index b7d6d354801b3..4c5704fd9336b 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -11,7 +11,7 @@ Copyright (c) 2000-2005 Fred L. Drake, Jr. Copyright (c) 2001-2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Cristian Rodr?guez Copyright (c) 2016 Thomas Beutlich Copyright (c) 2017 Rhodri James @@ -1041,7 +1041,7 @@ XML_SetBillionLaughsAttackProtectionActivationThreshold( */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 4 -#define XML_MICRO_VERSION 1 +#define XML_MICRO_VERSION 4 #ifdef __cplusplus } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 5ba56eaea6357..4b43e61321691 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 8539b9040d9d901366a62560a064af7cb99811335784b363abc039c5b0ebc416 (2.4.1+) +/* 2e2c8ce5f11a473d65ec313ab20ceee6afefb355f5405afc06e7204e2e41c8c0 (2.4.4+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -13,7 +13,7 @@ Copyright (c) 2002-2016 Karl Waclawek Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016 Eric Rahm - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Gaurav Copyright (c) 2016 Thomas Beutlich Copyright (c) 2016 Gustavo Grieco @@ -32,6 +32,8 @@ Copyright (c) 2019 David Loffredo Copyright (c) 2019-2020 Ben Wagner Copyright (c) 2019 Vadim Zeitlin + Copyright (c) 2021 Dong-hee Na + Copyright (c) 2022 Samanta Navarro Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -54,6 +56,10 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#define XML_BUILDING_EXPAT 1 + +#include + #if ! defined(_GNU_SOURCE) # define _GNU_SOURCE 1 /* syscall prototype */ #endif @@ -84,14 +90,10 @@ # include #endif -#define XML_BUILDING_EXPAT 1 - #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "ascii.h" #include "expat.h" #include "siphash.h" @@ -973,7 +975,7 @@ parserCreate(const XML_Char *encodingName, if (memsuite) { XML_Memory_Handling_Suite *mtemp; - parser = (XML_Parser)memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); + parser = memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); if (parser != NULL) { mtemp = (XML_Memory_Handling_Suite *)&(parser->m_mem); mtemp->malloc_fcn = memsuite->malloc_fcn; @@ -2066,6 +2068,11 @@ XML_GetBuffer(XML_Parser parser, int len) { keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; + /* Detect and prevent integer overflow */ + if (keep > INT_MAX - neededSize) { + parser->m_errorCode = XML_ERROR_NO_MEMORY; + return NULL; + } neededSize += keep; #endif /* defined XML_CONTEXT_BYTES */ if (neededSize @@ -3260,13 +3267,38 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, /* get the attributes from the tokenizer */ n = XmlGetAttributes(enc, attStr, parser->m_attsSize, parser->m_atts); + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - nDefaultAtts) { + return XML_ERROR_NO_MEMORY; + } + if (n + nDefaultAtts > parser->m_attsSize) { int oldAttsSize = parser->m_attsSize; ATTRIBUTE *temp; #ifdef XML_ATTR_INFO XML_AttrInfo *temp2; #endif + + /* Detect and prevent integer overflow */ + if ((nDefaultAtts > INT_MAX - INIT_ATTS_SIZE) + || (n > INT_MAX - (nDefaultAtts + INIT_ATTS_SIZE))) { + return XML_ERROR_NO_MEMORY; + } + parser->m_attsSize = n + nDefaultAtts + INIT_ATTS_SIZE; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(ATTRIBUTE)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (ATTRIBUTE *)REALLOC(parser, (void *)parser->m_atts, parser->m_attsSize * sizeof(ATTRIBUTE)); if (temp == NULL) { @@ -3275,6 +3307,17 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, } parser->m_atts = temp; #ifdef XML_ATTR_INFO + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +# if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(XML_AttrInfo)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +# endif + temp2 = (XML_AttrInfo *)REALLOC(parser, (void *)parser->m_attInfo, parser->m_attsSize * sizeof(XML_AttrInfo)); if (temp2 == NULL) { @@ -3413,7 +3456,13 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, if (nPrefixes) { int j; /* hash table index */ unsigned long version = parser->m_nsAttsVersion; - int nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(unsigned int) * 8 /* bits per byte */) { + return XML_ERROR_NO_MEMORY; + } + + unsigned int nsAttsSize = 1u << parser->m_nsAttsPower; unsigned char oldNsAttsPower = parser->m_nsAttsPower; /* size of hash table must be at least 2 * (# of prefixed attributes) */ if ((nPrefixes << 1) @@ -3424,7 +3473,28 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, ; if (parser->m_nsAttsPower < 3) parser->m_nsAttsPower = 3; - nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(nsAttsSize) * 8 /* bits per byte */) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } + + nsAttsSize = 1u << parser->m_nsAttsPower; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (nsAttsSize > (size_t)(-1) / sizeof(NS_ATT)) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (NS_ATT *)REALLOC(parser, parser->m_nsAtts, nsAttsSize * sizeof(NS_ATT)); if (! temp) { @@ -3582,9 +3652,31 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, tagNamePtr->prefixLen = prefixLen; for (i = 0; localPart[i++];) ; /* i includes null terminator */ + + /* Detect and prevent integer overflow */ + if (binding->uriLen > INT_MAX - prefixLen + || i > INT_MAX - (binding->uriLen + prefixLen)) { + return XML_ERROR_NO_MEMORY; + } + n = i + binding->uriLen + prefixLen; if (n > binding->uriAlloc) { TAG *p; + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(n + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + uri = (XML_Char *)MALLOC(parser, (n + EXPAND_SPARE) * sizeof(XML_Char)); if (! uri) return XML_ERROR_NO_MEMORY; @@ -3680,6 +3772,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (parser->m_freeBindingList) { b = parser->m_freeBindingList; if (len > b->uriAlloc) { + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + XML_Char *temp = (XML_Char *)REALLOC( parser, b->uri, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (temp == NULL) @@ -3692,6 +3799,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, b = (BINDING *)MALLOC(parser, sizeof(BINDING)); if (! b) return XML_ERROR_NO_MEMORY; + + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + b->uri = (XML_Char *)MALLOC(parser, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (! b->uri) { @@ -3976,7 +4098,7 @@ initializeEncoding(XML_Parser parser) { const char *s; #ifdef XML_UNICODE char encodingBuf[128]; - /* See comments abount `protoclEncodingName` in parserInit() */ + /* See comments about `protocolEncodingName` in parserInit() */ if (! parser->m_protocolEncodingName) s = NULL; else { @@ -5018,6 +5140,11 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (parser->m_prologState.level >= parser->m_groupSize) { if (parser->m_groupSize) { { + /* Detect and prevent integer overflow */ + if (parser->m_groupSize > (unsigned int)(-1) / 2u) { + return XML_ERROR_NO_MEMORY; + } + char *const new_connector = (char *)REALLOC( parser, parser->m_groupConnector, parser->m_groupSize *= 2); if (new_connector == NULL) { @@ -5028,6 +5155,16 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, } if (dtd->scaffIndex) { + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (parser->m_groupSize > (size_t)(-1) / sizeof(int)) { + return XML_ERROR_NO_MEMORY; + } +#endif + int *const new_scaff_index = (int *)REALLOC( parser, dtd->scaffIndex, parser->m_groupSize * sizeof(int)); if (new_scaff_index == NULL) @@ -5236,7 +5373,7 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (dtd->in_eldecl) { ELEMENT_TYPE *el; const XML_Char *name; - int nameLen; + size_t nameLen; const char *nxt = (quant == XML_CQUANT_NONE ? next : next - enc->minBytesPerChar); int myindex = nextScaffoldPart(parser); @@ -5252,7 +5389,13 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, nameLen = 0; for (; name[nameLen++];) ; - dtd->contentStringLen += nameLen; + + /* Detect and prevent integer overflow */ + if (nameLen > UINT_MAX - dtd->contentStringLen) { + return XML_ERROR_NO_MEMORY; + } + + dtd->contentStringLen += (unsigned)nameLen; if (parser->m_elementDeclHandler) handleDefault = XML_FALSE; } @@ -6098,7 +6241,24 @@ defineAttribute(ELEMENT_TYPE *type, ATTRIBUTE_ID *attId, XML_Bool isCdata, } } else { DEFAULT_ATTRIBUTE *temp; + + /* Detect and prevent integer overflow */ + if (type->allocDefaultAtts > INT_MAX / 2) { + return 0; + } + int count = type->allocDefaultAtts * 2; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)count > (size_t)(-1) / sizeof(DEFAULT_ATTRIBUTE)) { + return 0; + } +#endif + temp = (DEFAULT_ATTRIBUTE *)REALLOC(parser, type->defaultAtts, (count * sizeof(DEFAULT_ATTRIBUTE))); if (temp == NULL) @@ -6388,7 +6548,7 @@ normalizePublicId(XML_Char *publicId) { static DTD * dtdCreate(const XML_Memory_Handling_Suite *ms) { - DTD *p = (DTD *)ms->malloc_fcn(sizeof(DTD)); + DTD *p = ms->malloc_fcn(sizeof(DTD)); if (p == NULL) return p; poolInit(&(p->pool), ms); @@ -6561,8 +6721,8 @@ dtdCopy(XML_Parser oldParser, DTD *newDtd, const DTD *oldDtd, if (! newE) return 0; if (oldE->nDefaultAtts) { - newE->defaultAtts = (DEFAULT_ATTRIBUTE *)ms->malloc_fcn( - oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); + newE->defaultAtts + = ms->malloc_fcn(oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); if (! newE->defaultAtts) { return 0; } @@ -6724,7 +6884,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* table->size is a power of 2 */ table->size = (size_t)1 << INIT_POWER; tsize = table->size * sizeof(NAMED *); - table->v = (NAMED **)table->mem->malloc_fcn(tsize); + table->v = table->mem->malloc_fcn(tsize); if (! table->v) { table->size = 0; return NULL; @@ -6749,10 +6909,22 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* check for overflow (table is half full) */ if (table->used >> (table->power - 1)) { unsigned char newPower = table->power + 1; + + /* Detect and prevent invalid shift */ + if (newPower >= sizeof(unsigned long) * 8 /* bits per byte */) { + return NULL; + } + size_t newSize = (size_t)1 << newPower; unsigned long newMask = (unsigned long)newSize - 1; + + /* Detect and prevent integer overflow */ + if (newSize > (size_t)(-1) / sizeof(NAMED *)) { + return NULL; + } + size_t tsize = newSize * sizeof(NAMED *); - NAMED **newV = (NAMED **)table->mem->malloc_fcn(tsize); + NAMED **newV = table->mem->malloc_fcn(tsize); if (! newV) return NULL; memset(newV, 0, tsize); @@ -6781,7 +6953,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { } } } - table->v[i] = (NAMED *)table->mem->malloc_fcn(createSize); + table->v[i] = table->mem->malloc_fcn(createSize); if (! table->v[i]) return NULL; memset(table->v[i], 0, createSize); @@ -7069,7 +7241,7 @@ poolGrow(STRING_POOL *pool) { if (bytesToAllocate == 0) return XML_FALSE; - tem = (BLOCK *)pool->mem->malloc_fcn(bytesToAllocate); + tem = pool->mem->malloc_fcn(bytesToAllocate); if (! tem) return XML_FALSE; tem->size = blockSize; @@ -7100,6 +7272,20 @@ nextScaffoldPart(XML_Parser parser) { if (dtd->scaffCount >= dtd->scaffSize) { CONTENT_SCAFFOLD *temp; if (dtd->scaffold) { + /* Detect and prevent integer overflow */ + if (dtd->scaffSize > UINT_MAX / 2u) { + return -1; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffSize > (size_t)(-1) / 2u / sizeof(CONTENT_SCAFFOLD)) { + return -1; + } +#endif + temp = (CONTENT_SCAFFOLD *)REALLOC( parser, dtd->scaffold, dtd->scaffSize * 2 * sizeof(CONTENT_SCAFFOLD)); if (temp == NULL) @@ -7169,8 +7355,26 @@ build_model(XML_Parser parser) { XML_Content *ret; XML_Content *cpos; XML_Char *str; - int allocsize = (dtd->scaffCount * sizeof(XML_Content) - + (dtd->contentStringLen * sizeof(XML_Char))); + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffCount > (size_t)(-1) / sizeof(XML_Content)) { + return NULL; + } + if (dtd->contentStringLen > (size_t)(-1) / sizeof(XML_Char)) { + return NULL; + } +#endif + if (dtd->scaffCount * sizeof(XML_Content) + > (size_t)(-1) - dtd->contentStringLen * sizeof(XML_Char)) { + return NULL; + } + + const size_t allocsize = (dtd->scaffCount * sizeof(XML_Content) + + (dtd->contentStringLen * sizeof(XML_Char))); ret = (XML_Content *)MALLOC(parser, allocsize); if (! ret) diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 08173b0fd541d..77746ee42d10a 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -15,6 +15,7 @@ Copyright (c) 2016-2021 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -37,14 +38,14 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#include + #include #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "expat_external.h" #include "internal.h" #include "xmlrole.h" diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index f2b6b406067ea..502ca1adc33b9 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -20,6 +20,7 @@ Copyright (c) 2017 Benbuck Nason Copyright (c) 2017 Jos? Guti?rrez de la Concha Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -42,6 +43,8 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#include + #include #include /* memcpy */ #include @@ -50,8 +53,6 @@ # include "winconfig.h" #endif -#include - #include "expat_external.h" #include "internal.h" #include "xmltok.h" diff --git a/Modules/expat/xmltok_ns.c b/Modules/expat/xmltok_ns.c index 5fd8392235940..fbdd3e3c7b799 100644 --- a/Modules/expat/xmltok_ns.c +++ b/Modules/expat/xmltok_ns.c @@ -11,7 +11,7 @@ Copyright (c) 2002 Greg Stein Copyright (c) 2002 Fred L. Drake, Jr. Copyright (c) 2002-2006 Karl Waclawek - Copyright (c) 2017 Sebastian Pipping + Copyright (c) 2017-2021 Sebastian Pipping Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -93,7 +93,7 @@ NS(XmlInitEncoding)(INIT_ENCODING *p, const ENCODING **encPtr, static const ENCODING * NS(findEncoding)(const ENCODING *enc, const char *ptr, const char *end) { # define ENCODING_MAX 128 - char buf[ENCODING_MAX]; + char buf[ENCODING_MAX] = ""; char *p = buf; int i; XmlUtf8Convert(enc, &ptr, end, &p, p + ENCODING_MAX - 1); From webhook-mailer at python.org Mon Feb 21 10:42:49 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Mon, 21 Feb 2022 15:42:49 -0000 Subject: [Python-checkins] fix __bool__ docstring (GH-31301) Message-ID: https://github.com/python/cpython/commit/0a222db2bca63070f429c0e613707da1bdfaf0e0 commit: 0a222db2bca63070f429c0e613707da1bdfaf0e0 branch: main author: Jelle Zijlstra committer: JelleZijlstra date: 2022-02-21T07:42:27-08:00 summary: fix __bool__ docstring (GH-31301) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst new file mode 100644 index 0000000000000..9937116bb2e7b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst @@ -0,0 +1,2 @@ +Correct the docstring for the :meth:`__bool__` method. Patch by Jelle +Zijlstra. diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 8c4901119de7d..4b7035c81e533 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -8065,7 +8065,7 @@ static slotdef slotdefs[] = { UNSLOT("__abs__", nb_absolute, slot_nb_absolute, wrap_unaryfunc, "abs(self)"), UNSLOT("__bool__", nb_bool, slot_nb_bool, wrap_inquirypred, - "self != 0"), + "True if self else False"), UNSLOT("__invert__", nb_invert, slot_nb_invert, wrap_unaryfunc, "~self"), BINSLOT("__lshift__", nb_lshift, slot_nb_lshift, "<<"), RBINSLOT("__rlshift__", nb_lshift, slot_nb_lshift, "<<"), From webhook-mailer at python.org Mon Feb 21 11:16:27 2022 From: webhook-mailer at python.org (ambv) Date: Mon, 21 Feb 2022 16:16:27 -0000 Subject: [Python-checkins] bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31472) Message-ID: https://github.com/python/cpython/commit/7da97f61816f3cadaa6788804b22a2434b40e8c5 commit: 7da97f61816f3cadaa6788804b22a2434b40e8c5 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2022-02-21T17:16:09+01:00 summary: bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31472) Curly brackets were never allowed in namespace URIs according to RFC 3986, and so-called namespace-validating XML parsers have the right to reject them a invalid URIs. libexpat >=2.4.5 has become strcter in that regard due to related security issues; with ET.XML instantiating a namespace-aware parser under the hood, this test has no future in CPython. References: - https://datatracker.ietf.org/doc/html/rfc3968 - https://www.w3.org/TR/xml-names/ Also, test_minidom.py: Support Expat >=2.4.5 (cherry picked from commit 2cae93832f46b245847bdc252456ddf7742ef45e) Co-authored-by: Sebastian Pipping files: A Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst M Lib/test/test_minidom.py M Lib/test/test_xml_etree.py diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py index 1663b1f1143dd..97620258d82f6 100644 --- a/Lib/test/test_minidom.py +++ b/Lib/test/test_minidom.py @@ -6,10 +6,12 @@ from test import support import unittest +import pyexpat import xml.dom.minidom from xml.dom.minidom import parse, Node, Document, parseString from xml.dom.minidom import getDOMImplementation +from xml.parsers.expat import ExpatError tstfile = support.findfile("test.xml", subdir="xmltestdata") @@ -1147,7 +1149,13 @@ def testEncodings(self): # Verify that character decoding errors raise exceptions instead # of crashing - self.assertRaises(UnicodeDecodeError, parseString, + if pyexpat.version_info >= (2, 4, 5): + self.assertRaises(ExpatError, parseString, + b'') + self.assertRaises(ExpatError, parseString, + b'Comment \xe7a va ? Tr\xe8s bien ?') + else: + self.assertRaises(UnicodeDecodeError, parseString, b'Comment \xe7a va ? Tr\xe8s bien ?') doc.unlink() @@ -1609,7 +1617,12 @@ def testEmptyXMLNSValue(self): self.confirm(doc2.namespaceURI == xml.dom.EMPTY_NAMESPACE) def testExceptionOnSpacesInXMLNSValue(self): - with self.assertRaisesRegex(ValueError, 'Unsupported syntax'): + if pyexpat.version_info >= (2, 4, 5): + context = self.assertRaisesRegex(ExpatError, 'syntax error') + else: + context = self.assertRaisesRegex(ValueError, 'Unsupported syntax') + + with context: parseString('') def testDocRemoveChild(self): diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index a25f536134c7b..c5292b5e9ef68 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -2192,12 +2192,6 @@ def test_issue6233(self): b"\n" b'tãg') - def test_issue3151(self): - e = ET.XML('') - self.assertEqual(e.tag, '{${stuff}}localname') - t = ET.ElementTree(e) - self.assertEqual(ET.tostring(e), b'') - def test_issue6565(self): elem = ET.XML("") self.assertEqual(summarize_list(elem), ['tag']) diff --git a/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst new file mode 100644 index 0000000000000..6969bd1898f65 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst @@ -0,0 +1 @@ +Make test suite support Expat >=2.4.5 From webhook-mailer at python.org Mon Feb 21 11:16:27 2022 From: webhook-mailer at python.org (ambv) Date: Mon, 21 Feb 2022 16:16:27 -0000 Subject: [Python-checkins] bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31469) Message-ID: https://github.com/python/cpython/commit/336a916f75642dfe2d87e237981686051d5d51f8 commit: 336a916f75642dfe2d87e237981686051d5d51f8 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ambv date: 2022-02-21T17:16:23+01:00 summary: bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31469) Curly brackets were never allowed in namespace URIs according to RFC 3986, and so-called namespace-validating XML parsers have the right to reject them a invalid URIs. libexpat >=2.4.5 has become strcter in that regard due to related security issues; with ET.XML instantiating a namespace-aware parser under the hood, this test has no future in CPython. References: - https://datatracker.ietf.org/doc/html/rfc3968 - https://www.w3.org/TR/xml-names/ Also, test_minidom.py: Support Expat >=2.4.5 (cherry picked from commit 2cae93832f46b245847bdc252456ddf7742ef45e) Co-authored-by: Sebastian Pipping Co-authored-by: Sebastian Pipping files: A Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst M Lib/test/test_minidom.py M Lib/test/test_xml_etree.py diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py index 1663b1f1143dd..97620258d82f6 100644 --- a/Lib/test/test_minidom.py +++ b/Lib/test/test_minidom.py @@ -6,10 +6,12 @@ from test import support import unittest +import pyexpat import xml.dom.minidom from xml.dom.minidom import parse, Node, Document, parseString from xml.dom.minidom import getDOMImplementation +from xml.parsers.expat import ExpatError tstfile = support.findfile("test.xml", subdir="xmltestdata") @@ -1147,7 +1149,13 @@ def testEncodings(self): # Verify that character decoding errors raise exceptions instead # of crashing - self.assertRaises(UnicodeDecodeError, parseString, + if pyexpat.version_info >= (2, 4, 5): + self.assertRaises(ExpatError, parseString, + b'') + self.assertRaises(ExpatError, parseString, + b'Comment \xe7a va ? Tr\xe8s bien ?') + else: + self.assertRaises(UnicodeDecodeError, parseString, b'Comment \xe7a va ? Tr\xe8s bien ?') doc.unlink() @@ -1609,7 +1617,12 @@ def testEmptyXMLNSValue(self): self.confirm(doc2.namespaceURI == xml.dom.EMPTY_NAMESPACE) def testExceptionOnSpacesInXMLNSValue(self): - with self.assertRaisesRegex(ValueError, 'Unsupported syntax'): + if pyexpat.version_info >= (2, 4, 5): + context = self.assertRaisesRegex(ExpatError, 'syntax error') + else: + context = self.assertRaisesRegex(ValueError, 'Unsupported syntax') + + with context: parseString('') def testDocRemoveChild(self): diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index 8cdf451fbdbac..762b0bfed2c33 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -2168,12 +2168,6 @@ def test_issue6233(self): b"\n" b'tãg') - def test_issue3151(self): - e = ET.XML('') - self.assertEqual(e.tag, '{${stuff}}localname') - t = ET.ElementTree(e) - self.assertEqual(ET.tostring(e), b'') - def test_issue6565(self): elem = ET.XML("") self.assertEqual(summarize_list(elem), ['tag']) diff --git a/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst new file mode 100644 index 0000000000000..6969bd1898f65 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst @@ -0,0 +1 @@ +Make test suite support Expat >=2.4.5 From webhook-mailer at python.org Mon Feb 21 12:18:57 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 21 Feb 2022 17:18:57 -0000 Subject: [Python-checkins] [3.10] bpo-46732: fix __bool__ docstring (GH-31301) (GH-31473) Message-ID: https://github.com/python/cpython/commit/c596ecbf821843de0e044f0d4da34c6b49a06907 commit: c596ecbf821843de0e044f0d4da34c6b49a06907 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-21T09:18:48-08:00 summary: [3.10] bpo-46732: fix __bool__ docstring (GH-31301) (GH-31473) (cherry picked from commit 0a222db2bca63070f429c0e613707da1bdfaf0e0) Co-authored-by: Jelle Zijlstra files: A Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst new file mode 100644 index 0000000000000..9937116bb2e7b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst @@ -0,0 +1,2 @@ +Correct the docstring for the :meth:`__bool__` method. Patch by Jelle +Zijlstra. diff --git a/Objects/typeobject.c b/Objects/typeobject.c index deffeb2ccc25d..b3ba1208eb253 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -7996,7 +7996,7 @@ static slotdef slotdefs[] = { UNSLOT("__abs__", nb_absolute, slot_nb_absolute, wrap_unaryfunc, "abs(self)"), UNSLOT("__bool__", nb_bool, slot_nb_bool, wrap_inquirypred, - "self != 0"), + "True if self else False"), UNSLOT("__invert__", nb_invert, slot_nb_invert, wrap_unaryfunc, "~self"), BINSLOT("__lshift__", nb_lshift, slot_nb_lshift, "<<"), RBINSLOT("__rlshift__", nb_lshift, slot_nb_lshift, "<<"), From webhook-mailer at python.org Mon Feb 21 12:18:59 2022 From: webhook-mailer at python.org (miss-islington) Date: Mon, 21 Feb 2022 17:18:59 -0000 Subject: [Python-checkins] [3.9] bpo-46732: fix __bool__ docstring (GH-31301) (GH-31474) Message-ID: https://github.com/python/cpython/commit/8eb18d842c37c37c1f9316c7e171aad36e875b9a commit: 8eb18d842c37c37c1f9316c7e171aad36e875b9a branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-21T09:18:55-08:00 summary: [3.9] bpo-46732: fix __bool__ docstring (GH-31301) (GH-31474) (cherry picked from commit 0a222db2bca63070f429c0e613707da1bdfaf0e0) Co-authored-by: Jelle Zijlstra files: A Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst new file mode 100644 index 0000000000000..9937116bb2e7b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-12-11-16-40.bpo-46732.3Z_qxd.rst @@ -0,0 +1,2 @@ +Correct the docstring for the :meth:`__bool__` method. Patch by Jelle +Zijlstra. diff --git a/Objects/typeobject.c b/Objects/typeobject.c index d9ea9e8626478..cb0bb46145c6a 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -7236,7 +7236,7 @@ static slotdef slotdefs[] = { UNSLOT("__abs__", nb_absolute, slot_nb_absolute, wrap_unaryfunc, "abs(self)"), UNSLOT("__bool__", nb_bool, slot_nb_bool, wrap_inquirypred, - "self != 0"), + "True if self else False"), UNSLOT("__invert__", nb_invert, slot_nb_invert, wrap_unaryfunc, "~self"), BINSLOT("__lshift__", nb_lshift, slot_nb_lshift, "<<"), RBINSLOT("__rlshift__", nb_lshift, slot_nb_lshift, "<<"), From webhook-mailer at python.org Mon Feb 21 12:59:00 2022 From: webhook-mailer at python.org (ned-deily) Date: Mon, 21 Feb 2022 17:59:00 -0000 Subject: [Python-checkins] bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-31476) Message-ID: https://github.com/python/cpython/commit/7a5850987010ca5cb0f1e2844cfe183935916e7e commit: 7a5850987010ca5cb0f1e2844cfe183935916e7e branch: 3.7 author: Ned Deily committer: ned-deily date: 2022-02-21T12:58:35-05:00 summary: bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-31476) files: M Doc/Makefile diff --git a/Doc/Makefile b/Doc/Makefile index 8a5e03079925d..1393bcb52500e 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -127,7 +127,7 @@ clean: venv: $(PYTHON) -m venv $(VENVDIR) - $(VENVDIR)/bin/python3 -m pip install -U Sphinx==2.3.1 blurb + $(VENVDIR)/bin/python3 -m pip install -U Sphinx==2.3.1 blurb docutils==0.17.1 @echo "The venv has been created in the $(VENVDIR) directory" dist: From webhook-mailer at python.org Mon Feb 21 13:26:57 2022 From: webhook-mailer at python.org (markshannon) Date: Mon, 21 Feb 2022 18:26:57 -0000 Subject: [Python-checkins] bpo-46329: Streamline calling sequence a bit. (GH-31465) Message-ID: https://github.com/python/cpython/commit/59585d6b2ea50d7bc3a9b336da5bde61367f527c commit: 59585d6b2ea50d7bc3a9b336da5bde61367f527c branch: main author: Mark Shannon committer: markshannon date: 2022-02-21T18:26:47Z summary: bpo-46329: Streamline calling sequence a bit. (GH-31465) * Move handling of bound-methods to PRECALL. * Remove call_shape.postcall_shrink * Remove call_shape.callable * Remove call_shape.callable. Change CALL oparg to match PRECALL oparg. * Move KW_NAMES before PRECALL. * Update opcode docs in dis.rst files: A Misc/NEWS.d/next/Core and Builtins/2022-02-21-10-29-20.bpo-46329.cbkt7u.rst M Doc/library/dis.rst M Lib/importlib/_bootstrap_external.py M Lib/test/test_dis.py M Programs/test_frozenmain.h M Python/ceval.c M Python/compile.c diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index ef64f75ce9f83..003d79864cbf0 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -36,11 +36,12 @@ the following command can be used to display the disassembly of >>> dis.dis(myfunc) 1 0 RESUME 0 - 2 2 LOAD_GLOBAL 0 (len) - 4 LOAD_FAST 0 (alist) - 6 PRECALL_FUNCTION 1 - 8 CALL 0 - 10 RETURN_VALUE + 2 2 PUSH_NULL + 4 LOAD_GLOBAL 0 (len) + 6 LOAD_FAST 0 (alist) + 8 PRECALL 1 + 10 CALL 1 + 12 RETURN_VALUE (The "2" is a line number). @@ -106,9 +107,10 @@ Example:: ... print(instr.opname) ... RESUME + PUSH_NULL LOAD_GLOBAL LOAD_FAST - PRECALL_FUNCTION + PRECALL CALL RETURN_VALUE @@ -1063,18 +1065,28 @@ iterations of the loop. with ``__cause__`` set to ``TOS``) -.. opcode:: CALL (named) +.. opcode:: CALL (argc) - Calls a callable object with the number of positional arguments specified by - the preceding :opcode:`PRECALL_FUNCTION` or :opcode:`PRECALL_METHOD` and - the named arguments specified by the preceding :opcode:`KW_NAMES`, if any. - *named* indicates the number of named arguments. - On the stack are (in ascending order): + Calls a callable object with the number of arguments specified by ``argc``, + including the named arguments specified by the preceding + :opcode:`KW_NAMES`, if any. + On the stack are (in ascending order), either: + * NULL * The callable * The positional arguments * The named arguments + or: + + * The callable + * ``self`` + * The remaining positional arguments + * The named arguments + + ``argc`` is the total of the positional and named arguments, excluding + ``self`` when a ``NULL`` is not present. + ``CALL`` pops all arguments and the callable object off the stack, calls the callable object with those arguments, and pushes the return value returned by the callable object. @@ -1102,33 +1114,34 @@ iterations of the loop. Loads a method named ``co_names[namei]`` from the TOS object. TOS is popped. This bytecode distinguishes two cases: if TOS has a method with the correct name, the bytecode pushes the unbound method and TOS. TOS will be used as - the first argument (``self``) by :opcode:`PRECALL_METHOD` when calling the + the first argument (``self``) by :opcode:`CALL` when calling the unbound method. Otherwise, ``NULL`` and the object return by the attribute lookup are pushed. .. versionadded:: 3.7 -.. opcode:: PRECALL_METHOD (argc) +.. opcode:: PRECALL (argc) - Prefixes :opcode:`CALL` (possibly with an intervening ``KW_NAMES``). - This opcode is designed to be used with :opcode:`LOAD_METHOD`. - Sets internal variables, so that :opcode:`CALL` - clean up after :opcode:`LOAD_METHOD` correctly. + Prefixes :opcode:`CALL`. Logically this is a no op. + It exists to enable effective specialization of calls. + ``argc`` is the number of arguments as described in :opcode:`CALL`. .. versionadded:: 3.11 -.. opcode:: PRECALL_FUNCTION (args) +.. opcode:: PUSH_NULL - Prefixes :opcode:`CALL` (possibly with an intervening ``KW_NAMES``). - Sets internal variables, so that :opcode:`CALL` can execute correctly. + Pushes a ``NULL`` to the stack. + Used in the call sequence to match the ``NULL`` pushed by + :opcode:`LOAD_METHOD` for non-method calls. .. versionadded:: 3.11 .. opcode:: KW_NAMES (i) + Prefixes :opcode:`PRECALL`. Stores a reference to ``co_consts[consti]`` into an internal variable for use by :opcode:`CALL`. ``co_consts[consti]`` must be a tuple of strings. diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py index 85f50eb958e31..5255d7e19ce8e 100644 --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -386,6 +386,7 @@ def _write_atomic(path, data, mode=0o666): # ROT_TWO/ROT_THREE/ROT_FOUR/ROT_N with SWAP) # Python 3.11a5 3478 (New CALL opcodes) # Python 3.11a5 3479 (Add PUSH_NULL opcode) +# Python 3.11a5 3480 (New CALL opcodes, second iteration) # Python 3.12 will start with magic number 3500 @@ -403,7 +404,7 @@ def _write_atomic(path, data, mode=0o666): # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # in PC/launcher.c must also be updated. -MAGIC_NUMBER = (3479).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3480).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index 0ab2556da2f4f..e81c834878f08 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -109,7 +109,7 @@ def _f(a): LOAD_GLOBAL 0 (print) LOAD_FAST 0 (a) PRECALL 1 - CALL 0 + CALL 1 POP_TOP %3d LOAD_CONST 1 (1) @@ -125,7 +125,7 @@ def _f(a): LOAD_GLOBAL 0 LOAD_FAST 0 PRECALL 1 - CALL 0 + CALL 1 POP_TOP LOAD_CONST 1 RETURN_VALUE @@ -147,7 +147,7 @@ def bug708901(): %3d LOAD_CONST 2 (10) %3d PRECALL 2 - CALL 0 + CALL 2 GET_ITER >> FOR_ITER 2 (to 22) STORE_FAST 0 (res) @@ -319,7 +319,7 @@ def bug42562(): LOAD_NAME 3 (fun) LOAD_CONST 0 (1) PRECALL 1 - CALL 0 + CALL 1 LOAD_NAME 2 (__annotations__) LOAD_CONST 2 ('y') STORE_SUBSCR @@ -330,7 +330,7 @@ def bug42562(): LOAD_NAME 3 (fun) LOAD_CONST 3 (0) PRECALL 1 - CALL 0 + CALL 1 STORE_SUBSCR LOAD_NAME 1 (int) POP_TOP @@ -1164,7 +1164,7 @@ def _prepare_test_cases(): Instruction(opname='BUILD_MAP', opcode=105, arg=0, argval=0, argrepr='', offset=34, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Hello world!', argrepr="'Hello world!'", offset=36, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=7, argval=7, argrepr='', offset=38, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=7, argval=7, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=42, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='f', argrepr='f', offset=44, starts_line=8, is_jump_target=False, positions=None), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=46, starts_line=None, is_jump_target=False, positions=None), @@ -1191,7 +1191,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_DEREF', opcode=137, arg=0, argval='c', argrepr='c', offset=34, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_DEREF', opcode=137, arg=1, argval='d', argrepr='d', offset=36, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=4, argval=4, argrepr='', offset=38, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=4, argval=4, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=42, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='inner', argrepr='inner', offset=44, starts_line=6, is_jump_target=False, positions=None), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=46, starts_line=None, is_jump_target=False, positions=None), @@ -1209,7 +1209,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='e', argrepr='e', offset=16, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=1, argval='f', argrepr='f', offset=18, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=6, argval=6, argrepr='', offset=20, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=22, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=6, argval=6, argrepr='', offset=22, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=24, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=26, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=28, starts_line=None, is_jump_target=False, positions=None), @@ -1221,7 +1221,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='range', argrepr='range', offset=4, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval=10, argrepr='10', offset=6, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=8, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=10, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=10, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='GET_ITER', opcode=68, arg=None, argval=None, argrepr='', offset=12, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='FOR_ITER', opcode=93, arg=19, argval=54, argrepr='to 54', offset=14, starts_line=None, is_jump_target=True, positions=None), Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=16, starts_line=None, is_jump_target=False, positions=None), @@ -1229,7 +1229,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=20, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=22, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=24, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=26, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=26, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=28, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=30, starts_line=5, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=32, starts_line=None, is_jump_target=False, positions=None), @@ -1247,7 +1247,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=56, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='I can haz else clause?', argrepr="'I can haz else clause?'", offset=58, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=60, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=62, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=62, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=64, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=66, starts_line=11, is_jump_target=True, positions=None), Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=57, argval=114, argrepr='to 114', offset=68, starts_line=None, is_jump_target=False, positions=None), @@ -1255,7 +1255,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=72, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=74, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=76, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=78, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=78, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=80, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=82, starts_line=13, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=84, starts_line=None, is_jump_target=False, positions=None), @@ -1277,7 +1277,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=116, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=118, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=120, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=122, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=122, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=124, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=126, starts_line=20, is_jump_target=True, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=128, starts_line=21, is_jump_target=False, positions=None), @@ -1291,13 +1291,13 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=144, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Never reach this', argrepr="'Never reach this'", offset=146, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=148, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=150, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=150, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=152, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=154, starts_line=25, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=156, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=158, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=2, argval=2, argrepr='', offset=160, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=162, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=2, argval=2, argrepr='', offset=162, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=190, argrepr='to 190', offset=166, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=168, starts_line=None, is_jump_target=False, positions=None), @@ -1320,7 +1320,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=202, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=204, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=206, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=208, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=208, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=210, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=212, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='JUMP_FORWARD', opcode=110, arg=12, argval=240, argrepr='to 240', offset=214, starts_line=None, is_jump_target=False, positions=None), @@ -1332,7 +1332,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=226, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=228, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=230, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=232, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=232, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=234, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=236, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=238, starts_line=None, is_jump_target=False, positions=None), @@ -1341,7 +1341,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=244, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=246, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=248, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=250, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=250, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=252, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=254, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=256, starts_line=None, is_jump_target=False, positions=None), @@ -1350,7 +1350,7 @@ def _prepare_test_cases(): Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=262, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=264, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=266, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=0, argval=0, argrepr='', offset=268, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=268, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=270, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=272, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=274, starts_line=None, is_jump_target=False, positions=None), diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-21-10-29-20.bpo-46329.cbkt7u.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-21-10-29-20.bpo-46329.cbkt7u.rst new file mode 100644 index 0000000000000..c04db9460283e --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-21-10-29-20.bpo-46329.cbkt7u.rst @@ -0,0 +1,2 @@ +Move ``KW_NAMES`` before ``PRECALL`` instruction in call sequence. Change +``operand`` of ``CALL`` to match ``PRECALL`` for easier specialization. diff --git a/Programs/test_frozenmain.h b/Programs/test_frozenmain.h index a7f6b6022a49e..11593a9ba3d68 100644 --- a/Programs/test_frozenmain.h +++ b/Programs/test_frozenmain.h @@ -3,11 +3,11 @@ unsigned char M_test_frozenmain[] = { 227,0,0,0,0,0,0,0,0,0,0,0,0,8,0,0, 0,0,0,0,0,115,104,0,0,0,151,0,100,0,100,1, 108,0,90,0,100,0,100,1,108,1,90,1,2,0,101,2, - 100,2,166,1,171,0,1,0,2,0,101,2,100,3,101,0, - 106,3,166,2,171,0,1,0,2,0,101,1,106,4,166,0, + 100,2,166,1,171,1,1,0,2,0,101,2,100,3,101,0, + 106,3,166,2,171,2,1,0,2,0,101,1,106,4,166,0, 171,0,100,4,25,0,90,5,100,5,68,0,93,16,90,6, 2,0,101,2,100,6,101,6,155,0,100,7,101,5,101,6, - 25,0,155,0,157,4,166,1,171,0,1,0,113,33,100,1, + 25,0,155,0,157,4,166,1,171,1,1,0,113,33,100,1, 83,0,41,8,233,0,0,0,0,78,122,18,70,114,111,122, 101,110,32,72,101,108,108,111,32,87,111,114,108,100,122,8, 115,121,115,46,97,114,103,118,218,6,99,111,110,102,105,103, diff --git a/Python/ceval.c b/Python/ceval.c index 471bbde46f9db..d3ab1da76dea4 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1585,15 +1585,19 @@ pop_frame(PyThreadState *tstate, InterpreterFrame *frame) } /* It is only between the PRECALL instruction and the following CALL, - * that these values have any meaning. + * that this has any meaning. */ typedef struct { - PyObject *callable; PyObject *kwnames; - int total_args; - int postcall_shrink; } CallShape; +static inline bool +is_method(PyObject **stack_pointer, int args) { + return PEEK(args+2) != NULL; +} + +#define KWNAMES_LEN() \ + (call_shape.kwnames == NULL ? 0 : ((int)PyTuple_GET_SIZE(call_shape.kwnames))) PyObject* _Py_HOT_FUNCTION _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int throwflag) @@ -1616,11 +1620,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr CFrame cframe; CallShape call_shape; call_shape.kwnames = NULL; // Borrowed reference. Reset by CALL instructions. - /* The following three values are always set by the PRECALL instructions. - They are set here to keep the compiler happy. */ - call_shape.postcall_shrink = 0; - call_shape.total_args = 0; - call_shape.callable = NULL; // Strong reference /* WARNING: Because the CFrame lives on the C stack, * but can be accessed from a heap allocated object (tstate) @@ -4513,23 +4512,31 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int nargs = oparg + is_method; /* Move ownership of reference from stack to call_shape * and make sure that NULL is cleared from stack */ - call_shape.callable = PEEK(nargs + 1); - call_shape.postcall_shrink = 2-is_method; - - call_shape.total_args = nargs; - assert(call_shape.kwnames == NULL); + PyObject *function = PEEK(nargs + 1); #ifdef Py_STATS extern int _PySpecialization_ClassifyCallable(PyObject *); SpecializationStats *stats = &_py_stats.opcode_stats[PRECALL].specialization; stats->failure++; - int kind = _PySpecialization_ClassifyCallable(call_shape.callable); + int kind = _PySpecialization_ClassifyCallable(function); stats->failure_kinds[kind]++; #endif + if (!is_method && Py_TYPE(function) == &PyMethod_Type) { + PyObject *meth = ((PyMethodObject *)function)->im_func; + PyObject *self = ((PyMethodObject *)function)->im_self; + Py_INCREF(meth); + Py_INCREF(self); + PEEK(oparg+1) = self; + PEEK(oparg+2) = meth; + Py_DECREF(function); + function = meth; + } + DISPATCH(); } TARGET(KW_NAMES) { + assert(call_shape.kwnames == NULL); assert(oparg < PyTuple_GET_SIZE(consts)); call_shape.kwnames = GETITEM(consts, oparg); DISPATCH(); @@ -4537,25 +4544,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL) { PREDICTED(CALL); - PyObject *function; - assert((oparg == 0 && call_shape.kwnames == NULL) - || (oparg != 0 && oparg == PyTuple_GET_SIZE(call_shape.kwnames))); + int is_meth; call_function: - function = call_shape.callable; - if (Py_TYPE(function) == &PyMethod_Type) { - PyObject *meth = ((PyMethodObject *)function)->im_func; - PyObject *self = ((PyMethodObject *)function)->im_self; - Py_INCREF(meth); - Py_INCREF(self); - PEEK(call_shape.total_args + 1) = self; - Py_DECREF(function); - function = meth; - call_shape.total_args++; - assert(call_shape.postcall_shrink >= 1); - call_shape.postcall_shrink--; - } - int total_args = call_shape.total_args; - int positional_args = total_args - oparg; + is_meth = is_method(stack_pointer, oparg); + int total_args = oparg + is_meth; + PyObject *function = PEEK(total_args + 1); + int positional_args = total_args - KWNAMES_LEN(); // Check if the call can be inlined or not if (Py_TYPE(function) == &PyFunction_Type && tstate->interp->eval_frame == NULL) { int code_flags = ((PyCodeObject*)PyFunction_GET_CODE(function))->co_flags; @@ -4566,7 +4560,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr stack_pointer, positional_args, call_shape.kwnames ); call_shape.kwnames = NULL; - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); // The frame has stolen all the arguments from the stack, // so there is no need to clean them up. if (new_frame == NULL) { @@ -4599,7 +4593,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr for (int i = 0; i < total_args; i++) { Py_DECREF(stack_pointer[i]); } - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); PUSH(res); if (res == NULL) { goto error; @@ -4610,14 +4604,14 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL_ADAPTIVE) { SpecializedCacheEntry *cache = GET_CACHE(); - int named_args = cache->adaptive.original_oparg; - assert((named_args == 0 && call_shape.kwnames == NULL) - || (named_args != 0 && named_args == PyTuple_GET_SIZE(call_shape.kwnames))); + int original_oparg = cache->adaptive.original_oparg; if (cache->adaptive.counter == 0) { next_instr--; - int nargs = call_shape.total_args; + int is_meth = is_method(stack_pointer, original_oparg); + int nargs = original_oparg + is_meth; + PyObject *callable = PEEK(nargs + 1); int err = _Py_Specialize_CallNoKw( - call_shape.callable, next_instr, nargs, + callable, next_instr, nargs, call_shape.kwnames, cache, BUILTINS()); if (err < 0) { goto error; @@ -4627,7 +4621,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr else { STAT_INC(CALL, deferred); cache->adaptive.counter--; - oparg = named_args; + oparg = original_oparg; goto call_function; } } @@ -4635,10 +4629,13 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL_PY_EXACT_ARGS) { assert(call_shape.kwnames == NULL); SpecializedCacheEntry *caches = GET_CACHE(); - int argcount = call_shape.total_args; - DEOPT_IF(!PyFunction_Check(call_shape.callable), CALL); + int original_oparg = caches->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int argcount = original_oparg + is_meth; + PyObject *callable = PEEK(argcount + 1); + DEOPT_IF(!PyFunction_Check(callable), CALL); _PyCallCache *cache1 = &caches[-1].call; - PyFunctionObject *func = (PyFunctionObject *)call_shape.callable; + PyFunctionObject *func = (PyFunctionObject *)callable; DEOPT_IF(func->func_version != cache1->func_version, CALL); PyCodeObject *code = (PyCodeObject *)func->func_code; DEOPT_IF(code->co_argcount != argcount, CALL); @@ -4654,7 +4651,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr for (int i = argcount; i < code->co_nlocalsplus; i++) { new_frame->localsplus[i] = NULL; } - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); _PyFrame_SetStackPointer(frame, stack_pointer); new_frame->previous = frame; frame = cframe.current_frame = new_frame; @@ -4664,10 +4661,13 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL_PY_WITH_DEFAULTS) { assert(call_shape.kwnames == NULL); SpecializedCacheEntry *caches = GET_CACHE(); - int argcount = call_shape.total_args; - DEOPT_IF(!PyFunction_Check(call_shape.callable), CALL); + int original_oparg = caches->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int argcount = original_oparg + is_meth; + PyObject *callable = PEEK(argcount + 1); + DEOPT_IF(!PyFunction_Check(callable), CALL); _PyCallCache *cache1 = &caches[-1].call; - PyFunctionObject *func = (PyFunctionObject *)call_shape.callable; + PyFunctionObject *func = (PyFunctionObject *)callable; DEOPT_IF(func->func_version != cache1->func_version, CALL); PyCodeObject *code = (PyCodeObject *)func->func_code; DEOPT_IF(argcount > code->co_argcount, CALL); @@ -4691,7 +4691,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr for (int i = code->co_argcount; i < code->co_nlocalsplus; i++) { new_frame->localsplus[i] = NULL; } - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); _PyFrame_SetStackPointer(frame, stack_pointer); new_frame->previous = frame; frame = cframe.current_frame = new_frame; @@ -4701,14 +4701,15 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL_NO_KW_TYPE_1) { assert(call_shape.kwnames == NULL); assert(cframe.use_tracing == 0); - DEOPT_IF(call_shape.total_args != 1, CALL); + assert(GET_CACHE()->adaptive.original_oparg == 1); + DEOPT_IF(is_method(stack_pointer, 1), CALL); PyObject *obj = TOP(); PyObject *callable = SECOND(); DEOPT_IF(callable != (PyObject *)&PyType_Type, CALL); PyObject *res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(callable); Py_DECREF(obj); - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2); SET_TOP(res); NOTRACE_DISPATCH(); } @@ -4716,16 +4717,18 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL_NO_KW_STR_1) { assert(call_shape.kwnames == NULL); assert(cframe.use_tracing == 0); - DEOPT_IF(!PyType_Check(call_shape.callable), CALL); - PyTypeObject *tp = (PyTypeObject *)call_shape.callable; - DEOPT_IF(call_shape.total_args != 1, CALL); + assert(GET_CACHE()->adaptive.original_oparg == 1); + PyObject *callable = PEEK(2); + DEOPT_IF(!PyType_Check(callable), CALL); + PyTypeObject *tp = (PyTypeObject *)callable; + DEOPT_IF(is_method(stack_pointer, 1), CALL); DEOPT_IF(tp != &PyUnicode_Type, CALL); STAT_INC(CALL, hit); PyObject *arg = TOP(); PyObject *res = PyObject_Str(arg); Py_DECREF(arg); Py_DECREF(&PyUnicode_Type); - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2); SET_TOP(res); if (res == NULL) { goto error; @@ -4736,16 +4739,19 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL_NO_KW_TUPLE_1) { assert(call_shape.kwnames == NULL); - DEOPT_IF(!PyType_Check(call_shape.callable), CALL); - PyTypeObject *tp = (PyTypeObject *)call_shape.callable; - DEOPT_IF(call_shape.total_args != 1, CALL); + assert(GET_CACHE()->adaptive.original_oparg == 1); + int is_meth = is_method(stack_pointer, 1); + PyObject *callable = PEEK(2); + DEOPT_IF(!PyType_Check(callable), CALL); + PyTypeObject *tp = (PyTypeObject *)callable; + DEOPT_IF(is_meth, CALL); DEOPT_IF(tp != &PyTuple_Type, CALL); STAT_INC(CALL, hit); PyObject *arg = TOP(); PyObject *res = PySequence_Tuple(arg); Py_DECREF(arg); Py_DECREF(&PyTuple_Type); - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2); SET_TOP(res); if (res == NULL) { goto error; @@ -4755,22 +4761,25 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(CALL_BUILTIN_CLASS) { - DEOPT_IF(!PyType_Check(call_shape.callable), CALL); - PyTypeObject *tp = (PyTypeObject *)call_shape.callable; + int original_oparg = GET_CACHE()->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + int kwnames_len = KWNAMES_LEN(); + PyObject *callable = PEEK(total_args + 1); + DEOPT_IF(!PyType_Check(callable), CALL); + PyTypeObject *tp = (PyTypeObject *)callable; DEOPT_IF(tp->tp_vectorcall == NULL, CALL); STAT_INC(CALL, hit); - int kwnames_len = GET_CACHE()->adaptive.original_oparg; - - int nargs = call_shape.total_args - kwnames_len; - STACK_SHRINK(call_shape.total_args); - PyObject *res = tp->tp_vectorcall((PyObject *)tp, stack_pointer, nargs, call_shape.kwnames); + STACK_SHRINK(total_args); + PyObject *res = tp->tp_vectorcall((PyObject *)tp, stack_pointer, + total_args-kwnames_len, call_shape.kwnames); call_shape.kwnames = NULL; /* Free the arguments. */ - for (int i = 0; i < call_shape.total_args; i++) { + for (int i = 0; i < total_args; i++) { Py_DECREF(stack_pointer[i]); } Py_DECREF(tp); - STACK_SHRINK(call_shape.postcall_shrink-1); + STACK_SHRINK(1-is_meth); SET_TOP(res); if (res == NULL) { goto error; @@ -4783,8 +4792,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr assert(cframe.use_tracing == 0); /* Builtin METH_O functions */ assert(call_shape.kwnames == NULL); - DEOPT_IF(call_shape.total_args != 1, CALL); - PyObject *callable = call_shape.callable; + SpecializedCacheEntry *caches = GET_CACHE(); + int original_oparg = caches->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + DEOPT_IF(total_args != 1, CALL); + PyObject *callable = PEEK(total_args + 1); DEOPT_IF(!PyCFunction_CheckExact(callable), CALL); DEOPT_IF(PyCFunction_GET_FLAGS(callable) != METH_O, CALL); STAT_INC(CALL, hit); @@ -4802,7 +4815,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr Py_DECREF(arg); Py_DECREF(callable); - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); SET_TOP(res); if (res == NULL) { goto error; @@ -4815,27 +4828,30 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr assert(cframe.use_tracing == 0); /* Builtin METH_FASTCALL functions, without keywords */ assert(call_shape.kwnames == NULL); - PyObject *callable = call_shape.callable; + SpecializedCacheEntry *caches = GET_CACHE(); + int original_oparg = caches->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + PyObject *callable = PEEK(total_args + 1); DEOPT_IF(!PyCFunction_CheckExact(callable), CALL); DEOPT_IF(PyCFunction_GET_FLAGS(callable) != METH_FASTCALL, CALL); STAT_INC(CALL, hit); - int nargs = call_shape.total_args; PyCFunction cfunc = PyCFunction_GET_FUNCTION(callable); - STACK_SHRINK(nargs); + STACK_SHRINK(total_args); /* res = func(self, args, nargs) */ PyObject *res = ((_PyCFunctionFast)(void(*)(void))cfunc)( PyCFunction_GET_SELF(callable), stack_pointer, - nargs); + total_args); assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); /* Free the arguments. */ - for (int i = 0; i < nargs; i++) { + for (int i = 0; i < total_args; i++) { Py_DECREF(stack_pointer[i]); } - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); PUSH(res); Py_DECREF(callable); if (res == NULL) { @@ -4853,19 +4869,16 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL_BUILTIN_FAST_WITH_KEYWORDS) { assert(cframe.use_tracing == 0); /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ - PyObject *callable = call_shape.callable; + SpecializedCacheEntry *caches = GET_CACHE(); + int original_oparg = caches->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + PyObject *callable = PEEK(total_args + 1); DEOPT_IF(!PyCFunction_CheckExact(callable), CALL); DEOPT_IF(PyCFunction_GET_FLAGS(callable) != (METH_FASTCALL | METH_KEYWORDS), CALL); STAT_INC(CALL, hit); - int kwnames_len = GET_CACHE()->adaptive.original_oparg; - assert( - (call_shape.kwnames == NULL && kwnames_len == 0) || - (call_shape.kwnames != NULL && - PyTuple_GET_SIZE(call_shape.kwnames) == kwnames_len) - ); - int nargs = call_shape.total_args - kwnames_len; - STACK_SHRINK(call_shape.total_args); + STACK_SHRINK(total_args); /* res = func(self, args, nargs, kwnames) */ _PyCFunctionFastWithKeywords cfunc = (_PyCFunctionFastWithKeywords)(void(*)(void)) @@ -4873,17 +4886,17 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyObject *res = cfunc( PyCFunction_GET_SELF(callable), stack_pointer, - nargs, + total_args - KWNAMES_LEN(), call_shape.kwnames ); assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); call_shape.kwnames = NULL; /* Free the arguments. */ - for (int i = 0; i < call_shape.total_args; i++) { + for (int i = 0; i < total_args; i++) { Py_DECREF(stack_pointer[i]); } - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); PUSH(res); Py_DECREF(callable); if (res == NULL) { @@ -4898,11 +4911,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr assert(call_shape.kwnames == NULL); /* len(o) */ SpecializedCacheEntry *caches = GET_CACHE(); - DEOPT_IF(call_shape.total_args != 1, CALL); - assert(caches[0].adaptive.original_oparg == 0); + int original_oparg = caches->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + DEOPT_IF(total_args != 1, CALL); _PyObjectCache *cache1 = &caches[-1].obj; - - PyObject *callable = call_shape.callable; + PyObject *callable = PEEK(total_args + 1); DEOPT_IF(callable != cache1->obj, CALL); STAT_INC(CALL, hit); @@ -4914,7 +4928,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyObject *res = PyLong_FromSsize_t(len_i); assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); SET_TOP(res); Py_DECREF(callable); Py_DECREF(arg); @@ -4929,11 +4943,14 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr assert(call_shape.kwnames == NULL); /* isinstance(o, o2) */ SpecializedCacheEntry *caches = GET_CACHE(); - assert(caches[0].adaptive.original_oparg == 0); - DEOPT_IF(call_shape.total_args != 2, CALL); + int original_oparg = caches->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + PyObject *callable = PEEK(total_args + 1); + DEOPT_IF(total_args != 2, CALL); _PyObjectCache *cache1 = &caches[-1].obj; - DEOPT_IF(call_shape.callable != cache1->obj, CALL); + DEOPT_IF(callable != cache1->obj, CALL); STAT_INC(CALL, hit); PyObject *cls = POP(); @@ -4946,11 +4963,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyObject *res = PyBool_FromLong(retval); assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); SET_TOP(res); Py_DECREF(inst); Py_DECREF(cls); - Py_DECREF(call_shape.callable); + Py_DECREF(callable); if (res == NULL) { goto error; } @@ -4961,9 +4978,13 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr assert(cframe.use_tracing == 0); assert(call_shape.kwnames == NULL); SpecializedCacheEntry *caches = GET_CACHE(); + int original_oparg = caches->adaptive.original_oparg; _PyObjectCache *cache1 = &caches[-1].obj; - DEOPT_IF(call_shape.total_args != 2, CALL); - DEOPT_IF(call_shape.callable != cache1->obj, CALL); + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + PyObject *callable = PEEK(total_args + 1); + DEOPT_IF(total_args != 2, CALL); + DEOPT_IF(callable != cache1->obj, CALL); PyObject *list = SECOND(); DEOPT_IF(!PyList_Check(list), CALL); STAT_INC(CALL, hit); @@ -4974,18 +4995,22 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } Py_DECREF(arg); Py_DECREF(list); - STACK_SHRINK(call_shape.postcall_shrink+1); + STACK_SHRINK(3-is_meth); Py_INCREF(Py_None); SET_TOP(Py_None); - Py_DECREF(call_shape.callable); + Py_DECREF(callable); NOTRACE_DISPATCH(); } TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { assert(call_shape.kwnames == NULL); - DEOPT_IF(call_shape.total_args != 2, CALL); - DEOPT_IF(!Py_IS_TYPE(call_shape.callable, &PyMethodDescr_Type), CALL); - PyMethodDef *meth = ((PyMethodDescrObject *)call_shape.callable)->d_method; + int original_oparg = GET_CACHE()->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + PyObject *callable = PEEK(total_args + 1); + DEOPT_IF(total_args != 2, CALL); + DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), CALL); + PyMethodDef *meth = ((PyMethodDescrObject *)callable)->d_method; DEOPT_IF(meth->ml_flags != METH_O, CALL); STAT_INC(CALL, hit); PyCFunction cfunc = meth->ml_meth; @@ -5001,9 +5026,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); Py_DECREF(self); Py_DECREF(arg); - STACK_SHRINK(call_shape.postcall_shrink+1); + STACK_SHRINK(3-is_meth); SET_TOP(res); - Py_DECREF(call_shape.callable); + Py_DECREF(callable); if (res == NULL) { goto error; } @@ -5013,9 +5038,13 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS) { assert(call_shape.kwnames == NULL); - DEOPT_IF(call_shape.total_args != 1, CALL); - DEOPT_IF(!Py_IS_TYPE(call_shape.callable, &PyMethodDescr_Type), CALL); - PyMethodDef *meth = ((PyMethodDescrObject *)call_shape.callable)->d_method; + int original_oparg = GET_CACHE()->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + PyObject *callable = PEEK(total_args + 1); + DEOPT_IF(total_args != 1, CALL); + DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), CALL); + PyMethodDef *meth = ((PyMethodDescrObject *)callable)->d_method; DEOPT_IF(meth->ml_flags != METH_NOARGS, CALL); STAT_INC(CALL, hit); PyCFunction cfunc = meth->ml_meth; @@ -5029,9 +5058,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr _Py_LeaveRecursiveCall(tstate); assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); Py_DECREF(self); - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); SET_TOP(res); - Py_DECREF(call_shape.callable); + Py_DECREF(callable); if (res == NULL) { goto error; } @@ -5041,13 +5070,17 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_FAST) { assert(call_shape.kwnames == NULL); + int original_oparg = GET_CACHE()->adaptive.original_oparg; + int is_meth = is_method(stack_pointer, original_oparg); + int total_args = original_oparg + is_meth; + PyObject *callable = PEEK(total_args + 1); /* Builtin METH_FASTCALL methods, without keywords */ - DEOPT_IF(!Py_IS_TYPE(call_shape.callable, &PyMethodDescr_Type), CALL); - PyMethodDef *meth = ((PyMethodDescrObject *)call_shape.callable)->d_method; + DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), CALL); + PyMethodDef *meth = ((PyMethodDescrObject *)callable)->d_method; DEOPT_IF(meth->ml_flags != METH_FASTCALL, CALL); STAT_INC(CALL, hit); _PyCFunctionFast cfunc = (_PyCFunctionFast)(void(*)(void))meth->ml_meth; - int nargs = call_shape.total_args-1; + int nargs = total_args-1; STACK_SHRINK(nargs); PyObject *self = TOP(); PyObject *res = cfunc(self, stack_pointer, nargs); @@ -5057,9 +5090,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr Py_DECREF(stack_pointer[i]); } Py_DECREF(self); - STACK_SHRINK(call_shape.postcall_shrink); + STACK_SHRINK(2-is_meth); SET_TOP(res); - Py_DECREF(call_shape.callable); + Py_DECREF(callable); if (res == NULL) { goto error; } diff --git a/Python/compile.c b/Python/compile.c index 645213b192eae..7f0a6f096d723 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -1800,7 +1800,7 @@ compiler_call_exit_with_nones(struct compiler *c) { ADDOP_LOAD_CONST(c, Py_None); ADDOP_LOAD_CONST(c, Py_None); ADDOP_I(c, PRECALL, 2); - ADDOP_I(c, CALL, 0); + ADDOP_I(c, CALL, 2); return 1; } @@ -4679,16 +4679,12 @@ maybe_optimize_method_call(struct compiler *c, expr_ty e) if (kwdsl) { VISIT_SEQ(c, keyword, kwds); - ADDOP_I(c, PRECALL, argsl + kwdsl); if (!compiler_call_simple_kw_helper(c, kwds, kwdsl)) { return 0; }; - ADDOP_I(c, CALL, kwdsl); - } - else { - ADDOP_I(c, PRECALL, argsl); - ADDOP_I(c, CALL, 0); } + ADDOP_I(c, PRECALL, argsl + kwdsl); + ADDOP_I(c, CALL, argsl + kwdsl); c->u->u_lineno = old_lineno; return 1; } @@ -4758,7 +4754,7 @@ compiler_joined_str(struct compiler *c, expr_ty e) ADDOP_I(c, LIST_APPEND, 1); } ADDOP_I(c, PRECALL, 1); - ADDOP_I(c, CALL, 0); + ADDOP_I(c, CALL, 1); } else { VISIT_SEQ(c, expr, e->v.JoinedStr.values); @@ -4927,18 +4923,13 @@ compiler_call_helper(struct compiler *c, } if (nkwelts) { VISIT_SEQ(c, keyword, keywords); - ADDOP_I(c, PRECALL, n + nelts + nkwelts); if (!compiler_call_simple_kw_helper(c, keywords, nkwelts)) { return 0; }; - ADDOP_I(c, CALL, nkwelts); - return 1; - } - else { - ADDOP_I(c, PRECALL, n + nelts); - ADDOP_I(c, CALL, 0); - return 1; } + ADDOP_I(c, PRECALL, n + nelts + nkwelts); + ADDOP_I(c, CALL, n + nelts + nkwelts); + return 1; ex_call: From webhook-mailer at python.org Mon Feb 21 13:46:11 2022 From: webhook-mailer at python.org (ned-deily) Date: Mon, 21 Feb 2022 18:46:11 -0000 Subject: [Python-checkins] bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) (GH-31298) Message-ID: https://github.com/python/cpython/commit/5fdacac8cecb123ae12669ceb3504b2f41075c20 commit: 5fdacac8cecb123ae12669ceb3504b2f41075c20 branch: 3.7 author: Dong-hee Na committer: ned-deily date: 2022-02-21T13:45:55-05:00 summary: bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) (GH-31298) Co-authored-by: Cyril Jouve files: A Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst M Modules/expat/expat.h M Modules/expat/xmlparse.c M Modules/expat/xmlrole.c M Modules/expat/xmltok.c M Modules/expat/xmltok_ns.c diff --git a/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst new file mode 100644 index 0000000000000..9c1f24c0e5171 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-30-15-16-12.bpo-46400.vweUiO.rst @@ -0,0 +1 @@ +expat: Update libexpat from 2.4.1 to 2.4.4 diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index b7d6d354801b3..4c5704fd9336b 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -11,7 +11,7 @@ Copyright (c) 2000-2005 Fred L. Drake, Jr. Copyright (c) 2001-2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Cristian Rodr?guez Copyright (c) 2016 Thomas Beutlich Copyright (c) 2017 Rhodri James @@ -1041,7 +1041,7 @@ XML_SetBillionLaughsAttackProtectionActivationThreshold( */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 4 -#define XML_MICRO_VERSION 1 +#define XML_MICRO_VERSION 4 #ifdef __cplusplus } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 5ba56eaea6357..4b43e61321691 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 8539b9040d9d901366a62560a064af7cb99811335784b363abc039c5b0ebc416 (2.4.1+) +/* 2e2c8ce5f11a473d65ec313ab20ceee6afefb355f5405afc06e7204e2e41c8c0 (2.4.4+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -13,7 +13,7 @@ Copyright (c) 2002-2016 Karl Waclawek Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016 Eric Rahm - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Gaurav Copyright (c) 2016 Thomas Beutlich Copyright (c) 2016 Gustavo Grieco @@ -32,6 +32,8 @@ Copyright (c) 2019 David Loffredo Copyright (c) 2019-2020 Ben Wagner Copyright (c) 2019 Vadim Zeitlin + Copyright (c) 2021 Dong-hee Na + Copyright (c) 2022 Samanta Navarro Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -54,6 +56,10 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#define XML_BUILDING_EXPAT 1 + +#include + #if ! defined(_GNU_SOURCE) # define _GNU_SOURCE 1 /* syscall prototype */ #endif @@ -84,14 +90,10 @@ # include #endif -#define XML_BUILDING_EXPAT 1 - #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "ascii.h" #include "expat.h" #include "siphash.h" @@ -973,7 +975,7 @@ parserCreate(const XML_Char *encodingName, if (memsuite) { XML_Memory_Handling_Suite *mtemp; - parser = (XML_Parser)memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); + parser = memsuite->malloc_fcn(sizeof(struct XML_ParserStruct)); if (parser != NULL) { mtemp = (XML_Memory_Handling_Suite *)&(parser->m_mem); mtemp->malloc_fcn = memsuite->malloc_fcn; @@ -2066,6 +2068,11 @@ XML_GetBuffer(XML_Parser parser, int len) { keep = (int)EXPAT_SAFE_PTR_DIFF(parser->m_bufferPtr, parser->m_buffer); if (keep > XML_CONTEXT_BYTES) keep = XML_CONTEXT_BYTES; + /* Detect and prevent integer overflow */ + if (keep > INT_MAX - neededSize) { + parser->m_errorCode = XML_ERROR_NO_MEMORY; + return NULL; + } neededSize += keep; #endif /* defined XML_CONTEXT_BYTES */ if (neededSize @@ -3260,13 +3267,38 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, /* get the attributes from the tokenizer */ n = XmlGetAttributes(enc, attStr, parser->m_attsSize, parser->m_atts); + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - nDefaultAtts) { + return XML_ERROR_NO_MEMORY; + } + if (n + nDefaultAtts > parser->m_attsSize) { int oldAttsSize = parser->m_attsSize; ATTRIBUTE *temp; #ifdef XML_ATTR_INFO XML_AttrInfo *temp2; #endif + + /* Detect and prevent integer overflow */ + if ((nDefaultAtts > INT_MAX - INIT_ATTS_SIZE) + || (n > INT_MAX - (nDefaultAtts + INIT_ATTS_SIZE))) { + return XML_ERROR_NO_MEMORY; + } + parser->m_attsSize = n + nDefaultAtts + INIT_ATTS_SIZE; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(ATTRIBUTE)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (ATTRIBUTE *)REALLOC(parser, (void *)parser->m_atts, parser->m_attsSize * sizeof(ATTRIBUTE)); if (temp == NULL) { @@ -3275,6 +3307,17 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, } parser->m_atts = temp; #ifdef XML_ATTR_INFO + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +# if UINT_MAX >= SIZE_MAX + if ((unsigned)parser->m_attsSize > (size_t)(-1) / sizeof(XML_AttrInfo)) { + parser->m_attsSize = oldAttsSize; + return XML_ERROR_NO_MEMORY; + } +# endif + temp2 = (XML_AttrInfo *)REALLOC(parser, (void *)parser->m_attInfo, parser->m_attsSize * sizeof(XML_AttrInfo)); if (temp2 == NULL) { @@ -3413,7 +3456,13 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, if (nPrefixes) { int j; /* hash table index */ unsigned long version = parser->m_nsAttsVersion; - int nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(unsigned int) * 8 /* bits per byte */) { + return XML_ERROR_NO_MEMORY; + } + + unsigned int nsAttsSize = 1u << parser->m_nsAttsPower; unsigned char oldNsAttsPower = parser->m_nsAttsPower; /* size of hash table must be at least 2 * (# of prefixed attributes) */ if ((nPrefixes << 1) @@ -3424,7 +3473,28 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, ; if (parser->m_nsAttsPower < 3) parser->m_nsAttsPower = 3; - nsAttsSize = (int)1 << parser->m_nsAttsPower; + + /* Detect and prevent invalid shift */ + if (parser->m_nsAttsPower >= sizeof(nsAttsSize) * 8 /* bits per byte */) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } + + nsAttsSize = 1u << parser->m_nsAttsPower; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (nsAttsSize > (size_t)(-1) / sizeof(NS_ATT)) { + /* Restore actual size of memory in m_nsAtts */ + parser->m_nsAttsPower = oldNsAttsPower; + return XML_ERROR_NO_MEMORY; + } +#endif + temp = (NS_ATT *)REALLOC(parser, parser->m_nsAtts, nsAttsSize * sizeof(NS_ATT)); if (! temp) { @@ -3582,9 +3652,31 @@ storeAtts(XML_Parser parser, const ENCODING *enc, const char *attStr, tagNamePtr->prefixLen = prefixLen; for (i = 0; localPart[i++];) ; /* i includes null terminator */ + + /* Detect and prevent integer overflow */ + if (binding->uriLen > INT_MAX - prefixLen + || i > INT_MAX - (binding->uriLen + prefixLen)) { + return XML_ERROR_NO_MEMORY; + } + n = i + binding->uriLen + prefixLen; if (n > binding->uriAlloc) { TAG *p; + + /* Detect and prevent integer overflow */ + if (n > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(n + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + uri = (XML_Char *)MALLOC(parser, (n + EXPAND_SPARE) * sizeof(XML_Char)); if (! uri) return XML_ERROR_NO_MEMORY; @@ -3680,6 +3772,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (parser->m_freeBindingList) { b = parser->m_freeBindingList; if (len > b->uriAlloc) { + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + XML_Char *temp = (XML_Char *)REALLOC( parser, b->uri, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (temp == NULL) @@ -3692,6 +3799,21 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, b = (BINDING *)MALLOC(parser, sizeof(BINDING)); if (! b) return XML_ERROR_NO_MEMORY; + + /* Detect and prevent integer overflow */ + if (len > INT_MAX - EXPAND_SPARE) { + return XML_ERROR_NO_MEMORY; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)(len + EXPAND_SPARE) > (size_t)(-1) / sizeof(XML_Char)) { + return XML_ERROR_NO_MEMORY; + } +#endif + b->uri = (XML_Char *)MALLOC(parser, sizeof(XML_Char) * (len + EXPAND_SPARE)); if (! b->uri) { @@ -3976,7 +4098,7 @@ initializeEncoding(XML_Parser parser) { const char *s; #ifdef XML_UNICODE char encodingBuf[128]; - /* See comments abount `protoclEncodingName` in parserInit() */ + /* See comments about `protocolEncodingName` in parserInit() */ if (! parser->m_protocolEncodingName) s = NULL; else { @@ -5018,6 +5140,11 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (parser->m_prologState.level >= parser->m_groupSize) { if (parser->m_groupSize) { { + /* Detect and prevent integer overflow */ + if (parser->m_groupSize > (unsigned int)(-1) / 2u) { + return XML_ERROR_NO_MEMORY; + } + char *const new_connector = (char *)REALLOC( parser, parser->m_groupConnector, parser->m_groupSize *= 2); if (new_connector == NULL) { @@ -5028,6 +5155,16 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, } if (dtd->scaffIndex) { + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (parser->m_groupSize > (size_t)(-1) / sizeof(int)) { + return XML_ERROR_NO_MEMORY; + } +#endif + int *const new_scaff_index = (int *)REALLOC( parser, dtd->scaffIndex, parser->m_groupSize * sizeof(int)); if (new_scaff_index == NULL) @@ -5236,7 +5373,7 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, if (dtd->in_eldecl) { ELEMENT_TYPE *el; const XML_Char *name; - int nameLen; + size_t nameLen; const char *nxt = (quant == XML_CQUANT_NONE ? next : next - enc->minBytesPerChar); int myindex = nextScaffoldPart(parser); @@ -5252,7 +5389,13 @@ doProlog(XML_Parser parser, const ENCODING *enc, const char *s, const char *end, nameLen = 0; for (; name[nameLen++];) ; - dtd->contentStringLen += nameLen; + + /* Detect and prevent integer overflow */ + if (nameLen > UINT_MAX - dtd->contentStringLen) { + return XML_ERROR_NO_MEMORY; + } + + dtd->contentStringLen += (unsigned)nameLen; if (parser->m_elementDeclHandler) handleDefault = XML_FALSE; } @@ -6098,7 +6241,24 @@ defineAttribute(ELEMENT_TYPE *type, ATTRIBUTE_ID *attId, XML_Bool isCdata, } } else { DEFAULT_ATTRIBUTE *temp; + + /* Detect and prevent integer overflow */ + if (type->allocDefaultAtts > INT_MAX / 2) { + return 0; + } + int count = type->allocDefaultAtts * 2; + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if ((unsigned)count > (size_t)(-1) / sizeof(DEFAULT_ATTRIBUTE)) { + return 0; + } +#endif + temp = (DEFAULT_ATTRIBUTE *)REALLOC(parser, type->defaultAtts, (count * sizeof(DEFAULT_ATTRIBUTE))); if (temp == NULL) @@ -6388,7 +6548,7 @@ normalizePublicId(XML_Char *publicId) { static DTD * dtdCreate(const XML_Memory_Handling_Suite *ms) { - DTD *p = (DTD *)ms->malloc_fcn(sizeof(DTD)); + DTD *p = ms->malloc_fcn(sizeof(DTD)); if (p == NULL) return p; poolInit(&(p->pool), ms); @@ -6561,8 +6721,8 @@ dtdCopy(XML_Parser oldParser, DTD *newDtd, const DTD *oldDtd, if (! newE) return 0; if (oldE->nDefaultAtts) { - newE->defaultAtts = (DEFAULT_ATTRIBUTE *)ms->malloc_fcn( - oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); + newE->defaultAtts + = ms->malloc_fcn(oldE->nDefaultAtts * sizeof(DEFAULT_ATTRIBUTE)); if (! newE->defaultAtts) { return 0; } @@ -6724,7 +6884,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* table->size is a power of 2 */ table->size = (size_t)1 << INIT_POWER; tsize = table->size * sizeof(NAMED *); - table->v = (NAMED **)table->mem->malloc_fcn(tsize); + table->v = table->mem->malloc_fcn(tsize); if (! table->v) { table->size = 0; return NULL; @@ -6749,10 +6909,22 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { /* check for overflow (table is half full) */ if (table->used >> (table->power - 1)) { unsigned char newPower = table->power + 1; + + /* Detect and prevent invalid shift */ + if (newPower >= sizeof(unsigned long) * 8 /* bits per byte */) { + return NULL; + } + size_t newSize = (size_t)1 << newPower; unsigned long newMask = (unsigned long)newSize - 1; + + /* Detect and prevent integer overflow */ + if (newSize > (size_t)(-1) / sizeof(NAMED *)) { + return NULL; + } + size_t tsize = newSize * sizeof(NAMED *); - NAMED **newV = (NAMED **)table->mem->malloc_fcn(tsize); + NAMED **newV = table->mem->malloc_fcn(tsize); if (! newV) return NULL; memset(newV, 0, tsize); @@ -6781,7 +6953,7 @@ lookup(XML_Parser parser, HASH_TABLE *table, KEY name, size_t createSize) { } } } - table->v[i] = (NAMED *)table->mem->malloc_fcn(createSize); + table->v[i] = table->mem->malloc_fcn(createSize); if (! table->v[i]) return NULL; memset(table->v[i], 0, createSize); @@ -7069,7 +7241,7 @@ poolGrow(STRING_POOL *pool) { if (bytesToAllocate == 0) return XML_FALSE; - tem = (BLOCK *)pool->mem->malloc_fcn(bytesToAllocate); + tem = pool->mem->malloc_fcn(bytesToAllocate); if (! tem) return XML_FALSE; tem->size = blockSize; @@ -7100,6 +7272,20 @@ nextScaffoldPart(XML_Parser parser) { if (dtd->scaffCount >= dtd->scaffSize) { CONTENT_SCAFFOLD *temp; if (dtd->scaffold) { + /* Detect and prevent integer overflow */ + if (dtd->scaffSize > UINT_MAX / 2u) { + return -1; + } + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffSize > (size_t)(-1) / 2u / sizeof(CONTENT_SCAFFOLD)) { + return -1; + } +#endif + temp = (CONTENT_SCAFFOLD *)REALLOC( parser, dtd->scaffold, dtd->scaffSize * 2 * sizeof(CONTENT_SCAFFOLD)); if (temp == NULL) @@ -7169,8 +7355,26 @@ build_model(XML_Parser parser) { XML_Content *ret; XML_Content *cpos; XML_Char *str; - int allocsize = (dtd->scaffCount * sizeof(XML_Content) - + (dtd->contentStringLen * sizeof(XML_Char))); + + /* Detect and prevent integer overflow. + * The preprocessor guard addresses the "always false" warning + * from -Wtype-limits on platforms where + * sizeof(unsigned int) < sizeof(size_t), e.g. on x86_64. */ +#if UINT_MAX >= SIZE_MAX + if (dtd->scaffCount > (size_t)(-1) / sizeof(XML_Content)) { + return NULL; + } + if (dtd->contentStringLen > (size_t)(-1) / sizeof(XML_Char)) { + return NULL; + } +#endif + if (dtd->scaffCount * sizeof(XML_Content) + > (size_t)(-1) - dtd->contentStringLen * sizeof(XML_Char)) { + return NULL; + } + + const size_t allocsize = (dtd->scaffCount * sizeof(XML_Content) + + (dtd->contentStringLen * sizeof(XML_Char))); ret = (XML_Content *)MALLOC(parser, allocsize); if (! ret) diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 08173b0fd541d..77746ee42d10a 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -15,6 +15,7 @@ Copyright (c) 2016-2021 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -37,14 +38,14 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#include + #include #ifdef _WIN32 # include "winconfig.h" #endif -#include - #include "expat_external.h" #include "internal.h" #include "xmlrole.h" diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index f2b6b406067ea..502ca1adc33b9 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -20,6 +20,7 @@ Copyright (c) 2017 Benbuck Nason Copyright (c) 2017 Jos? Guti?rrez de la Concha Copyright (c) 2019 David Loffredo + Copyright (c) 2021 Dong-hee Na Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -42,6 +43,8 @@ USE OR OTHER DEALINGS IN THE SOFTWARE. */ +#include + #include #include /* memcpy */ #include @@ -50,8 +53,6 @@ # include "winconfig.h" #endif -#include - #include "expat_external.h" #include "internal.h" #include "xmltok.h" diff --git a/Modules/expat/xmltok_ns.c b/Modules/expat/xmltok_ns.c index 5fd8392235940..fbdd3e3c7b799 100644 --- a/Modules/expat/xmltok_ns.c +++ b/Modules/expat/xmltok_ns.c @@ -11,7 +11,7 @@ Copyright (c) 2002 Greg Stein Copyright (c) 2002 Fred L. Drake, Jr. Copyright (c) 2002-2006 Karl Waclawek - Copyright (c) 2017 Sebastian Pipping + Copyright (c) 2017-2021 Sebastian Pipping Licensed under the MIT license: Permission is hereby granted, free of charge, to any person obtaining @@ -93,7 +93,7 @@ NS(XmlInitEncoding)(INIT_ENCODING *p, const ENCODING **encPtr, static const ENCODING * NS(findEncoding)(const ENCODING *enc, const char *ptr, const char *end) { # define ENCODING_MAX 128 - char buf[ENCODING_MAX]; + char buf[ENCODING_MAX] = ""; char *p = buf; int i; XmlUtf8Convert(enc, &ptr, end, &p, p + ENCODING_MAX - 1); From webhook-mailer at python.org Mon Feb 21 14:03:17 2022 From: webhook-mailer at python.org (ned-deily) Date: Mon, 21 Feb 2022 19:03:17 -0000 Subject: [Python-checkins] bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31471) Message-ID: https://github.com/python/cpython/commit/d4f5bb912e67299b59b814b89a5afd9a8821a14e commit: d4f5bb912e67299b59b814b89a5afd9a8821a14e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ned-deily date: 2022-02-21T14:03:08-05:00 summary: bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) (GH-31471) Curly brackets were never allowed in namespace URIs according to RFC 3986, and so-called namespace-validating XML parsers have the right to reject them a invalid URIs. libexpat >=2.4.5 has become strcter in that regard due to related security issues; with ET.XML instantiating a namespace-aware parser under the hood, this test has no future in CPython. References: - https://datatracker.ietf.org/doc/html/rfc3968 - https://www.w3.org/TR/xml-names/ Also, test_minidom.py: Support Expat >=2.4.5 (cherry picked from commit 2cae93832f46b245847bdc252456ddf7742ef45e) Co-authored-by: Sebastian Pipping files: A Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst M Lib/test/test_minidom.py M Lib/test/test_xml_etree.py diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py index 041a41511314a..0b76f6e87bc50 100644 --- a/Lib/test/test_minidom.py +++ b/Lib/test/test_minidom.py @@ -5,10 +5,12 @@ from test import support import unittest +import pyexpat import xml.dom.minidom from xml.dom.minidom import parse, Node, Document, parseString from xml.dom.minidom import getDOMImplementation +from xml.parsers.expat import ExpatError tstfile = support.findfile("test.xml", subdir="xmltestdata") @@ -1146,7 +1148,13 @@ def testEncodings(self): # Verify that character decoding errors raise exceptions instead # of crashing - self.assertRaises(UnicodeDecodeError, parseString, + if pyexpat.version_info >= (2, 4, 5): + self.assertRaises(ExpatError, parseString, + b'') + self.assertRaises(ExpatError, parseString, + b'Comment \xe7a va ? Tr\xe8s bien ?') + else: + self.assertRaises(UnicodeDecodeError, parseString, b'Comment \xe7a va ? Tr\xe8s bien ?') doc.unlink() @@ -1592,7 +1600,12 @@ def testEmptyXMLNSValue(self): self.confirm(doc2.namespaceURI == xml.dom.EMPTY_NAMESPACE) def testExceptionOnSpacesInXMLNSValue(self): - with self.assertRaisesRegex(ValueError, 'Unsupported syntax'): + if pyexpat.version_info >= (2, 4, 5): + context = self.assertRaisesRegex(ExpatError, 'syntax error') + else: + context = self.assertRaisesRegex(ValueError, 'Unsupported syntax') + + with context: parseString('') def testDocRemoveChild(self): diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index 75c9c25a8b639..5ba0de82cdf9a 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -1683,12 +1683,6 @@ def test_issue6233(self): b"\n" b'tãg') - def test_issue3151(self): - e = ET.XML('') - self.assertEqual(e.tag, '{${stuff}}localname') - t = ET.ElementTree(e) - self.assertEqual(ET.tostring(e), b'') - def test_issue6565(self): elem = ET.XML("") self.assertEqual(summarize_list(elem), ['tag']) diff --git a/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst new file mode 100644 index 0000000000000..6969bd1898f65 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst @@ -0,0 +1 @@ +Make test suite support Expat >=2.4.5 From webhook-mailer at python.org Mon Feb 21 14:18:36 2022 From: webhook-mailer at python.org (ned-deily) Date: Mon, 21 Feb 2022 19:18:36 -0000 Subject: [Python-checkins] bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397) (GH-31418) Message-ID: https://github.com/python/cpython/commit/61f3c308e435c5294e674ef59fed1d51f47e0089 commit: 61f3c308e435c5294e674ef59fed1d51f47e0089 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ned-deily date: 2022-02-21T14:18:26-05:00 summary: bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397) (GH-31418) The libexpat 2.4.1 upgrade from introduced the following new exported symbols: * `testingAccountingGetCountBytesDirect` * `testingAccountingGetCountBytesIndirect` * `unsignedCharToPrintable` * `XML_SetBillionLaughsAttackProtectionActivationThreshold` * `XML_SetBillionLaughsAttackProtectionMaximumAmplification` We need to adjust [Modules/expat/pyexpatns.h](https://github.com/python/cpython/blob/master/Modules/expat/pyexpatns.h) (The newer libexpat upgrade has no new symbols). Automerge-Triggered-By: GH:gpshead (cherry picked from commit 6312c1052c0186b4596fc45c42fd3ade9f8f5911) Co-authored-by: Yilei "Dolee" Yang files: A Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst M Modules/expat/pyexpatns.h diff --git a/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst b/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst new file mode 100644 index 0000000000000..d190816637ae8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-18-22-10-30.bpo-46784.SVOQJx.rst @@ -0,0 +1 @@ +Fix libexpat symbols collisions with user dynamically loaded or statically linked libexpat in embedded Python. diff --git a/Modules/expat/pyexpatns.h b/Modules/expat/pyexpatns.h index cfb742ee000b0..d45d9b6c45715 100644 --- a/Modules/expat/pyexpatns.h +++ b/Modules/expat/pyexpatns.h @@ -38,6 +38,9 @@ #ifndef PYEXPATNS_H #define PYEXPATNS_H +#define testingAccountingGetCountBytesDirect PyExpat_testingAccountingGetCountBytesDirect +#define testingAccountingGetCountBytesIndirect PyExpat_testingAccountingGetCountBytesIndirect +#define unsignedCharToPrintable PyExpat_unsignedCharToPrintable #define XML_DefaultCurrent PyExpat_XML_DefaultCurrent #define XML_ErrorString PyExpat_XML_ErrorString #define XML_ExpatVersion PyExpat_XML_ExpatVersion @@ -81,6 +84,8 @@ #define XML_ResumeParser PyExpat_XML_ResumeParser #define XML_SetAttlistDeclHandler PyExpat_XML_SetAttlistDeclHandler #define XML_SetBase PyExpat_XML_SetBase +#define XML_SetBillionLaughsAttackProtectionActivationThreshold PyExpat_XML_SetBillionLaughsAttackProtectionActivationThreshold +#define XML_SetBillionLaughsAttackProtectionMaximumAmplification PyExpat_XML_SetBillionLaughsAttackProtectionMaximumAmplification #define XML_SetCdataSectionHandler PyExpat_XML_SetCdataSectionHandler #define XML_SetCharacterDataHandler PyExpat_XML_SetCharacterDataHandler #define XML_SetCommentHandler PyExpat_XML_SetCommentHandler From webhook-mailer at python.org Mon Feb 21 14:44:57 2022 From: webhook-mailer at python.org (ned-deily) Date: Mon, 21 Feb 2022 19:44:57 -0000 Subject: [Python-checkins] Update copyright year to 2022. (GH-30335) (GH-31477) Message-ID: https://github.com/python/cpython/commit/24a549fcea682b8de88adf79924ccbc3a448cb0d commit: 24a549fcea682b8de88adf79924ccbc3a448cb0d branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ned-deily date: 2022-02-21T14:44:49-05:00 summary: Update copyright year to 2022. (GH-30335) (GH-31477) Automerge-Triggered-By: GH:benjaminp (cherry picked from commit ba00f0d93a4aea85ae8089f139856a7c450584d7) Co-authored-by: Benjamin Peterson files: M Doc/copyright.rst M Doc/license.rst M LICENSE M Mac/IDLE/IDLE.app/Contents/Info.plist M Mac/PythonLauncher/Info.plist.in M Mac/Resources/app/Info.plist.in M PC/python_ver_rc.h M Python/getcopyright.c M README.rst diff --git a/Doc/copyright.rst b/Doc/copyright.rst index 4191c0bb63a2c..e64a49328b472 100644 --- a/Doc/copyright.rst +++ b/Doc/copyright.rst @@ -4,7 +4,7 @@ Copyright Python and this documentation is: -Copyright ? 2001-2021 Python Software Foundation. All rights reserved. +Copyright ? 2001-2022 Python Software Foundation. All rights reserved. Copyright ? 2000 BeOpen.com. All rights reserved. diff --git a/Doc/license.rst b/Doc/license.rst index cf80a3caedaa3..c5528834514b8 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -87,7 +87,7 @@ PSF LICENSE AGREEMENT FOR PYTHON |release| analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python |release| alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of - copyright, i.e., "Copyright ? 2001-2021 Python Software Foundation; All Rights + copyright, i.e., "Copyright ? 2001-2022 Python Software Foundation; All Rights Reserved" are retained in Python |release| alone or in any derivative version prepared by Licensee. diff --git a/LICENSE b/LICENSE index 877c067513781..ea33d9c47b41f 100644 --- a/LICENSE +++ b/LICENSE @@ -73,7 +73,7 @@ analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021 Python Software Foundation; +2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee. diff --git a/Mac/IDLE/IDLE.app/Contents/Info.plist b/Mac/IDLE/IDLE.app/Contents/Info.plist index f6b5cfe8d5451..d197c77ed4b1a 100644 --- a/Mac/IDLE/IDLE.app/Contents/Info.plist +++ b/Mac/IDLE/IDLE.app/Contents/Info.plist @@ -36,7 +36,7 @@ CFBundleExecutable IDLE CFBundleGetInfoString - %version%, ? 2001-2021 Python Software Foundation + %version%, ? 2001-2022 Python Software Foundation CFBundleIconFile IDLE.icns CFBundleIdentifier diff --git a/Mac/PythonLauncher/Info.plist.in b/Mac/PythonLauncher/Info.plist.in index 3d8bc3e4154ee..70f215d07249b 100644 --- a/Mac/PythonLauncher/Info.plist.in +++ b/Mac/PythonLauncher/Info.plist.in @@ -40,7 +40,7 @@ CFBundleExecutable Python Launcher CFBundleGetInfoString - %VERSION%, ? 2001-2021 Python Software Foundation + %VERSION%, ? 2001-2022 Python Software Foundation CFBundleIconFile PythonLauncher.icns CFBundleIdentifier diff --git a/Mac/Resources/app/Info.plist.in b/Mac/Resources/app/Info.plist.in index 2c801332332b3..84843b734e3d6 100644 --- a/Mac/Resources/app/Info.plist.in +++ b/Mac/Resources/app/Info.plist.in @@ -37,7 +37,7 @@ CFBundleInfoDictionaryVersion 6.0 CFBundleLongVersionString - %version%, (c) 2001-2021 Python Software Foundation. + %version%, (c) 2001-2022 Python Software Foundation. CFBundleName Python CFBundlePackageType diff --git a/PC/python_ver_rc.h b/PC/python_ver_rc.h index bc4ab34c71278..f0255d988cf83 100644 --- a/PC/python_ver_rc.h +++ b/PC/python_ver_rc.h @@ -4,7 +4,7 @@ #include "winver.h" #define PYTHON_COMPANY "Python Software Foundation" -#define PYTHON_COPYRIGHT "Copyright \xA9 2001-2021 Python Software Foundation. Copyright \xA9 2000 BeOpen.com. Copyright \xA9 1995-2001 CNRI. Copyright \xA9 1991-1995 SMC." +#define PYTHON_COPYRIGHT "Copyright \xA9 2001-2022 Python Software Foundation. Copyright \xA9 2000 BeOpen.com. Copyright \xA9 1995-2001 CNRI. Copyright \xA9 1991-1995 SMC." #define MS_WINDOWS #include "modsupport.h" diff --git a/Python/getcopyright.c b/Python/getcopyright.c index 7fdeb314d5261..88d1d0536253a 100644 --- a/Python/getcopyright.c +++ b/Python/getcopyright.c @@ -4,7 +4,7 @@ static const char cprt[] = "\ -Copyright (c) 2001-2021 Python Software Foundation.\n\ +Copyright (c) 2001-2022 Python Software Foundation.\n\ All Rights Reserved.\n\ \n\ Copyright (c) 2000 BeOpen.com.\n\ diff --git a/README.rst b/README.rst index 6c39a39280491..5fa20143336f9 100644 --- a/README.rst +++ b/README.rst @@ -13,7 +13,7 @@ This is Python version 3.7.12+ :alt: CPython code coverage on Codecov :target: https://codecov.io/gh/python/cpython/branch/3.7 -Copyright (c) 2001-2021 Python Software Foundation. All rights reserved. +Copyright (c) 2001-2022 Python Software Foundation. All rights reserved. See the end of this file for further copyright and license information. @@ -242,7 +242,7 @@ See :pep:`537` for Python 3.7 release details. Copyright and License Information --------------------------------- -Copyright (c) 2001-2021 Python Software Foundation. All rights reserved. +Copyright (c) 2001-2022 Python Software Foundation. All rights reserved. Copyright (c) 2000 BeOpen.com. All rights reserved. From webhook-mailer at python.org Mon Feb 21 15:59:11 2022 From: webhook-mailer at python.org (asvetlov) Date: Mon, 21 Feb 2022 20:59:11 -0000 Subject: [Python-checkins] bpo-45390: Propagate CancelledError's message from cancelled task to its awaiter (GH-31383) Message-ID: https://github.com/python/cpython/commit/4140bcb1cd76dec5cf8d398f4d0e86c438c987d0 commit: 4140bcb1cd76dec5cf8d398f4d0e86c438c987d0 branch: main author: Andrew Svetlov committer: asvetlov date: 2022-02-21T22:59:04+02:00 summary: bpo-45390: Propagate CancelledError's message from cancelled task to its awaiter (GH-31383) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2022-02-17-11-00-16.bpo-45390.sVhG6M.rst M Doc/library/asyncio-task.rst M Lib/asyncio/futures.py M Lib/test/test_asyncio/test_tasks.py M Modules/_asynciomodule.c diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 731d1559bc09e..b30b2894277a2 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -843,6 +843,9 @@ Task Object .. versionchanged:: 3.9 Added the *msg* parameter. + .. versionchanged:: 3.11 + The ``msg`` parameter is propagated from cancelled task to its awaiter. + .. _asyncio_example_task_cancel: The following example illustrates how coroutines can intercept diff --git a/Lib/asyncio/futures.py b/Lib/asyncio/futures.py index 8e8cd87612588..5a92f1769fa9f 100644 --- a/Lib/asyncio/futures.py +++ b/Lib/asyncio/futures.py @@ -132,6 +132,11 @@ def _make_cancelled_error(self): This should only be called once when handling a cancellation since it erases the saved context exception value. """ + if self._cancelled_exc is not None: + exc = self._cancelled_exc + self._cancelled_exc = None + return exc + if self._cancel_message is None: exc = exceptions.CancelledError() else: diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index 40b33deb17ee0..b30f8f56dfa72 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -124,9 +124,11 @@ async def coro(): t.cancel('my message') self.assertEqual(t._cancel_message, 'my message') - with self.assertRaises(asyncio.CancelledError): + with self.assertRaises(asyncio.CancelledError) as cm: self.loop.run_until_complete(t) + self.assertEqual('my message', cm.exception.args[0]) + def test_task_cancel_message_setter(self): async def coro(): pass @@ -135,9 +137,11 @@ async def coro(): t._cancel_message = 'my new message' self.assertEqual(t._cancel_message, 'my new message') - with self.assertRaises(asyncio.CancelledError): + with self.assertRaises(asyncio.CancelledError) as cm: self.loop.run_until_complete(t) + self.assertEqual('my new message', cm.exception.args[0]) + def test_task_del_collect(self): class Evil: def __del__(self): @@ -590,11 +594,11 @@ async def coro(): with self.assertRaises(asyncio.CancelledError) as cm: loop.run_until_complete(task) exc = cm.exception - self.assertEqual(exc.args, ()) + self.assertEqual(exc.args, expected_args) actual = get_innermost_context(exc) self.assertEqual(actual, - (asyncio.CancelledError, expected_args, 2)) + (asyncio.CancelledError, expected_args, 0)) def test_cancel_with_message_then_future_exception(self): # Test Future.exception() after calling cancel() with a message. @@ -624,11 +628,39 @@ async def coro(): with self.assertRaises(asyncio.CancelledError) as cm: loop.run_until_complete(task) exc = cm.exception - self.assertEqual(exc.args, ()) + self.assertEqual(exc.args, expected_args) actual = get_innermost_context(exc) self.assertEqual(actual, - (asyncio.CancelledError, expected_args, 2)) + (asyncio.CancelledError, expected_args, 0)) + + def test_cancellation_exception_context(self): + loop = asyncio.new_event_loop() + self.set_event_loop(loop) + fut = loop.create_future() + + async def sleep(): + fut.set_result(None) + await asyncio.sleep(10) + + async def coro(): + inner_task = self.new_task(loop, sleep()) + await fut + loop.call_soon(inner_task.cancel, 'msg') + try: + await inner_task + except asyncio.CancelledError as ex: + raise ValueError("cancelled") from ex + + task = self.new_task(loop, coro()) + with self.assertRaises(ValueError) as cm: + loop.run_until_complete(task) + exc = cm.exception + self.assertEqual(exc.args, ('cancelled',)) + + actual = get_innermost_context(exc) + self.assertEqual(actual, + (asyncio.CancelledError, ('msg',), 1)) def test_cancel_with_message_before_starting_task(self): loop = asyncio.new_event_loop() @@ -648,11 +680,11 @@ async def coro(): with self.assertRaises(asyncio.CancelledError) as cm: loop.run_until_complete(task) exc = cm.exception - self.assertEqual(exc.args, ()) + self.assertEqual(exc.args, ('my message',)) actual = get_innermost_context(exc) self.assertEqual(actual, - (asyncio.CancelledError, ('my message',), 2)) + (asyncio.CancelledError, ('my message',), 0)) def test_cancel_yield(self): async def task(): @@ -2296,15 +2328,17 @@ async def main(): try: loop.run_until_complete(main()) except asyncio.CancelledError as exc: - self.assertEqual(exc.args, ()) - exc_type, exc_args, depth = get_innermost_context(exc) - self.assertEqual((exc_type, exc_args), - (asyncio.CancelledError, expected_args)) - # The exact traceback seems to vary in CI. - self.assertIn(depth, (2, 3)) + self.assertEqual(exc.args, expected_args) + actual = get_innermost_context(exc) + self.assertEqual( + actual, + (asyncio.CancelledError, expected_args, 0), + ) else: - self.fail('gather did not propagate the cancellation ' - 'request') + self.fail( + 'gather() does not propagate CancelledError ' + 'raised by inner task to the gather() caller.' + ) def test_exception_traceback(self): # See http://bugs.python.org/issue28843 diff --git a/Misc/NEWS.d/next/Library/2022-02-17-11-00-16.bpo-45390.sVhG6M.rst b/Misc/NEWS.d/next/Library/2022-02-17-11-00-16.bpo-45390.sVhG6M.rst new file mode 100644 index 0000000000000..5f1eafa5f25dd --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-17-11-00-16.bpo-45390.sVhG6M.rst @@ -0,0 +1,2 @@ +Propagate :exc:`asyncio.CancelledError` message from inner task to outer +awaiter. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 6725e2eba79bc..0d6b21d3b0c39 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -77,7 +77,7 @@ typedef enum { int prefix##_blocking; \ PyObject *dict; \ PyObject *prefix##_weakreflist; \ - _PyErr_StackItem prefix##_cancelled_exc_state; + PyObject *prefix##_cancelled_exc; typedef struct { FutureObj_HEAD(fut) @@ -496,7 +496,7 @@ future_init(FutureObj *fut, PyObject *loop) Py_CLEAR(fut->fut_exception); Py_CLEAR(fut->fut_source_tb); Py_CLEAR(fut->fut_cancel_msg); - _PyErr_ClearExcState(&fut->fut_cancelled_exc_state); + Py_CLEAR(fut->fut_cancelled_exc); fut->fut_state = STATE_PENDING; fut->fut_log_tb = 0; @@ -612,25 +612,32 @@ future_set_exception(FutureObj *fut, PyObject *exc) } static PyObject * -create_cancelled_error(PyObject *msg) +create_cancelled_error(FutureObj *fut) { PyObject *exc; + if (fut->fut_cancelled_exc != NULL) { + /* transfer ownership */ + exc = fut->fut_cancelled_exc; + fut->fut_cancelled_exc = NULL; + return exc; + } + PyObject *msg = fut->fut_cancel_msg; if (msg == NULL || msg == Py_None) { exc = PyObject_CallNoArgs(asyncio_CancelledError); } else { exc = PyObject_CallOneArg(asyncio_CancelledError, msg); } + PyException_SetContext(exc, fut->fut_cancelled_exc); + Py_CLEAR(fut->fut_cancelled_exc); return exc; } static void future_set_cancelled_error(FutureObj *fut) { - PyObject *exc = create_cancelled_error(fut->fut_cancel_msg); + PyObject *exc = create_cancelled_error(fut); PyErr_SetObject(asyncio_CancelledError, exc); Py_DECREF(exc); - - _PyErr_ChainStackItem(&fut->fut_cancelled_exc_state); } static int @@ -793,7 +800,7 @@ FutureObj_clear(FutureObj *fut) Py_CLEAR(fut->fut_exception); Py_CLEAR(fut->fut_source_tb); Py_CLEAR(fut->fut_cancel_msg); - _PyErr_ClearExcState(&fut->fut_cancelled_exc_state); + Py_CLEAR(fut->fut_cancelled_exc); Py_CLEAR(fut->dict); return 0; } @@ -809,11 +816,8 @@ FutureObj_traverse(FutureObj *fut, visitproc visit, void *arg) Py_VISIT(fut->fut_exception); Py_VISIT(fut->fut_source_tb); Py_VISIT(fut->fut_cancel_msg); + Py_VISIT(fut->fut_cancelled_exc); Py_VISIT(fut->dict); - - _PyErr_StackItem *exc_state = &fut->fut_cancelled_exc_state; - Py_VISIT(exc_state->exc_value); - return 0; } @@ -1369,15 +1373,7 @@ static PyObject * _asyncio_Future__make_cancelled_error_impl(FutureObj *self) /*[clinic end generated code: output=a5df276f6c1213de input=ac6effe4ba795ecc]*/ { - PyObject *exc = create_cancelled_error(self->fut_cancel_msg); - _PyErr_StackItem *exc_state = &self->fut_cancelled_exc_state; - - if (exc_state->exc_value) { - PyException_SetContext(exc, Py_NewRef(exc_state->exc_value)); - _PyErr_ClearExcState(exc_state); - } - - return exc; + return create_cancelled_error(self); } /*[clinic input] @@ -2677,7 +2673,7 @@ task_step_impl(TaskObj *task, PyObject *exc) if (!exc) { /* exc was not a CancelledError */ - exc = create_cancelled_error(task->task_cancel_msg); + exc = create_cancelled_error((FutureObj*)task); if (!exc) { goto fail; @@ -2751,8 +2747,8 @@ task_step_impl(TaskObj *task, PyObject *exc) Py_XDECREF(et); FutureObj *fut = (FutureObj*)task; - _PyErr_StackItem *exc_state = &fut->fut_cancelled_exc_state; - exc_state->exc_value = ev; + /* transfer ownership */ + fut->fut_cancelled_exc = ev; return future_cancel(fut, NULL); } From webhook-mailer at python.org Mon Feb 21 21:59:36 2022 From: webhook-mailer at python.org (methane) Date: Tue, 22 Feb 2022 02:59:36 -0000 Subject: [Python-checkins] bpo-46606: Reduce stack usage of getgroups and setgroups (GH-31073) Message-ID: https://github.com/python/cpython/commit/74127b89a8224d021fc76f679422b76510844ff9 commit: 74127b89a8224d021fc76f679422b76510844ff9 branch: main author: Inada Naoki committer: methane date: 2022-02-22T11:59:27+09:00 summary: bpo-46606: Reduce stack usage of getgroups and setgroups (GH-31073) NGROUPS_MAX was 32 before Linux 2.6.4 but 65536 since Linux 2.6.4. files: M Modules/posixmodule.c diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 6ef3610fce7c5..5e942c43dd6c7 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -7623,21 +7623,15 @@ static PyObject * os_getgroups_impl(PyObject *module) /*[clinic end generated code: output=42b0c17758561b56 input=d3f109412e6a155c]*/ { - PyObject *result = NULL; - gid_t grouplist[MAX_GROUPS]; - /* On MacOSX getgroups(2) can return more than MAX_GROUPS results * This is a helper variable to store the intermediate result when * that happens. * - * To keep the code readable the OSX behaviour is unconditional, - * according to the POSIX spec this should be safe on all unix-y - * systems. + * See bpo-7900. */ - gid_t* alt_grouplist = grouplist; + gid_t *grouplist = NULL; int n; -#ifdef __APPLE__ /* Issue #17557: As of OS X 10.8, getgroups(2) no longer raises EINVAL if * there are more groups than can fit in grouplist. Therefore, on OS X * always first call getgroups with length 0 to get the actual number @@ -7646,56 +7640,25 @@ os_getgroups_impl(PyObject *module) n = getgroups(0, NULL); if (n < 0) { return posix_error(); - } else if (n <= MAX_GROUPS) { - /* groups will fit in existing array */ - alt_grouplist = grouplist; } else { - alt_grouplist = PyMem_New(gid_t, n); - if (alt_grouplist == NULL) { + n++; // Avoid malloc(0) + grouplist = PyMem_New(gid_t, n+1); + if (grouplist == NULL) { return PyErr_NoMemory(); } } - n = getgroups(n, alt_grouplist); + n = getgroups(n, grouplist); if (n == -1) { - if (alt_grouplist != grouplist) { - PyMem_Free(alt_grouplist); - } + PyMem_Free(grouplist); return posix_error(); } -#else - n = getgroups(MAX_GROUPS, grouplist); - if (n < 0) { - if (errno == EINVAL) { - n = getgroups(0, NULL); - if (n == -1) { - return posix_error(); - } - if (n == 0) { - /* Avoid malloc(0) */ - alt_grouplist = grouplist; - } else { - alt_grouplist = PyMem_New(gid_t, n); - if (alt_grouplist == NULL) { - return PyErr_NoMemory(); - } - n = getgroups(n, alt_grouplist); - if (n == -1) { - PyMem_Free(alt_grouplist); - return posix_error(); - } - } - } else { - return posix_error(); - } - } -#endif - result = PyList_New(n); + PyObject *result = PyList_New(n); if (result != NULL) { int i; for (i = 0; i < n; ++i) { - PyObject *o = _PyLong_FromGid(alt_grouplist[i]); + PyObject *o = _PyLong_FromGid(grouplist[i]); if (o == NULL) { Py_DECREF(result); result = NULL; @@ -7705,9 +7668,7 @@ os_getgroups_impl(PyObject *module) } } - if (alt_grouplist != grouplist) { - PyMem_Free(alt_grouplist); - } + PyMem_Free(grouplist); return result; } @@ -8212,14 +8173,11 @@ static PyObject * os_setgroups(PyObject *module, PyObject *groups) /*[clinic end generated code: output=3fcb32aad58c5ecd input=fa742ca3daf85a7e]*/ { - Py_ssize_t i, len; - gid_t grouplist[MAX_GROUPS]; - if (!PySequence_Check(groups)) { PyErr_SetString(PyExc_TypeError, "setgroups argument must be a sequence"); return NULL; } - len = PySequence_Size(groups); + Py_ssize_t len = PySequence_Size(groups); if (len < 0) { return NULL; } @@ -8227,27 +8185,36 @@ os_setgroups(PyObject *module, PyObject *groups) PyErr_SetString(PyExc_ValueError, "too many groups"); return NULL; } - for(i = 0; i < len; i++) { + + gid_t *grouplist = PyMem_New(gid_t, len+1); // Avoid malloc(0) + for (Py_ssize_t i = 0; i < len; i++) { PyObject *elem; elem = PySequence_GetItem(groups, i); - if (!elem) + if (!elem) { + PyMem_Free(grouplist); return NULL; + } if (!PyLong_Check(elem)) { PyErr_SetString(PyExc_TypeError, "groups must be integers"); Py_DECREF(elem); + PyMem_Free(grouplist); return NULL; } else { if (!_Py_Gid_Converter(elem, &grouplist[i])) { Py_DECREF(elem); + PyMem_Free(grouplist); return NULL; } } Py_DECREF(elem); } - if (setgroups(len, grouplist) < 0) + if (setgroups(len, grouplist) < 0) { + PyMem_Free(grouplist); return posix_error(); + } + PyMem_Free(grouplist); Py_RETURN_NONE; } #endif /* HAVE_SETGROUPS */ From webhook-mailer at python.org Tue Feb 22 02:52:02 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 22 Feb 2022 07:52:02 -0000 Subject: [Python-checkins] bpo-46820: Fix a SyntaxError in a numeric literal followed by "not in" (GH-31479) Message-ID: https://github.com/python/cpython/commit/090e5c4b946b28f50fce445916c5d3ec45c8f45f commit: 090e5c4b946b28f50fce445916c5d3ec45c8f45f branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2022-02-22T09:51:51+02:00 summary: bpo-46820: Fix a SyntaxError in a numeric literal followed by "not in" (GH-31479) Fix parsing a numeric literal immediately (without spaces) followed by "not in" keywords, like in "1not in x". Now the parser only emits a warning, not a syntax error. files: A Misc/NEWS.d/next/Core and Builtins/2022-02-21-21-55-23.bpo-46820.4RfUZh.rst M Lib/test/test_grammar.py M Parser/tokenizer.c diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py index a2460add4c9ee..3f5a91a851d8e 100644 --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -251,6 +251,15 @@ def check_error(test): check("1e3in x") check("1jin x") + check("0xfnot in x") + check("0o7not in x") + check("0b1not in x") + check("9not in x") + check("0not in x") + check("1.not in x") + check("1e3not in x") + check("1jnot in x") + with warnings.catch_warnings(): warnings.simplefilter('ignore', SyntaxWarning) check("0xfis x") diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-21-21-55-23.bpo-46820.4RfUZh.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-21-21-55-23.bpo-46820.4RfUZh.rst new file mode 100644 index 0000000000000..117a84d0cbfaf --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-21-21-55-23.bpo-46820.4RfUZh.rst @@ -0,0 +1,3 @@ +Fix parsing a numeric literal immediately (without spaces) followed by "not +in" keywords, like in ``1not in x``. Now the parser only emits a warning, +not a syntax error. diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index b14506aaca352..6698d35ea9427 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1253,6 +1253,9 @@ verify_end_of_number(struct tok_state *tok, int c, const char *kind) else if (c == 'o') { r = lookahead(tok, "r"); } + else if (c == 'n') { + r = lookahead(tok, "ot"); + } if (r) { tok_backup(tok, c); if (parser_warn(tok, "invalid %s literal", kind)) { From webhook-mailer at python.org Tue Feb 22 05:00:55 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Tue, 22 Feb 2022 10:00:55 -0000 Subject: [Python-checkins] bpo-46820: Fix a SyntaxError in a numeric literal followed by "not in" (GH-31479) (GH-31493) Message-ID: https://github.com/python/cpython/commit/f20ac2ed076df63a77f65ff2660148af9f1a9b3c commit: f20ac2ed076df63a77f65ff2660148af9f1a9b3c branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: serhiy-storchaka date: 2022-02-22T12:00:50+02:00 summary: bpo-46820: Fix a SyntaxError in a numeric literal followed by "not in" (GH-31479) (GH-31493) Fix parsing a numeric literal immediately (without spaces) followed by "not in" keywords, like in "1not in x". Now the parser only emits a warning, not a syntax error. (cherry picked from commit 090e5c4b946b28f50fce445916c5d3ec45c8f45f) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Core and Builtins/2022-02-21-21-55-23.bpo-46820.4RfUZh.rst M Lib/test/test_grammar.py M Parser/tokenizer.c diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py index b6c457465609a..9d473770f3b20 100644 --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -251,6 +251,15 @@ def check_error(test): check("1e3in x") check("1jin x") + check("0xfnot in x") + check("0o7not in x") + check("0b1not in x") + check("9not in x") + check("0not in x") + check("1.not in x") + check("1e3not in x") + check("1jnot in x") + with warnings.catch_warnings(): warnings.simplefilter('ignore', SyntaxWarning) check("0xfis x") diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-21-21-55-23.bpo-46820.4RfUZh.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-21-21-55-23.bpo-46820.4RfUZh.rst new file mode 100644 index 0000000000000..117a84d0cbfaf --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-21-21-55-23.bpo-46820.4RfUZh.rst @@ -0,0 +1,3 @@ +Fix parsing a numeric literal immediately (without spaces) followed by "not +in" keywords, like in ``1not in x``. Now the parser only emits a warning, +not a syntax error. diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index eb15ef7a5d5f7..267ccec33fd77 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1251,6 +1251,9 @@ verify_end_of_number(struct tok_state *tok, int c, const char *kind) else if (c == 'o') { r = lookahead(tok, "r"); } + else if (c == 'n') { + r = lookahead(tok, "ot"); + } if (r) { tok_backup(tok, c); if (parser_warn(tok, "invalid %s literal", kind)) { From webhook-mailer at python.org Tue Feb 22 06:03:21 2022 From: webhook-mailer at python.org (markshannon) Date: Tue, 22 Feb 2022 11:03:21 -0000 Subject: [Python-checkins] dict: Add dk_log2_index_bytes (GH-31439) Message-ID: https://github.com/python/cpython/commit/1e344684d8d42206858c4eca8ec7950e644f4220 commit: 1e344684d8d42206858c4eca8ec7950e644f4220 branch: main author: Inada Naoki committer: markshannon date: 2022-02-22T11:03:15Z summary: dict: Add dk_log2_index_bytes (GH-31439) files: M Include/internal/pycore_dict.h M Objects/dictobject.c diff --git a/Include/internal/pycore_dict.h b/Include/internal/pycore_dict.h index 64d70d187df4e..9cd652b2fb6be 100644 --- a/Include/internal/pycore_dict.h +++ b/Include/internal/pycore_dict.h @@ -68,6 +68,9 @@ struct _dictkeysobject { /* Size of the hash table (dk_indices). It must be a power of 2. */ uint8_t dk_log2_size; + /* Size of the hash table (dk_indices) by bytes. */ + uint8_t dk_log2_index_bytes; + /* Kind of keys */ uint8_t dk_kind; @@ -129,7 +132,7 @@ struct _dictvalues { 2 : sizeof(int32_t)) #endif #define DK_ENTRIES(dk) \ - ((PyDictKeyEntry*)(&((int8_t*)((dk)->dk_indices))[DK_SIZE(dk) * DK_IXSIZE(dk)])) + ((PyDictKeyEntry*)(&((int8_t*)((dk)->dk_indices))[(size_t)1 << (dk)->dk_log2_index_bytes])) extern uint64_t _pydict_global_version; diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 63e3eda49881a..fca879f4e3cdd 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -16,19 +16,20 @@ As of Python 3.6, this is compact and ordered. Basic idea is described here: layout: -+---------------+ -| dk_refcnt | -| dk_log2_size | -| dk_kind | -| dk_usable | -| dk_nentries | -+---------------+ -| dk_indices | -| | -+---------------+ -| dk_entries | -| | -+---------------+ ++---------------------+ +| dk_refcnt | +| dk_log2_size | +| dk_log2_index_bytes | +| dk_kind | +| dk_usable | +| dk_nentries | ++---------------------+ +| dk_indices[] | +| | ++---------------------+ +| dk_entries[] | +| | ++---------------------+ dk_indices is actual hashtable. It holds index in entries, or DKIX_EMPTY(-1) or DKIX_DUMMY(-2). @@ -444,6 +445,7 @@ estimate_log2_keysize(Py_ssize_t n) static PyDictKeysObject empty_keys_struct = { 1, /* dk_refcnt */ 0, /* dk_log2_size */ + 0, /* dk_log2_index_bytes */ DICT_KEYS_SPLIT, /* dk_kind */ 1, /* dk_version */ 0, /* dk_usable (immutable) */ @@ -562,24 +564,25 @@ static PyDictKeysObject* new_keys_object(uint8_t log2_size) { PyDictKeysObject *dk; - Py_ssize_t es, usable; + Py_ssize_t usable; + int log2_bytes; assert(log2_size >= PyDict_LOG_MINSIZE); usable = USABLE_FRACTION(1< 4 - else if (log2_size <= 31) { - es = 4; + else if (log2_size >= 32) { + log2_bytes = log2_size + 3; } #endif else { - es = sizeof(Py_ssize_t); + log2_bytes = log2_size + 2; } #if PyDict_MAXFREELIST > 0 @@ -595,7 +598,7 @@ new_keys_object(uint8_t log2_size) #endif { dk = PyObject_Malloc(sizeof(PyDictKeysObject) - + (es<dk_refcnt = 1; dk->dk_log2_size = log2_size; + dk->dk_log2_index_bytes = log2_bytes; dk->dk_kind = DICT_KEYS_UNICODE; dk->dk_nentries = 0; dk->dk_usable = usable; dk->dk_version = 0; - memset(&dk->dk_indices[0], 0xff, es<dk_indices[0], 0xff, ((size_t)1 << log2_bytes)); memset(DK_ENTRIES(dk), 0, sizeof(PyDictKeyEntry) * usable); return dk; } From webhook-mailer at python.org Tue Feb 22 09:57:22 2022 From: webhook-mailer at python.org (markshannon) Date: Tue, 22 Feb 2022 14:57:22 -0000 Subject: [Python-checkins] Move call specializations from CALL to PRECALL. (GH-31496) Message-ID: https://github.com/python/cpython/commit/9058a35558422810061989f41571fdfea7ea8cbe commit: 9058a35558422810061989f41571fdfea7ea8cbe branch: main author: Mark Shannon committer: markshannon date: 2022-02-22T14:57:01Z summary: Move call specializations from CALL to PRECALL. (GH-31496) files: M Include/internal/pycore_code.h M Include/opcode.h M Lib/opcode.py M Python/ceval.c M Python/opcode_targets.h M Python/specialize.c diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index ead9541c29843..d7a514d8dc988 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -271,7 +271,9 @@ int _Py_Specialize_LoadGlobal(PyObject *globals, PyObject *builtins, _Py_CODEUNI int _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); int _Py_Specialize_BinarySubscr(PyObject *sub, PyObject *container, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); int _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *instr); -int _Py_Specialize_CallNoKw(PyObject *callable, _Py_CODEUNIT *instr, int nargs, +int _Py_Specialize_Call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, + PyObject *kwnames, SpecializedCacheEntry *cache); +int _Py_Specialize_Precall(PyObject *callable, _Py_CODEUNIT *instr, int nargs, PyObject *kwnames, SpecializedCacheEntry *cache, PyObject *builtins); void _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); diff --git a/Include/opcode.h b/Include/opcode.h index df93a93fbbd98..7820af64829b6 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -135,49 +135,52 @@ extern "C" { #define STORE_SUBSCR_LIST_INT 28 #define STORE_SUBSCR_DICT 29 #define CALL_ADAPTIVE 34 -#define CALL_BUILTIN_CLASS 36 -#define CALL_NO_KW_BUILTIN_O 37 -#define CALL_NO_KW_BUILTIN_FAST 38 -#define CALL_BUILTIN_FAST_WITH_KEYWORDS 39 -#define CALL_NO_KW_LEN 40 -#define CALL_NO_KW_ISINSTANCE 41 -#define CALL_PY_EXACT_ARGS 42 -#define CALL_PY_WITH_DEFAULTS 43 -#define CALL_NO_KW_LIST_APPEND 44 -#define CALL_NO_KW_METHOD_DESCRIPTOR_O 45 -#define CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 46 -#define CALL_NO_KW_STR_1 47 -#define CALL_NO_KW_TUPLE_1 48 -#define CALL_NO_KW_TYPE_1 55 -#define CALL_NO_KW_METHOD_DESCRIPTOR_FAST 56 -#define JUMP_ABSOLUTE_QUICK 57 -#define LOAD_ATTR_ADAPTIVE 58 -#define LOAD_ATTR_INSTANCE_VALUE 59 -#define LOAD_ATTR_WITH_HINT 62 -#define LOAD_ATTR_SLOT 63 -#define LOAD_ATTR_MODULE 64 -#define LOAD_GLOBAL_ADAPTIVE 65 -#define LOAD_GLOBAL_MODULE 66 -#define LOAD_GLOBAL_BUILTIN 67 -#define LOAD_METHOD_ADAPTIVE 72 -#define LOAD_METHOD_CACHED 76 -#define LOAD_METHOD_CLASS 77 -#define LOAD_METHOD_MODULE 78 -#define LOAD_METHOD_NO_DICT 79 -#define RESUME_QUICK 80 -#define STORE_ATTR_ADAPTIVE 81 -#define STORE_ATTR_INSTANCE_VALUE 131 -#define STORE_ATTR_SLOT 140 -#define STORE_ATTR_WITH_HINT 141 -#define UNPACK_SEQUENCE_ADAPTIVE 143 -#define UNPACK_SEQUENCE_LIST 150 -#define UNPACK_SEQUENCE_TUPLE 153 -#define UNPACK_SEQUENCE_TWO_TUPLE 154 -#define LOAD_FAST__LOAD_FAST 158 -#define STORE_FAST__LOAD_FAST 159 -#define LOAD_FAST__LOAD_CONST 161 -#define LOAD_CONST__LOAD_FAST 167 -#define STORE_FAST__STORE_FAST 168 +#define CALL_PY_EXACT_ARGS 36 +#define CALL_PY_WITH_DEFAULTS 37 +#define JUMP_ABSOLUTE_QUICK 38 +#define LOAD_ATTR_ADAPTIVE 39 +#define LOAD_ATTR_INSTANCE_VALUE 40 +#define LOAD_ATTR_WITH_HINT 41 +#define LOAD_ATTR_SLOT 42 +#define LOAD_ATTR_MODULE 43 +#define LOAD_GLOBAL_ADAPTIVE 44 +#define LOAD_GLOBAL_MODULE 45 +#define LOAD_GLOBAL_BUILTIN 46 +#define LOAD_METHOD_ADAPTIVE 47 +#define LOAD_METHOD_CACHED 48 +#define LOAD_METHOD_CLASS 55 +#define LOAD_METHOD_MODULE 56 +#define LOAD_METHOD_NO_DICT 57 +#define PRECALL_ADAPTIVE 58 +#define PRECALL_BUILTIN_CLASS 59 +#define PRECALL_NO_KW_BUILTIN_O 62 +#define PRECALL_NO_KW_BUILTIN_FAST 63 +#define PRECALL_BUILTIN_FAST_WITH_KEYWORDS 64 +#define PRECALL_NO_KW_LEN 65 +#define PRECALL_NO_KW_ISINSTANCE 66 +#define PRECALL_NO_KW_LIST_APPEND 67 +#define PRECALL_NO_KW_METHOD_DESCRIPTOR_O 72 +#define PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 76 +#define PRECALL_NO_KW_STR_1 77 +#define PRECALL_NO_KW_TUPLE_1 78 +#define PRECALL_NO_KW_TYPE_1 79 +#define PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST 80 +#define PRECALL_BOUND_METHOD 81 +#define PRECALL_PYFUNC 131 +#define RESUME_QUICK 140 +#define STORE_ATTR_ADAPTIVE 141 +#define STORE_ATTR_INSTANCE_VALUE 143 +#define STORE_ATTR_SLOT 150 +#define STORE_ATTR_WITH_HINT 153 +#define UNPACK_SEQUENCE_ADAPTIVE 154 +#define UNPACK_SEQUENCE_LIST 158 +#define UNPACK_SEQUENCE_TUPLE 159 +#define UNPACK_SEQUENCE_TWO_TUPLE 161 +#define LOAD_FAST__LOAD_FAST 167 +#define STORE_FAST__LOAD_FAST 168 +#define LOAD_FAST__LOAD_CONST 169 +#define LOAD_CONST__LOAD_FAST 170 +#define STORE_FAST__STORE_FAST 173 #define DO_TRACING 255 #ifdef NEED_OPCODE_JUMP_TABLES static uint32_t _PyOpcode_RelativeJump[8] = { diff --git a/Lib/opcode.py b/Lib/opcode.py index 95792459c377a..0c859c16b718c 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -248,21 +248,8 @@ def jabs_op(name, op): "STORE_SUBSCR_LIST_INT", "STORE_SUBSCR_DICT", "CALL_ADAPTIVE", - "CALL_BUILTIN_CLASS", - "CALL_NO_KW_BUILTIN_O", - "CALL_NO_KW_BUILTIN_FAST", - "CALL_BUILTIN_FAST_WITH_KEYWORDS", - "CALL_NO_KW_LEN", - "CALL_NO_KW_ISINSTANCE", "CALL_PY_EXACT_ARGS", "CALL_PY_WITH_DEFAULTS", - "CALL_NO_KW_LIST_APPEND", - "CALL_NO_KW_METHOD_DESCRIPTOR_O", - "CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS", - "CALL_NO_KW_STR_1", - "CALL_NO_KW_TUPLE_1", - "CALL_NO_KW_TYPE_1", - "CALL_NO_KW_METHOD_DESCRIPTOR_FAST", "JUMP_ABSOLUTE_QUICK", "LOAD_ATTR_ADAPTIVE", "LOAD_ATTR_INSTANCE_VALUE", @@ -277,6 +264,22 @@ def jabs_op(name, op): "LOAD_METHOD_CLASS", "LOAD_METHOD_MODULE", "LOAD_METHOD_NO_DICT", + "PRECALL_ADAPTIVE", + "PRECALL_BUILTIN_CLASS", + "PRECALL_NO_KW_BUILTIN_O", + "PRECALL_NO_KW_BUILTIN_FAST", + "PRECALL_BUILTIN_FAST_WITH_KEYWORDS", + "PRECALL_NO_KW_LEN", + "PRECALL_NO_KW_ISINSTANCE", + "PRECALL_NO_KW_LIST_APPEND", + "PRECALL_NO_KW_METHOD_DESCRIPTOR_O", + "PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS", + "PRECALL_NO_KW_STR_1", + "PRECALL_NO_KW_TUPLE_1", + "PRECALL_NO_KW_TYPE_1", + "PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST", + "PRECALL_BOUND_METHOD", + "PRECALL_PYFUNC", "RESUME_QUICK", "STORE_ATTR_ADAPTIVE", "STORE_ATTR_INSTANCE_VALUE", diff --git a/Python/ceval.c b/Python/ceval.c index d3ab1da76dea4..4d7c0d015bc76 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4482,6 +4482,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } TARGET(PRECALL) { + PREDICTED(PRECALL); /* Designed to work in tamdem with LOAD_METHOD. */ /* `meth` is NULL when LOAD_METHOD thinks that it's not a method call. @@ -4529,9 +4530,36 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PEEK(oparg+1) = self; PEEK(oparg+2) = meth; Py_DECREF(function); - function = meth; } + DISPATCH(); + } + + TARGET(PRECALL_BOUND_METHOD) { + SpecializedCacheEntry *cache = GET_CACHE(); + int original_oparg = cache->adaptive.original_oparg; + int is_method = (PEEK(original_oparg + 2) != NULL); + DEOPT_IF(is_method, PRECALL); + PyObject *function = PEEK(original_oparg + 1); + DEOPT_IF(Py_TYPE(function) != &PyMethod_Type, PRECALL); + STAT_INC(PRECALL, hit); + PyObject *meth = ((PyMethodObject *)function)->im_func; + PyObject *self = ((PyMethodObject *)function)->im_self; + Py_INCREF(meth); + Py_INCREF(self); + PEEK(original_oparg+1) = self; + PEEK(original_oparg+2) = meth; + Py_DECREF(function); + DISPATCH(); + } + TARGET(PRECALL_PYFUNC) { + SpecializedCacheEntry *cache = GET_CACHE(); + int original_oparg = cache->adaptive.original_oparg; + int is_method = (PEEK(original_oparg + 2) != NULL); + int nargs = original_oparg + is_method; + PyObject *function = PEEK(nargs + 1); + DEOPT_IF(Py_TYPE(function) != &PyFunction_Type, PRECALL); + STAT_INC(PRECALL, hit); DISPATCH(); } @@ -4602,7 +4630,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_ADAPTIVE) { + TARGET(PRECALL_ADAPTIVE) { SpecializedCacheEntry *cache = GET_CACHE(); int original_oparg = cache->adaptive.original_oparg; if (cache->adaptive.counter == 0) { @@ -4610,7 +4638,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int is_meth = is_method(stack_pointer, original_oparg); int nargs = original_oparg + is_meth; PyObject *callable = PEEK(nargs + 1); - int err = _Py_Specialize_CallNoKw( + int err = _Py_Specialize_Precall( callable, next_instr, nargs, call_shape.kwnames, cache, BUILTINS()); if (err < 0) { @@ -4618,6 +4646,30 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } DISPATCH(); } + else { + STAT_INC(PRECALL, deferred); + cache->adaptive.counter--; + oparg = original_oparg; + JUMP_TO_INSTRUCTION(PRECALL); + } + } + + TARGET(CALL_ADAPTIVE) { + SpecializedCacheEntry *cache = GET_CACHE(); + int original_oparg = cache->adaptive.original_oparg; + if (cache->adaptive.counter == 0) { + next_instr--; + int is_meth = is_method(stack_pointer, original_oparg); + int nargs = original_oparg + is_meth; + PyObject *callable = PEEK(nargs + 1); + int err = _Py_Specialize_Call( + callable, next_instr, nargs, + call_shape.kwnames, cache); + if (err < 0) { + goto error; + } + DISPATCH(); + } else { STAT_INC(CALL, deferred); cache->adaptive.counter--; @@ -4698,14 +4750,16 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr goto start_frame; } - TARGET(CALL_NO_KW_TYPE_1) { + TARGET(PRECALL_NO_KW_TYPE_1) { assert(call_shape.kwnames == NULL); assert(cframe.use_tracing == 0); assert(GET_CACHE()->adaptive.original_oparg == 1); - DEOPT_IF(is_method(stack_pointer, 1), CALL); + DEOPT_IF(is_method(stack_pointer, 1), PRECALL); PyObject *obj = TOP(); PyObject *callable = SECOND(); - DEOPT_IF(callable != (PyObject *)&PyType_Type, CALL); + DEOPT_IF(callable != (PyObject *)&PyType_Type, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); PyObject *res = Py_NewRef(Py_TYPE(obj)); Py_DECREF(callable); Py_DECREF(obj); @@ -4714,16 +4768,15 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr NOTRACE_DISPATCH(); } - TARGET(CALL_NO_KW_STR_1) { + TARGET(PRECALL_NO_KW_STR_1) { assert(call_shape.kwnames == NULL); assert(cframe.use_tracing == 0); assert(GET_CACHE()->adaptive.original_oparg == 1); + DEOPT_IF(is_method(stack_pointer, 1), PRECALL); PyObject *callable = PEEK(2); - DEOPT_IF(!PyType_Check(callable), CALL); - PyTypeObject *tp = (PyTypeObject *)callable; - DEOPT_IF(is_method(stack_pointer, 1), CALL); - DEOPT_IF(tp != &PyUnicode_Type, CALL); - STAT_INC(CALL, hit); + DEOPT_IF(callable != (PyObject *)&PyUnicode_Type, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); PyObject *arg = TOP(); PyObject *res = PyObject_Str(arg); Py_DECREF(arg); @@ -4737,16 +4790,14 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_NO_KW_TUPLE_1) { + TARGET(PRECALL_NO_KW_TUPLE_1) { assert(call_shape.kwnames == NULL); assert(GET_CACHE()->adaptive.original_oparg == 1); - int is_meth = is_method(stack_pointer, 1); + DEOPT_IF(is_method(stack_pointer, 1), PRECALL); PyObject *callable = PEEK(2); - DEOPT_IF(!PyType_Check(callable), CALL); - PyTypeObject *tp = (PyTypeObject *)callable; - DEOPT_IF(is_meth, CALL); - DEOPT_IF(tp != &PyTuple_Type, CALL); - STAT_INC(CALL, hit); + DEOPT_IF(callable != (PyObject *)&PyTuple_Type, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); PyObject *arg = TOP(); PyObject *res = PySequence_Tuple(arg); Py_DECREF(arg); @@ -4760,16 +4811,17 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_BUILTIN_CLASS) { + TARGET(PRECALL_BUILTIN_CLASS) { int original_oparg = GET_CACHE()->adaptive.original_oparg; int is_meth = is_method(stack_pointer, original_oparg); int total_args = original_oparg + is_meth; int kwnames_len = KWNAMES_LEN(); PyObject *callable = PEEK(total_args + 1); - DEOPT_IF(!PyType_Check(callable), CALL); + DEOPT_IF(!PyType_Check(callable), PRECALL); PyTypeObject *tp = (PyTypeObject *)callable; - DEOPT_IF(tp->tp_vectorcall == NULL, CALL); - STAT_INC(CALL, hit); + DEOPT_IF(tp->tp_vectorcall == NULL, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); STACK_SHRINK(total_args); PyObject *res = tp->tp_vectorcall((PyObject *)tp, stack_pointer, total_args-kwnames_len, call_shape.kwnames); @@ -4788,7 +4840,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_NO_KW_BUILTIN_O) { + TARGET(PRECALL_NO_KW_BUILTIN_O) { assert(cframe.use_tracing == 0); /* Builtin METH_O functions */ assert(call_shape.kwnames == NULL); @@ -4796,12 +4848,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int original_oparg = caches->adaptive.original_oparg; int is_meth = is_method(stack_pointer, original_oparg); int total_args = original_oparg + is_meth; - DEOPT_IF(total_args != 1, CALL); + DEOPT_IF(total_args != 1, PRECALL); PyObject *callable = PEEK(total_args + 1); - DEOPT_IF(!PyCFunction_CheckExact(callable), CALL); - DEOPT_IF(PyCFunction_GET_FLAGS(callable) != METH_O, CALL); - STAT_INC(CALL, hit); - + DEOPT_IF(!PyCFunction_CheckExact(callable), PRECALL); + DEOPT_IF(PyCFunction_GET_FLAGS(callable) != METH_O, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); PyCFunction cfunc = PyCFunction_GET_FUNCTION(callable); // This is slower but CPython promises to check all non-vectorcall // function calls. @@ -4824,7 +4876,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_NO_KW_BUILTIN_FAST) { + TARGET(PRECALL_NO_KW_BUILTIN_FAST) { assert(cframe.use_tracing == 0); /* Builtin METH_FASTCALL functions, without keywords */ assert(call_shape.kwnames == NULL); @@ -4833,11 +4885,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int is_meth = is_method(stack_pointer, original_oparg); int total_args = original_oparg + is_meth; PyObject *callable = PEEK(total_args + 1); - DEOPT_IF(!PyCFunction_CheckExact(callable), CALL); + DEOPT_IF(!PyCFunction_CheckExact(callable), PRECALL); DEOPT_IF(PyCFunction_GET_FLAGS(callable) != METH_FASTCALL, - CALL); - STAT_INC(CALL, hit); - + PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); PyCFunction cfunc = PyCFunction_GET_FUNCTION(callable); STACK_SHRINK(total_args); /* res = func(self, args, nargs) */ @@ -4866,7 +4918,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_BUILTIN_FAST_WITH_KEYWORDS) { + TARGET(PRECALL_BUILTIN_FAST_WITH_KEYWORDS) { assert(cframe.use_tracing == 0); /* Builtin METH_FASTCALL | METH_KEYWORDS functions */ SpecializedCacheEntry *caches = GET_CACHE(); @@ -4874,10 +4926,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int is_meth = is_method(stack_pointer, original_oparg); int total_args = original_oparg + is_meth; PyObject *callable = PEEK(total_args + 1); - DEOPT_IF(!PyCFunction_CheckExact(callable), CALL); + DEOPT_IF(!PyCFunction_CheckExact(callable), PRECALL); DEOPT_IF(PyCFunction_GET_FLAGS(callable) != - (METH_FASTCALL | METH_KEYWORDS), CALL); - STAT_INC(CALL, hit); + (METH_FASTCALL | METH_KEYWORDS), PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); STACK_SHRINK(total_args); /* res = func(self, args, nargs, kwnames) */ _PyCFunctionFastWithKeywords cfunc = @@ -4906,7 +4959,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_NO_KW_LEN) { + TARGET(PRECALL_NO_KW_LEN) { assert(cframe.use_tracing == 0); assert(call_shape.kwnames == NULL); /* len(o) */ @@ -4914,12 +4967,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int original_oparg = caches->adaptive.original_oparg; int is_meth = is_method(stack_pointer, original_oparg); int total_args = original_oparg + is_meth; - DEOPT_IF(total_args != 1, CALL); + DEOPT_IF(total_args != 1, PRECALL); _PyObjectCache *cache1 = &caches[-1].obj; PyObject *callable = PEEK(total_args + 1); - DEOPT_IF(callable != cache1->obj, CALL); - STAT_INC(CALL, hit); - + DEOPT_IF(callable != cache1->obj, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); PyObject *arg = TOP(); Py_ssize_t len_i = PyObject_Length(arg); if (len_i < 0) { @@ -4938,7 +4991,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_NO_KW_ISINSTANCE) { + TARGET(PRECALL_NO_KW_ISINSTANCE) { assert(cframe.use_tracing == 0); assert(call_shape.kwnames == NULL); /* isinstance(o, o2) */ @@ -4947,12 +5000,12 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int is_meth = is_method(stack_pointer, original_oparg); int total_args = original_oparg + is_meth; PyObject *callable = PEEK(total_args + 1); - DEOPT_IF(total_args != 2, CALL); + DEOPT_IF(total_args != 2, PRECALL); _PyObjectCache *cache1 = &caches[-1].obj; - DEOPT_IF(callable != cache1->obj, CALL); - STAT_INC(CALL, hit); - + DEOPT_IF(callable != cache1->obj, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); PyObject *cls = POP(); PyObject *inst = TOP(); int retval = PyObject_IsInstance(inst, cls); @@ -4974,20 +5027,19 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_NO_KW_LIST_APPEND) { + TARGET(PRECALL_NO_KW_LIST_APPEND) { assert(cframe.use_tracing == 0); assert(call_shape.kwnames == NULL); + assert(GET_CACHE()->adaptive.original_oparg == 1); SpecializedCacheEntry *caches = GET_CACHE(); - int original_oparg = caches->adaptive.original_oparg; _PyObjectCache *cache1 = &caches[-1].obj; - int is_meth = is_method(stack_pointer, original_oparg); - int total_args = original_oparg + is_meth; - PyObject *callable = PEEK(total_args + 1); - DEOPT_IF(total_args != 2, CALL); - DEOPT_IF(callable != cache1->obj, CALL); + assert(cache1->obj != NULL); + PyObject *callable = PEEK(3); + DEOPT_IF(callable != cache1->obj, PRECALL); PyObject *list = SECOND(); - DEOPT_IF(!PyList_Check(list), CALL); - STAT_INC(CALL, hit); + DEOPT_IF(!PyList_Check(list), PRECALL); + STAT_INC(PRECALL, hit); + next_instr++; // Skip following call PyObject *arg = TOP(); int err = PyList_Append(list, arg); if (err) { @@ -4995,24 +5047,25 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } Py_DECREF(arg); Py_DECREF(list); - STACK_SHRINK(3-is_meth); + STACK_SHRINK(2); Py_INCREF(Py_None); SET_TOP(Py_None); Py_DECREF(callable); NOTRACE_DISPATCH(); } - TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_O) { + TARGET(PRECALL_NO_KW_METHOD_DESCRIPTOR_O) { assert(call_shape.kwnames == NULL); int original_oparg = GET_CACHE()->adaptive.original_oparg; int is_meth = is_method(stack_pointer, original_oparg); int total_args = original_oparg + is_meth; PyObject *callable = PEEK(total_args + 1); - DEOPT_IF(total_args != 2, CALL); - DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), CALL); + DEOPT_IF(total_args != 2, PRECALL); + DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), PRECALL); PyMethodDef *meth = ((PyMethodDescrObject *)callable)->d_method; - DEOPT_IF(meth->ml_flags != METH_O, CALL); - STAT_INC(CALL, hit); + DEOPT_IF(meth->ml_flags != METH_O, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); PyCFunction cfunc = meth->ml_meth; // This is slower but CPython promises to check all non-vectorcall // function calls. @@ -5026,7 +5079,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); Py_DECREF(self); Py_DECREF(arg); - STACK_SHRINK(3-is_meth); + STACK_SHRINK(original_oparg+1); SET_TOP(res); Py_DECREF(callable); if (res == NULL) { @@ -5036,17 +5089,19 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS) { + TARGET(PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS) { assert(call_shape.kwnames == NULL); int original_oparg = GET_CACHE()->adaptive.original_oparg; + assert(original_oparg == 0 || original_oparg == 1); int is_meth = is_method(stack_pointer, original_oparg); int total_args = original_oparg + is_meth; - PyObject *callable = PEEK(total_args + 1); - DEOPT_IF(total_args != 1, CALL); - DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), CALL); + DEOPT_IF(total_args != 1, PRECALL); + PyObject *callable = SECOND(); + DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), PRECALL); PyMethodDef *meth = ((PyMethodDescrObject *)callable)->d_method; - DEOPT_IF(meth->ml_flags != METH_NOARGS, CALL); - STAT_INC(CALL, hit); + DEOPT_IF(meth->ml_flags != METH_NOARGS, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); PyCFunction cfunc = meth->ml_meth; // This is slower but CPython promises to check all non-vectorcall // function calls. @@ -5058,7 +5113,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr _Py_LeaveRecursiveCall(tstate); assert((res != NULL) ^ (_PyErr_Occurred(tstate) != NULL)); Py_DECREF(self); - STACK_SHRINK(2-is_meth); + STACK_SHRINK(original_oparg+1); SET_TOP(res); Py_DECREF(callable); if (res == NULL) { @@ -5068,17 +5123,18 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH(); } - TARGET(CALL_NO_KW_METHOD_DESCRIPTOR_FAST) { + TARGET(PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST) { assert(call_shape.kwnames == NULL); int original_oparg = GET_CACHE()->adaptive.original_oparg; int is_meth = is_method(stack_pointer, original_oparg); int total_args = original_oparg + is_meth; PyObject *callable = PEEK(total_args + 1); /* Builtin METH_FASTCALL methods, without keywords */ - DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), CALL); + DEOPT_IF(!Py_IS_TYPE(callable, &PyMethodDescr_Type), PRECALL); PyMethodDef *meth = ((PyMethodDescrObject *)callable)->d_method; - DEOPT_IF(meth->ml_flags != METH_FASTCALL, CALL); - STAT_INC(CALL, hit); + DEOPT_IF(meth->ml_flags != METH_FASTCALL, PRECALL); + next_instr++; // Skip following call + STAT_INC(PRECALL, hit); _PyCFunctionFast cfunc = (_PyCFunctionFast)(void(*)(void))meth->ml_meth; int nargs = total_args-1; STACK_SHRINK(nargs); @@ -5456,6 +5512,7 @@ MISS_WITH_CACHE(LOAD_ATTR) MISS_WITH_CACHE(STORE_ATTR) MISS_WITH_CACHE(LOAD_GLOBAL) MISS_WITH_CACHE(LOAD_METHOD) +MISS_WITH_CACHE(PRECALL) MISS_WITH_CACHE(CALL) MISS_WITH_CACHE(BINARY_OP) MISS_WITH_CACHE(COMPARE_OP) diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index f6cbec75089e5..cf5bdc772e421 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -35,52 +35,52 @@ static void *opcode_targets[256] = { &&TARGET_MATCH_KEYS, &&TARGET_CALL_ADAPTIVE, &&TARGET_PUSH_EXC_INFO, - &&TARGET_CALL_BUILTIN_CLASS, - &&TARGET_CALL_NO_KW_BUILTIN_O, - &&TARGET_CALL_NO_KW_BUILTIN_FAST, - &&TARGET_CALL_BUILTIN_FAST_WITH_KEYWORDS, - &&TARGET_CALL_NO_KW_LEN, - &&TARGET_CALL_NO_KW_ISINSTANCE, &&TARGET_CALL_PY_EXACT_ARGS, &&TARGET_CALL_PY_WITH_DEFAULTS, - &&TARGET_CALL_NO_KW_LIST_APPEND, - &&TARGET_CALL_NO_KW_METHOD_DESCRIPTOR_O, - &&TARGET_CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS, - &&TARGET_CALL_NO_KW_STR_1, - &&TARGET_CALL_NO_KW_TUPLE_1, - &&TARGET_WITH_EXCEPT_START, - &&TARGET_GET_AITER, - &&TARGET_GET_ANEXT, - &&TARGET_BEFORE_ASYNC_WITH, - &&TARGET_BEFORE_WITH, - &&TARGET_END_ASYNC_FOR, - &&TARGET_CALL_NO_KW_TYPE_1, - &&TARGET_CALL_NO_KW_METHOD_DESCRIPTOR_FAST, &&TARGET_JUMP_ABSOLUTE_QUICK, &&TARGET_LOAD_ATTR_ADAPTIVE, &&TARGET_LOAD_ATTR_INSTANCE_VALUE, - &&TARGET_STORE_SUBSCR, - &&TARGET_DELETE_SUBSCR, &&TARGET_LOAD_ATTR_WITH_HINT, &&TARGET_LOAD_ATTR_SLOT, &&TARGET_LOAD_ATTR_MODULE, &&TARGET_LOAD_GLOBAL_ADAPTIVE, &&TARGET_LOAD_GLOBAL_MODULE, &&TARGET_LOAD_GLOBAL_BUILTIN, + &&TARGET_LOAD_METHOD_ADAPTIVE, + &&TARGET_LOAD_METHOD_CACHED, + &&TARGET_WITH_EXCEPT_START, + &&TARGET_GET_AITER, + &&TARGET_GET_ANEXT, + &&TARGET_BEFORE_ASYNC_WITH, + &&TARGET_BEFORE_WITH, + &&TARGET_END_ASYNC_FOR, + &&TARGET_LOAD_METHOD_CLASS, + &&TARGET_LOAD_METHOD_MODULE, + &&TARGET_LOAD_METHOD_NO_DICT, + &&TARGET_PRECALL_ADAPTIVE, + &&TARGET_PRECALL_BUILTIN_CLASS, + &&TARGET_STORE_SUBSCR, + &&TARGET_DELETE_SUBSCR, + &&TARGET_PRECALL_NO_KW_BUILTIN_O, + &&TARGET_PRECALL_NO_KW_BUILTIN_FAST, + &&TARGET_PRECALL_BUILTIN_FAST_WITH_KEYWORDS, + &&TARGET_PRECALL_NO_KW_LEN, + &&TARGET_PRECALL_NO_KW_ISINSTANCE, + &&TARGET_PRECALL_NO_KW_LIST_APPEND, &&TARGET_GET_ITER, &&TARGET_GET_YIELD_FROM_ITER, &&TARGET_PRINT_EXPR, &&TARGET_LOAD_BUILD_CLASS, - &&TARGET_LOAD_METHOD_ADAPTIVE, + &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_O, &&TARGET_GET_AWAITABLE, &&TARGET_LOAD_ASSERTION_ERROR, &&TARGET_RETURN_GENERATOR, - &&TARGET_LOAD_METHOD_CACHED, - &&TARGET_LOAD_METHOD_CLASS, - &&TARGET_LOAD_METHOD_MODULE, - &&TARGET_LOAD_METHOD_NO_DICT, - &&TARGET_RESUME_QUICK, - &&TARGET_STORE_ATTR_ADAPTIVE, + &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS, + &&TARGET_PRECALL_NO_KW_STR_1, + &&TARGET_PRECALL_NO_KW_TUPLE_1, + &&TARGET_PRECALL_NO_KW_TYPE_1, + &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST, + &&TARGET_PRECALL_BOUND_METHOD, &&TARGET_LIST_TO_TUPLE, &&TARGET_RETURN_VALUE, &&TARGET_IMPORT_STAR, @@ -130,7 +130,7 @@ static void *opcode_targets[256] = { &&TARGET_POP_JUMP_IF_NOT_NONE, &&TARGET_POP_JUMP_IF_NONE, &&TARGET_RAISE_VARARGS, - &&TARGET_STORE_ATTR_INSTANCE_VALUE, + &&TARGET_PRECALL_PYFUNC, &&TARGET_MAKE_FUNCTION, &&TARGET_BUILD_SLICE, &&TARGET_JUMP_NO_INTERRUPT, @@ -139,40 +139,40 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_DEREF, &&TARGET_STORE_DEREF, &&TARGET_DELETE_DEREF, - &&TARGET_STORE_ATTR_SLOT, - &&TARGET_STORE_ATTR_WITH_HINT, + &&TARGET_RESUME_QUICK, + &&TARGET_STORE_ATTR_ADAPTIVE, &&TARGET_CALL_FUNCTION_EX, - &&TARGET_UNPACK_SEQUENCE_ADAPTIVE, + &&TARGET_STORE_ATTR_INSTANCE_VALUE, &&TARGET_EXTENDED_ARG, &&TARGET_LIST_APPEND, &&TARGET_SET_ADD, &&TARGET_MAP_ADD, &&TARGET_LOAD_CLASSDEREF, &&TARGET_COPY_FREE_VARS, - &&TARGET_UNPACK_SEQUENCE_LIST, + &&TARGET_STORE_ATTR_SLOT, &&TARGET_RESUME, &&TARGET_MATCH_CLASS, - &&TARGET_UNPACK_SEQUENCE_TUPLE, - &&TARGET_UNPACK_SEQUENCE_TWO_TUPLE, + &&TARGET_STORE_ATTR_WITH_HINT, + &&TARGET_UNPACK_SEQUENCE_ADAPTIVE, &&TARGET_FORMAT_VALUE, &&TARGET_BUILD_CONST_KEY_MAP, &&TARGET_BUILD_STRING, - &&TARGET_LOAD_FAST__LOAD_FAST, - &&TARGET_STORE_FAST__LOAD_FAST, + &&TARGET_UNPACK_SEQUENCE_LIST, + &&TARGET_UNPACK_SEQUENCE_TUPLE, &&TARGET_LOAD_METHOD, - &&TARGET_LOAD_FAST__LOAD_CONST, + &&TARGET_UNPACK_SEQUENCE_TWO_TUPLE, &&TARGET_LIST_EXTEND, &&TARGET_SET_UPDATE, &&TARGET_DICT_MERGE, &&TARGET_DICT_UPDATE, &&TARGET_PRECALL, + &&TARGET_LOAD_FAST__LOAD_FAST, + &&TARGET_STORE_FAST__LOAD_FAST, + &&TARGET_LOAD_FAST__LOAD_CONST, &&TARGET_LOAD_CONST__LOAD_FAST, - &&TARGET_STORE_FAST__STORE_FAST, - &&_unknown_opcode, - &&_unknown_opcode, &&TARGET_CALL, &&TARGET_KW_NAMES, - &&_unknown_opcode, + &&TARGET_STORE_FAST__STORE_FAST, &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, diff --git a/Python/specialize.c b/Python/specialize.c index 7dd12c7de2aeb..b46f701428923 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -48,6 +48,7 @@ static uint8_t adaptive_opcodes[256] = { [BINARY_SUBSCR] = BINARY_SUBSCR_ADAPTIVE, [STORE_SUBSCR] = STORE_SUBSCR_ADAPTIVE, [CALL] = CALL_ADAPTIVE, + [PRECALL] = PRECALL_ADAPTIVE, [STORE_ATTR] = STORE_ATTR_ADAPTIVE, [BINARY_OP] = BINARY_OP_ADAPTIVE, [COMPARE_OP] = COMPARE_OP_ADAPTIVE, @@ -62,6 +63,7 @@ static uint8_t cache_requirements[256] = { [BINARY_SUBSCR] = 2, /* _PyAdaptiveEntry, _PyObjectCache */ [STORE_SUBSCR] = 0, [CALL] = 2, /* _PyAdaptiveEntry and _PyObjectCache/_PyCallCache */ + [PRECALL] = 2, /* _PyAdaptiveEntry and _PyObjectCache/_PyCallCache */ [STORE_ATTR] = 2, /* _PyAdaptiveEntry and _PyAttrCache */ [BINARY_OP] = 1, // _PyAdaptiveEntry [COMPARE_OP] = 1, /* _PyAdaptiveEntry */ @@ -1454,35 +1456,36 @@ specialize_class_call( PyObject *callable, _Py_CODEUNIT *instr, int nargs, PyObject *kwnames, SpecializedCacheEntry *cache) { + assert(_Py_OPCODE(*instr) == PRECALL_ADAPTIVE); PyTypeObject *tp = _PyType_CAST(callable); if (tp->tp_new == PyBaseObject_Type.tp_new) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_PYTHON_CLASS); + SPECIALIZATION_FAIL(PRECALL, SPEC_FAIL_CALL_PYTHON_CLASS); return -1; } if (tp->tp_flags & Py_TPFLAGS_IMMUTABLETYPE) { - if (nargs == 1 && kwnames == NULL) { + if (nargs == 1 && kwnames == NULL && cache->adaptive.original_oparg == 1) { if (tp == &PyUnicode_Type) { - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_STR_1, _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_STR_1, _Py_OPARG(*instr)); return 0; } else if (tp == &PyType_Type) { - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_TYPE_1, _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_TYPE_1, _Py_OPARG(*instr)); return 0; } else if (tp == &PyTuple_Type) { - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_TUPLE_1, _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_TUPLE_1, _Py_OPARG(*instr)); return 0; } } if (tp->tp_vectorcall != NULL) { - *instr = _Py_MAKECODEUNIT(CALL_BUILTIN_CLASS, _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(PRECALL_BUILTIN_CLASS, _Py_OPARG(*instr)); return 0; } - SPECIALIZATION_FAIL(CALL, tp == &PyUnicode_Type ? + SPECIALIZATION_FAIL(PRECALL, tp == &PyUnicode_Type ? SPEC_FAIL_CALL_STR : SPEC_FAIL_CALL_CLASS_NO_VECTORCALL); return -1; } - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_CLASS_MUTABLE); + SPECIALIZATION_FAIL(PRECALL, SPEC_FAIL_CALL_CLASS_MUTABLE); return -1; } @@ -1517,8 +1520,9 @@ specialize_method_descriptor( PyMethodDescrObject *descr, _Py_CODEUNIT *instr, int nargs, PyObject *kwnames, SpecializedCacheEntry *cache) { + assert(_Py_OPCODE(*instr) == PRECALL_ADAPTIVE); if (kwnames) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_KWNAMES); + SPECIALIZATION_FAIL(PRECALL, SPEC_FAIL_CALL_KWNAMES); return -1; } if (_list_append == NULL) { @@ -1526,10 +1530,9 @@ specialize_method_descriptor( &_Py_ID(append)); } assert(_list_append != NULL); - if (nargs == 2 && descr == _list_append) { - assert(_Py_OPCODE(instr[-1]) == PRECALL); + if (nargs == 2 && descr == _list_append && cache->adaptive.original_oparg == 1) { cache[-1].obj.obj = (PyObject *)_list_append; - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_LIST_APPEND, _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_LIST_APPEND, _Py_OPARG(*instr)); return 0; } @@ -1538,29 +1541,29 @@ specialize_method_descriptor( METH_KEYWORDS | METH_METHOD)) { case METH_NOARGS: { if (nargs != 1) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_WRONG_NUMBER_ARGUMENTS); + SPECIALIZATION_FAIL(PRECALL, SPEC_FAIL_WRONG_NUMBER_ARGUMENTS); return -1; } - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS, + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS, _Py_OPARG(*instr)); return 0; } case METH_O: { if (nargs != 2) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_OUT_OF_RANGE); + SPECIALIZATION_FAIL(PRECALL, SPEC_FAIL_OUT_OF_RANGE); return -1; } - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_METHOD_DESCRIPTOR_O, + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_METHOD_DESCRIPTOR_O, _Py_OPARG(*instr)); return 0; } case METH_FASTCALL: { - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_METHOD_DESCRIPTOR_FAST, + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST, _Py_OPARG(*instr)); return 0; } } - SPECIALIZATION_FAIL(CALL, builtin_call_fail_kind(descr->d_method->ml_flags)); + SPECIALIZATION_FAIL(PRECALL, builtin_call_fail_kind(descr->d_method->ml_flags)); return -1; } @@ -1569,6 +1572,7 @@ specialize_py_call( PyFunctionObject *func, _Py_CODEUNIT *instr, int nargs, PyObject *kwnames, SpecializedCacheEntry *cache) { + assert(_Py_OPCODE(*instr) == CALL_ADAPTIVE); _PyCallCache *cache1 = &cache[-1].call; PyCodeObject *code = (PyCodeObject *)func->func_code; int kind = function_kind(code); @@ -1621,6 +1625,7 @@ static int specialize_c_call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, PyObject *kwnames, SpecializedCacheEntry *cache, PyObject *builtins) { + assert(_Py_OPCODE(*instr) == PRECALL_ADAPTIVE); _PyObjectCache *cache1 = &cache[-1].obj; if (PyCFunction_GET_FUNCTION(callable) == NULL) { return 1; @@ -1630,28 +1635,28 @@ specialize_c_call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, METH_KEYWORDS | METH_METHOD)) { case METH_O: { if (kwnames) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_KWNAMES); + SPECIALIZATION_FAIL(PRECALL, SPEC_FAIL_CALL_KWNAMES); return -1; } if (nargs != 1) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_WRONG_NUMBER_ARGUMENTS); + SPECIALIZATION_FAIL(PRECALL, SPEC_FAIL_WRONG_NUMBER_ARGUMENTS); return 1; } /* len(o) */ PyObject *builtin_len = PyDict_GetItemString(builtins, "len"); if (callable == builtin_len) { cache1->obj = builtin_len; // borrowed - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_LEN, + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_LEN, _Py_OPARG(*instr)); return 0; } - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_BUILTIN_O, + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_BUILTIN_O, _Py_OPARG(*instr)); return 0; } case METH_FASTCALL: { if (kwnames) { - SPECIALIZATION_FAIL(CALL, SPEC_FAIL_CALL_KWNAMES); + SPECIALIZATION_FAIL(PRECALL, SPEC_FAIL_CALL_KWNAMES); return -1; } if (nargs == 2) { @@ -1660,22 +1665,22 @@ specialize_c_call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, builtins, "isinstance"); if (callable == builtin_isinstance) { cache1->obj = builtin_isinstance; // borrowed - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_ISINSTANCE, + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_ISINSTANCE, _Py_OPARG(*instr)); return 0; } } - *instr = _Py_MAKECODEUNIT(CALL_NO_KW_BUILTIN_FAST, + *instr = _Py_MAKECODEUNIT(PRECALL_NO_KW_BUILTIN_FAST, _Py_OPARG(*instr)); return 0; } case METH_FASTCALL | METH_KEYWORDS: { - *instr = _Py_MAKECODEUNIT(CALL_BUILTIN_FAST_WITH_KEYWORDS, + *instr = _Py_MAKECODEUNIT(PRECALL_BUILTIN_FAST_WITH_KEYWORDS, _Py_OPARG(*instr)); return 0; } default: - SPECIALIZATION_FAIL(CALL, + SPECIALIZATION_FAIL(PRECALL, builtin_call_fail_kind(PyCFunction_GET_FLAGS(callable))); return 1; } @@ -1722,11 +1727,9 @@ call_fail_kind(PyObject *callable) } #endif -/* TODO: - - Specialize calling classes. -*/ + int -_Py_Specialize_CallNoKw( +_Py_Specialize_Precall( PyObject *callable, _Py_CODEUNIT *instr, int nargs, PyObject *kwnames, SpecializedCacheEntry *cache, PyObject *builtins) @@ -1737,7 +1740,8 @@ _Py_Specialize_CallNoKw( fail = specialize_c_call(callable, instr, nargs, kwnames, cache, builtins); } else if (PyFunction_Check(callable)) { - fail = specialize_py_call((PyFunctionObject *)callable, instr, nargs, kwnames, cache); + *instr = _Py_MAKECODEUNIT(PRECALL_PYFUNC, _Py_OPARG(*instr)); + fail = 0; } else if (PyType_Check(callable)) { fail = specialize_class_call(callable, instr, nargs, kwnames, cache); @@ -1746,6 +1750,42 @@ _Py_Specialize_CallNoKw( fail = specialize_method_descriptor( (PyMethodDescrObject *)callable, instr, nargs, kwnames, cache); } + else if (Py_TYPE(callable) == &PyMethod_Type) { + *instr = _Py_MAKECODEUNIT(PRECALL_BOUND_METHOD, _Py_OPARG(*instr)); + fail = 0; + } + else { + SPECIALIZATION_FAIL(CALL, call_fail_kind(callable)); + fail = -1; + } + if (fail) { + STAT_INC(CALL, failure); + assert(!PyErr_Occurred()); + cache_backoff(cache0); + } + else { + STAT_INC(CALL, success); + assert(!PyErr_Occurred()); + cache0->counter = initial_counter_value(); + } + return 0; +} + + +/* TODO: + - Specialize calling classes. +*/ +int +_Py_Specialize_Call( + PyObject *callable, _Py_CODEUNIT *instr, + int nargs, PyObject *kwnames, + SpecializedCacheEntry *cache) +{ + _PyAdaptiveEntry *cache0 = &cache->adaptive; + int fail; + if (PyFunction_Check(callable)) { + fail = specialize_py_call((PyFunctionObject *)callable, instr, nargs, kwnames, cache); + } else { SPECIALIZATION_FAIL(CALL, call_fail_kind(callable)); fail = -1; From webhook-mailer at python.org Tue Feb 22 10:44:49 2022 From: webhook-mailer at python.org (markshannon) Date: Tue, 22 Feb 2022 15:44:49 -0000 Subject: [Python-checkins] Remove pair-counts from specialization stats. (GH-31500) Message-ID: https://github.com/python/cpython/commit/1320d56ff73af290775d6ca5749ecd3ac5b67158 commit: 1320d56ff73af290775d6ca5749ecd3ac5b67158 branch: main author: Mark Shannon committer: markshannon date: 2022-02-22T15:44:31Z summary: Remove pair-counts from specialization stats. (GH-31500) files: M Tools/scripts/summarize_stats.py diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index 5805ba46afed6..df20959ea87b6 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -42,10 +42,13 @@ def print_specialization_stats(name, family_stats, defines): label = key[len("specialization."):] elif key == "execution_count": label = "unquickened" + elif key in ("specialization.success", "specialization.failure", "specializable"): + continue + elif key.startswith("pair"): + continue else: label = key - if key not in ("specialization.success", "specialization.failure", "specializable"): - rows.append((f"{label:>12}", f"{family_stats[key]:>12}", f"{100*family_stats[key]/total:0.1f}%")) + rows.append((f"{label:>12}", f"{family_stats[key]:>12}", f"{100*family_stats[key]/total:0.1f}%")) emit_table(("Kind", "Count", "Ratio"), rows) print_title("Specialization attempts", 4) total_attempts = 0 From webhook-mailer at python.org Tue Feb 22 11:25:15 2022 From: webhook-mailer at python.org (zooba) Date: Tue, 22 Feb 2022 16:25:15 -0000 Subject: [Python-checkins] bpo-46822: Increase timeout for test_create_server_ssl_over_ssl to match underlying timeouts (GH-31502) Message-ID: https://github.com/python/cpython/commit/77f31a91d55df2df79ac767690738081f27ad476 commit: 77f31a91d55df2df79ac767690738081f27ad476 branch: main author: Steve Dower committer: zooba date: 2022-02-22T16:25:05Z summary: bpo-46822: Increase timeout for test_create_server_ssl_over_ssl to match underlying timeouts (GH-31502) files: M Lib/test/test_asyncio/test_ssl.py diff --git a/Lib/test/test_asyncio/test_ssl.py b/Lib/test/test_asyncio/test_ssl.py index 8d1bb0360934d..67684ab09e688 100644 --- a/Lib/test/test_asyncio/test_ssl.py +++ b/Lib/test/test_asyncio/test_ssl.py @@ -1019,7 +1019,7 @@ async def run_main(): def test_create_server_ssl_over_ssl(self): CNT = 0 # number of clients that were successful TOTAL_CNT = 25 # total number of clients that test will create - TIMEOUT = 10.0 # timeout for this test + TIMEOUT = 30.0 # timeout for this test A_DATA = b'A' * 1024 * 1024 B_DATA = b'B' * 1024 * 1024 From webhook-mailer at python.org Tue Feb 22 12:18:19 2022 From: webhook-mailer at python.org (markshannon) Date: Tue, 22 Feb 2022 17:18:19 -0000 Subject: [Python-checkins] Fix reporting of specialization stats. (GH-31503) Message-ID: https://github.com/python/cpython/commit/09487c11a582be1f20d34319a82cfaf72d5c27a5 commit: 09487c11a582be1f20d34319a82cfaf72d5c27a5 branch: main author: Mark Shannon committer: markshannon date: 2022-02-22T17:18:10Z summary: Fix reporting of specialization stats. (GH-31503) files: M Python/ceval.c M Tools/scripts/summarize_stats.py diff --git a/Python/ceval.c b/Python/ceval.c index 4d7c0d015bc76..308271b8953de 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4691,6 +4691,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DEOPT_IF(func->func_version != cache1->func_version, CALL); PyCodeObject *code = (PyCodeObject *)func->func_code; DEOPT_IF(code->co_argcount != argcount, CALL); + STAT_INC(CALL, hit); InterpreterFrame *new_frame = _PyFrame_Push(tstate, func); if (new_frame == NULL) { goto error; @@ -4725,6 +4726,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DEOPT_IF(argcount > code->co_argcount, CALL); int minargs = cache1->min_args; DEOPT_IF(argcount < minargs, CALL); + STAT_INC(CALL, hit); InterpreterFrame *new_frame = _PyFrame_Push(tstate, func); if (new_frame == NULL) { goto error; diff --git a/Tools/scripts/summarize_stats.py b/Tools/scripts/summarize_stats.py index df20959ea87b6..69babfd2ddafc 100644 --- a/Tools/scripts/summarize_stats.py +++ b/Tools/scripts/summarize_stats.py @@ -38,7 +38,7 @@ def print_specialization_stats(name, family_stats, defines): for key in sorted(family_stats): if key.startswith("specialization.failure_kinds"): continue - if key.startswith("specialization."): + if key in ("specialization.hit", "specialization.miss"): label = key[len("specialization."):] elif key == "execution_count": label = "unquickened" From webhook-mailer at python.org Tue Feb 22 12:26:58 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 22 Feb 2022 17:26:58 -0000 Subject: [Python-checkins] bpo-46725: Document starred expressions in for statements (GH-31481) Message-ID: https://github.com/python/cpython/commit/7fb94fd7a88f14096a509dddd4d8a979a3912672 commit: 7fb94fd7a88f14096a509dddd4d8a979a3912672 branch: main author: Pablo Galindo Salgado committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T09:26:46-08:00 summary: bpo-46725: Document starred expressions in for statements (GH-31481) Automerge-Triggered-By: GH:pablogsal files: M Doc/reference/compound_stmts.rst M Doc/whatsnew/3.11.rst M Lib/test/test_grammar.py diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index c1ee20e5aee62..15a51e7fc7bea 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -154,17 +154,20 @@ The :keyword:`for` statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other iterable object: .. productionlist:: python-grammar - for_stmt: "for" `target_list` "in" `expression_list` ":" `suite` + for_stmt: "for" `target_list` "in" `starred_list` ":" `suite` : ["else" ":" `suite`] The expression list is evaluated once; it should yield an iterable object. An -iterator is created for the result of the ``expression_list``. The suite is -then executed once for each item provided by the iterator, in the order returned -by the iterator. Each item in turn is assigned to the target list using the -standard rules for assignments (see :ref:`assignment`), and then the suite is -executed. When the items are exhausted (which is immediately when the sequence -is empty or an iterator raises a :exc:`StopIteration` exception), the suite in -the :keyword:`!else` clause, if present, is executed, and the loop terminates. +iterator is created for the result of the ``starred_list``. The expression +list can contain starred elements (``*x, *y``) that will be unpacked in the +final iterator (as when constructing a ``tuple`` or ``list`` literal). The +suite is then executed once for each item provided by the iterator, in the +order returned by the iterator. Each item in turn is assigned to the target +list using the standard rules for assignments (see :ref:`assignment`), and then +the suite is executed. When the items are exhausted (which is immediately when +the sequence is empty or an iterator raises a :exc:`StopIteration` exception), +the suite in the :keyword:`!else` clause, if present, is executed, and the loop +terminates. .. index:: statement: break @@ -196,6 +199,8 @@ the built-in function :func:`range` returns an iterator of integers suitable to emulate the effect of Pascal's ``for i := a to b do``; e.g., ``list(range(3))`` returns the list ``[0, 1, 2]``. +.. versionchanged:: 3.11 + Starred elements are now allowed in the expression list. .. _try: .. _except: diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 98fa7a7da466d..85f12fe8b4fc7 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -160,6 +160,9 @@ traceback. (Contributed by Irit Katriel in :issue:`45607`.) Other Language Changes ====================== +* Starred expressions can be used in :ref:`for statements`. (See + :issue:`46725` for more details.) + * Asynchronous comprehensions are now allowed inside comprehensions in asynchronous functions. Outer comprehensions implicitly become asynchronous. (Contributed by Serhiy Storchaka in :issue:`33346`.) diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py index 3f5a91a851d8e..d35530e4803fa 100644 --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -1403,6 +1403,12 @@ def __getitem__(self, i): result.append(x) self.assertEqual(result, [1, 2, 3]) + result = [] + a = b = c = [1, 2, 3] + for x in *a, *b, *c: + result.append(x) + self.assertEqual(result, 3 * a) + def test_try(self): ### try_stmt: 'try' ':' suite (except_clause ':' suite)+ ['else' ':' suite] ### | 'try' ':' suite 'finally' ':' suite From webhook-mailer at python.org Tue Feb 22 12:36:19 2022 From: webhook-mailer at python.org (iritkatriel) Date: Tue, 22 Feb 2022 17:36:19 -0000 Subject: [Python-checkins] bpo-36557: Updated wording for using/windows (GH-31457) Message-ID: https://github.com/python/cpython/commit/9a0d941df4c3e1efb8b3017cd2c2de17e582fd5c commit: 9a0d941df4c3e1efb8b3017cd2c2de17e582fd5c branch: main author: slateny <46876382+slateny at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2022-02-22T17:36:13Z summary: bpo-36557: Updated wording for using/windows (GH-31457) files: M Doc/using/windows.rst diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index 041166f4f8584..26c19ddbbce78 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -129,8 +129,8 @@ suppressing the UI in order to change some of the defaults. To completely hide the installer UI and install Python silently, pass the ``/quiet`` option. To skip past the user interaction but still display progress and errors, pass the ``/passive`` option. The ``/uninstall`` -option may be passed to immediately begin removing Python - no prompt will be -displayed. +option may be passed to immediately begin removing Python - no confirmation +prompt will be displayed. All other options are passed as ``name=value``, where the value is usually ``0`` to disable a feature, ``1`` to enable a feature, or a path. The full list From webhook-mailer at python.org Tue Feb 22 12:57:11 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 22 Feb 2022 17:57:11 -0000 Subject: [Python-checkins] [3.10] bpo-36557: Updated wording for using/windows (GH-31457) (GH-31504) Message-ID: https://github.com/python/cpython/commit/d04fb9213a547def5604fbc60b0554c176d4c998 commit: d04fb9213a547def5604fbc60b0554c176d4c998 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T09:57:01-08:00 summary: [3.10] bpo-36557: Updated wording for using/windows (GH-31457) (GH-31504) (cherry picked from commit 9a0d941df4c3e1efb8b3017cd2c2de17e582fd5c) Co-authored-by: slateny <46876382+slateny at users.noreply.github.com> Automerge-Triggered-By: GH:iritkatriel files: M Doc/using/windows.rst diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index 68ee09c565e21..cad7f69806d03 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -129,8 +129,8 @@ suppressing the UI in order to change some of the defaults. To completely hide the installer UI and install Python silently, pass the ``/quiet`` option. To skip past the user interaction but still display progress and errors, pass the ``/passive`` option. The ``/uninstall`` -option may be passed to immediately begin removing Python - no prompt will be -displayed. +option may be passed to immediately begin removing Python - no confirmation +prompt will be displayed. All other options are passed as ``name=value``, where the value is usually ``0`` to disable a feature, ``1`` to enable a feature, or a path. The full list From webhook-mailer at python.org Tue Feb 22 13:03:47 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 22 Feb 2022 18:03:47 -0000 Subject: [Python-checkins] bpo-36557: Updated wording for using/windows (GH-31457) Message-ID: https://github.com/python/cpython/commit/aa9a5c4d72e083f8b4c635d79f7450dbe8319469 commit: aa9a5c4d72e083f8b4c635d79f7450dbe8319469 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T10:03:30-08:00 summary: bpo-36557: Updated wording for using/windows (GH-31457) (cherry picked from commit 9a0d941df4c3e1efb8b3017cd2c2de17e582fd5c) Co-authored-by: slateny <46876382+slateny at users.noreply.github.com> files: M Doc/using/windows.rst diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index c04a34a62f62e..be21b55fe801f 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -129,8 +129,8 @@ suppressing the UI in order to change some of the defaults. To completely hide the installer UI and install Python silently, pass the ``/quiet`` option. To skip past the user interaction but still display progress and errors, pass the ``/passive`` option. The ``/uninstall`` -option may be passed to immediately begin removing Python - no prompt will be -displayed. +option may be passed to immediately begin removing Python - no confirmation +prompt will be displayed. All other options are passed as ``name=value``, where the value is usually ``0`` to disable a feature, ``1`` to enable a feature, or a path. The full list From webhook-mailer at python.org Tue Feb 22 13:06:50 2022 From: webhook-mailer at python.org (iritkatriel) Date: Tue, 22 Feb 2022 18:06:50 -0000 Subject: [Python-checkins] bpo-20923 : [doc] Explain ConfigParser 'valid section name' and .SECTCRE (GH-31413) Message-ID: https://github.com/python/cpython/commit/bba8008f99d615a02984422a3825082bb5621f5a commit: bba8008f99d615a02984422a3825082bb5621f5a branch: main author: vidhya <96202776+Vidhyavinu at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2022-02-22T18:06:41Z summary: bpo-20923 : [doc] Explain ConfigParser 'valid section name' and .SECTCRE (GH-31413) files: M Doc/library/configparser.rst diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index 1ebda53ecda0f..323dd2affc78f 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -267,6 +267,9 @@ out. Values can also span multiple lines, as long as they are indented deeper than the first line of the value. Depending on the parser's mode, blank lines may be treated as parts of multiline values or ignored. +By default, a valid section name can be any string that does not contain '\\n' or ']'. +To change this, see :attr:`ConfigParser.SECTCRE`. + Configuration files may include comments, prefixed by specific characters (``#`` and ``;`` by default [1]_). Comments may appear on their own on an otherwise empty line, possibly indented. [1]_ From webhook-mailer at python.org Tue Feb 22 13:28:56 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 22 Feb 2022 18:28:56 -0000 Subject: [Python-checkins] [3.10] bpo-20923 : [doc] Explain ConfigParser 'valid section name' and .SECTCRE (GH-31413) (GH-31506) Message-ID: https://github.com/python/cpython/commit/a7af34d407b344de7f138c002854a3e16ecb9db5 commit: a7af34d407b344de7f138c002854a3e16ecb9db5 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T10:28:34-08:00 summary: [3.10] bpo-20923 : [doc] Explain ConfigParser 'valid section name' and .SECTCRE (GH-31413) (GH-31506) (cherry picked from commit bba8008f99d615a02984422a3825082bb5621f5a) Co-authored-by: vidhya <96202776+Vidhyavinu at users.noreply.github.com> Automerge-Triggered-By: GH:iritkatriel files: M Doc/library/configparser.rst diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index 1ebda53ecda0f..323dd2affc78f 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -267,6 +267,9 @@ out. Values can also span multiple lines, as long as they are indented deeper than the first line of the value. Depending on the parser's mode, blank lines may be treated as parts of multiline values or ignored. +By default, a valid section name can be any string that does not contain '\\n' or ']'. +To change this, see :attr:`ConfigParser.SECTCRE`. + Configuration files may include comments, prefixed by specific characters (``#`` and ``;`` by default [1]_). Comments may appear on their own on an otherwise empty line, possibly indented. [1]_ From webhook-mailer at python.org Tue Feb 22 13:29:03 2022 From: webhook-mailer at python.org (iritkatriel) Date: Tue, 22 Feb 2022 18:29:03 -0000 Subject: [Python-checkins] bpo-46729: add number of sub-exceptions in str() of BaseExceptionGroup (GH-31294) Message-ID: https://github.com/python/cpython/commit/38b5acf8673ce42a401263a2528202e44d6ae60a commit: 38b5acf8673ce42a401263a2528202e44d6ae60a branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2022-02-22T18:28:58Z summary: bpo-46729: add number of sub-exceptions in str() of BaseExceptionGroup (GH-31294) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-22-17-18-36.bpo-46729.ZwGTFq.rst M Lib/test/test_exception_group.py M Lib/test/test_traceback.py M Objects/exceptions.c diff --git a/Lib/test/test_exception_group.py b/Lib/test/test_exception_group.py index b7b53bb2f0ece..8a55c826b8328 100644 --- a/Lib/test/test_exception_group.py +++ b/Lib/test/test_exception_group.py @@ -102,6 +102,71 @@ class MyBEG(BaseExceptionGroup): MyBEG) +class StrAndReprTests(unittest.TestCase): + def test_ExceptionGroup(self): + eg = BaseExceptionGroup( + 'flat', [ValueError(1), TypeError(2)]) + + self.assertEqual(str(eg), "flat (2 sub-exceptions)") + self.assertEqual(repr(eg), + "ExceptionGroup('flat', [ValueError(1), TypeError(2)])") + + eg = BaseExceptionGroup( + 'nested', [eg, ValueError(1), eg, TypeError(2)]) + + self.assertEqual(str(eg), "nested (4 sub-exceptions)") + self.assertEqual(repr(eg), + "ExceptionGroup('nested', " + "[ExceptionGroup('flat', " + "[ValueError(1), TypeError(2)]), " + "ValueError(1), " + "ExceptionGroup('flat', " + "[ValueError(1), TypeError(2)]), TypeError(2)])") + + def test_BaseExceptionGroup(self): + eg = BaseExceptionGroup( + 'flat', [ValueError(1), KeyboardInterrupt(2)]) + + self.assertEqual(str(eg), "flat (2 sub-exceptions)") + self.assertEqual(repr(eg), + "BaseExceptionGroup(" + "'flat', " + "[ValueError(1), KeyboardInterrupt(2)])") + + eg = BaseExceptionGroup( + 'nested', [eg, ValueError(1), eg]) + + self.assertEqual(str(eg), "nested (3 sub-exceptions)") + self.assertEqual(repr(eg), + "BaseExceptionGroup('nested', " + "[BaseExceptionGroup('flat', " + "[ValueError(1), KeyboardInterrupt(2)]), " + "ValueError(1), " + "BaseExceptionGroup('flat', " + "[ValueError(1), KeyboardInterrupt(2)])])") + + def test_custom_exception(self): + class MyEG(ExceptionGroup): + pass + + eg = MyEG( + 'flat', [ValueError(1), TypeError(2)]) + + self.assertEqual(str(eg), "flat (2 sub-exceptions)") + self.assertEqual(repr(eg), "MyEG('flat', [ValueError(1), TypeError(2)])") + + eg = MyEG( + 'nested', [eg, ValueError(1), eg, TypeError(2)]) + + self.assertEqual(str(eg), "nested (4 sub-exceptions)") + self.assertEqual(repr(eg), ( + "MyEG('nested', " + "[MyEG('flat', [ValueError(1), TypeError(2)]), " + "ValueError(1), " + "MyEG('flat', [ValueError(1), TypeError(2)]), " + "TypeError(2)])")) + + def create_simple_eg(): excs = [] try: diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index 0aa76aeb4ce09..75d668df64d4c 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -1403,7 +1403,7 @@ def exc(): f' | File "{__file__}", line {exc.__code__.co_firstlineno + 1}, in exc\n' f' | raise ExceptionGroup("eg", [ValueError(1), TypeError(2)])\n' f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' - f' | ExceptionGroup: eg\n' + f' | ExceptionGroup: eg (2 sub-exceptions)\n' f' +-+---------------- 1 ----------------\n' f' | ValueError: 1\n' f' +---------------- 2 ----------------\n' @@ -1425,7 +1425,7 @@ def exc(): f' | File "{__file__}", line {exc.__code__.co_firstlineno + 3}, in exc\n' f' | raise EG("eg1", [ValueError(1), TypeError(2)])\n' f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' - f' | ExceptionGroup: eg1\n' + f' | ExceptionGroup: eg1 (2 sub-exceptions)\n' f' +-+---------------- 1 ----------------\n' f' | ValueError: 1\n' f' +---------------- 2 ----------------\n' @@ -1441,7 +1441,7 @@ def exc(): f' | File "{__file__}", line {exc.__code__.co_firstlineno + 5}, in exc\n' f' | raise EG("eg2", [ValueError(3), TypeError(4)]) from e\n' f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' - f' | ExceptionGroup: eg2\n' + f' | ExceptionGroup: eg2 (2 sub-exceptions)\n' f' +-+---------------- 1 ----------------\n' f' | ValueError: 3\n' f' +---------------- 2 ----------------\n' @@ -1467,7 +1467,7 @@ def exc(): f' | File "{__file__}", line {exc.__code__.co_firstlineno + 4}, in exc\n' f' | raise EG("eg1", [ValueError(1), TypeError(2)])\n' f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' - f' | ExceptionGroup: eg1\n' + f' | ExceptionGroup: eg1 (2 sub-exceptions)\n' f' +-+---------------- 1 ----------------\n' f' | ValueError: 1\n' f' +---------------- 2 ----------------\n' @@ -1480,7 +1480,7 @@ def exc(): f' | File "{__file__}", line {exc.__code__.co_firstlineno + 6}, in exc\n' f' | raise EG("eg2", [ValueError(3), TypeError(4)])\n' f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' - f' | ExceptionGroup: eg2\n' + f' | ExceptionGroup: eg2 (2 sub-exceptions)\n' f' +-+---------------- 1 ----------------\n' f' | ValueError: 3\n' f' +---------------- 2 ----------------\n' @@ -1519,7 +1519,7 @@ def exc(): f' | File "{__file__}", line {exc.__code__.co_firstlineno + 9}, in exc\n' f' | raise EG("eg", [VE(1), exc, VE(4)])\n' f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' - f' | ExceptionGroup: eg\n' + f' | ExceptionGroup: eg (3 sub-exceptions)\n' f' +-+---------------- 1 ----------------\n' f' | ValueError: 1\n' f' +---------------- 2 ----------------\n' @@ -1527,7 +1527,7 @@ def exc(): f' | File "{__file__}", line {exc.__code__.co_firstlineno + 6}, in exc\n' f' | raise EG("nested", [TE(2), TE(3)])\n' f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' - f' | ExceptionGroup: nested\n' + f' | ExceptionGroup: nested (2 sub-exceptions)\n' f' +-+---------------- 1 ----------------\n' f' | TypeError: 2\n' f' +---------------- 2 ----------------\n' @@ -1546,7 +1546,7 @@ def exc(): f' | File "{__file__}", line {exc.__code__.co_firstlineno + 11}, in exc\n' f' | raise EG("top", [VE(5)])\n' f' | ^^^^^^^^^^^^^^^^^^^^^^^^\n' - f' | ExceptionGroup: top\n' + f' | ExceptionGroup: top (1 sub-exception)\n' f' +-+---------------- 1 ----------------\n' f' | ValueError: 5\n' f' +------------------------------------\n') @@ -1560,7 +1560,7 @@ def test_exception_group_width_limit(self): excs.append(ValueError(i)) eg = ExceptionGroup('eg', excs) - expected = (' | ExceptionGroup: eg\n' + expected = (' | ExceptionGroup: eg (1000 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 0\n' ' +---------------- 2 ----------------\n' @@ -1605,43 +1605,43 @@ def test_exception_group_depth_limit(self): f'eg{i}', [ValueError(i), exc, ValueError(-i)]) - expected = (' | ExceptionGroup: eg999\n' + expected = (' | ExceptionGroup: eg999 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 999\n' ' +---------------- 2 ----------------\n' - ' | ExceptionGroup: eg998\n' + ' | ExceptionGroup: eg998 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 998\n' ' +---------------- 2 ----------------\n' - ' | ExceptionGroup: eg997\n' + ' | ExceptionGroup: eg997 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 997\n' ' +---------------- 2 ----------------\n' - ' | ExceptionGroup: eg996\n' + ' | ExceptionGroup: eg996 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 996\n' ' +---------------- 2 ----------------\n' - ' | ExceptionGroup: eg995\n' + ' | ExceptionGroup: eg995 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 995\n' ' +---------------- 2 ----------------\n' - ' | ExceptionGroup: eg994\n' + ' | ExceptionGroup: eg994 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 994\n' ' +---------------- 2 ----------------\n' - ' | ExceptionGroup: eg993\n' + ' | ExceptionGroup: eg993 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 993\n' ' +---------------- 2 ----------------\n' - ' | ExceptionGroup: eg992\n' + ' | ExceptionGroup: eg992 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 992\n' ' +---------------- 2 ----------------\n' - ' | ExceptionGroup: eg991\n' + ' | ExceptionGroup: eg991 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 991\n' ' +---------------- 2 ----------------\n' - ' | ExceptionGroup: eg990\n' + ' | ExceptionGroup: eg990 (3 sub-exceptions)\n' ' +-+---------------- 1 ----------------\n' ' | ValueError: 990\n' ' +---------------- 2 ----------------\n' @@ -1707,7 +1707,7 @@ def exc(): f' | File "{__file__}", line {exc.__code__.co_firstlineno + 9}, in exc\n' f' | raise ExceptionGroup("nested", excs)\n' f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' - f' | ExceptionGroup: nested\n' + f' | ExceptionGroup: nested (2 sub-exceptions)\n' f' | >> Multi line note\n' f' | >> Because I am such\n' f' | >> an important exception.\n' @@ -2460,7 +2460,7 @@ def test_exception_group_construction(self): def test_exception_group_format_exception_only(self): teg = traceback.TracebackException(*self.eg_info) formatted = ''.join(teg.format_exception_only()).split('\n') - expected = "ExceptionGroup: eg2\n".split('\n') + expected = "ExceptionGroup: eg2 (2 sub-exceptions)\n".split('\n') self.assertEqual(formatted, expected) @@ -2476,13 +2476,13 @@ def test_exception_group_format(self): f' | File "{__file__}", line {lno_g+23}, in _get_exception_group', f' | raise ExceptionGroup("eg2", [exc3, exc4])', f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', - f' | ExceptionGroup: eg2', + f' | ExceptionGroup: eg2 (2 sub-exceptions)', f' +-+---------------- 1 ----------------', f' | Exception Group Traceback (most recent call last):', f' | File "{__file__}", line {lno_g+16}, in _get_exception_group', f' | raise ExceptionGroup("eg1", [exc1, exc2])', f' | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', - f' | ExceptionGroup: eg1', + f' | ExceptionGroup: eg1 (2 sub-exceptions)', f' +-+---------------- 1 ----------------', f' | Traceback (most recent call last):', f' | File "{__file__}", line {lno_g+9}, in _get_exception_group', @@ -2531,9 +2531,9 @@ def test_max_group_width(self): formatted = ''.join(teg.format()).split('\n') expected = [ - f' | ExceptionGroup: eg', + f' | ExceptionGroup: eg (2 sub-exceptions)', f' +-+---------------- 1 ----------------', - f' | ExceptionGroup: eg1', + f' | ExceptionGroup: eg1 (3 sub-exceptions)', f' +-+---------------- 1 ----------------', f' | ValueError: 0', f' +---------------- 2 ----------------', @@ -2542,7 +2542,7 @@ def test_max_group_width(self): f' | and 1 more exception', f' +------------------------------------', f' +---------------- 2 ----------------', - f' | ExceptionGroup: eg2', + f' | ExceptionGroup: eg2 (10 sub-exceptions)', f' +-+---------------- 1 ----------------', f' | TypeError: 0', f' +---------------- 2 ----------------', @@ -2563,11 +2563,11 @@ def test_max_group_depth(self): formatted = ''.join(teg.format()).split('\n') expected = [ - f' | ExceptionGroup: exc', + f' | ExceptionGroup: exc (3 sub-exceptions)', f' +-+---------------- 1 ----------------', f' | ValueError: -2', f' +---------------- 2 ----------------', - f' | ExceptionGroup: exc', + f' | ExceptionGroup: exc (3 sub-exceptions)', f' +-+---------------- 1 ----------------', f' | ValueError: -1', f' +---------------- 2 ----------------', diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-22-17-18-36.bpo-46729.ZwGTFq.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-17-18-36.bpo-46729.ZwGTFq.rst new file mode 100644 index 0000000000000..dbfb05fcfd9eb --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-17-18-36.bpo-46729.ZwGTFq.rst @@ -0,0 +1 @@ +Add number of sub-exceptions to :meth:`BaseException.__str__`. diff --git a/Objects/exceptions.c b/Objects/exceptions.c index 9b1c9f1018077..977dce5020382 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -836,7 +836,12 @@ BaseExceptionGroup_str(PyBaseExceptionGroupObject *self) { assert(self->msg); assert(PyUnicode_Check(self->msg)); - return Py_NewRef(self->msg); + + assert(PyTuple_CheckExact(self->excs)); + Py_ssize_t num_excs = PyTuple_Size(self->excs); + return PyUnicode_FromFormat( + "%S (%zd sub-exception%s)", + self->msg, num_excs, num_excs > 1 ? "s" : ""); } static PyObject * From webhook-mailer at python.org Tue Feb 22 13:44:10 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 22 Feb 2022 18:44:10 -0000 Subject: [Python-checkins] [3.9] bpo-20923 : [doc] Explain ConfigParser 'valid section name' and .SECTCRE (GH-31413) (GH-31507) Message-ID: https://github.com/python/cpython/commit/2387aeacc78bfd9d314428b79590b2b47511bf33 commit: 2387aeacc78bfd9d314428b79590b2b47511bf33 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T10:43:55-08:00 summary: [3.9] bpo-20923 : [doc] Explain ConfigParser 'valid section name' and .SECTCRE (GH-31413) (GH-31507) (cherry picked from commit bba8008f99d615a02984422a3825082bb5621f5a) Co-authored-by: vidhya <96202776+Vidhyavinu at users.noreply.github.com> Automerge-Triggered-By: GH:iritkatriel files: M Doc/library/configparser.rst diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index 60d33f59276f9..e5c78246881f1 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -243,6 +243,9 @@ out. Values can also span multiple lines, as long as they are indented deeper than the first line of the value. Depending on the parser's mode, blank lines may be treated as parts of multiline values or ignored. +By default, a valid section name can be any string that does not contain '\\n' or ']'. +To change this, see :attr:`ConfigParser.SECTCRE`. + Configuration files may include comments, prefixed by specific characters (``#`` and ``;`` by default [1]_). Comments may appear on their own on an otherwise empty line, possibly indented. [1]_ From webhook-mailer at python.org Tue Feb 22 15:36:26 2022 From: webhook-mailer at python.org (asvetlov) Date: Tue, 22 Feb 2022 20:36:26 -0000 Subject: [Python-checkins] bpo-46827: pass sock.type to getaddrinfo in sock_connect (GH-31499) Message-ID: https://github.com/python/cpython/commit/8fb94893e4a870ed3533e80c4bc2f1ebf1cfa9e7 commit: 8fb94893e4a870ed3533e80c4bc2f1ebf1cfa9e7 branch: main author: Thomas Grainger committer: asvetlov date: 2022-02-22T22:35:57+02:00 summary: bpo-46827: pass sock.type to getaddrinfo in sock_connect (GH-31499) Co-authored-by: Thomas Grainger Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Andrew Svetlov files: A Misc/NEWS.d/next/Library/2022-02-22-15-08-30.bpo-46827.hvj38S.rst M Lib/asyncio/selector_events.py M Lib/test/test_asyncio/test_selector_events.py diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index ebb5cbec03e4b..c3c2ec12a7787 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -498,7 +498,9 @@ async def sock_connect(self, sock, address): if not hasattr(socket, 'AF_UNIX') or sock.family != socket.AF_UNIX: resolved = await self._ensure_resolved( - address, family=sock.family, proto=sock.proto, loop=self) + address, family=sock.family, type=sock.type, proto=sock.proto, + loop=self, + ) _, _, _, _, address = resolved[0] fut = self.create_future() diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py index 9c46018c9414b..d9d30fc25b513 100644 --- a/Lib/test/test_asyncio/test_selector_events.py +++ b/Lib/test/test_asyncio/test_selector_events.py @@ -151,6 +151,24 @@ def test_write_to_self_exception(self): self.loop._csock.send.side_effect = RuntimeError() self.assertRaises(RuntimeError, self.loop._write_to_self) + @mock.patch('socket.getaddrinfo') + def test_sock_connect_resolve_using_socket_params(self, m_gai): + addr = ('need-resolution.com', 8080) + for sock_type in [socket.SOCK_STREAM, socket.SOCK_DGRAM]: + with self.subTest(sock_type): + sock = test_utils.mock_nonblocking_socket(type=sock_type) + + m_gai.side_effect = \ + lambda *args: [(None, None, None, None, ('127.0.0.1', 0))] + + con = self.loop.create_task(self.loop.sock_connect(sock, addr)) + self.loop.run_until_complete(con) + m_gai.assert_called_with( + addr[0], addr[1], sock.family, sock.type, sock.proto, 0) + + self.loop.run_until_complete(con) + sock.connect.assert_called_with(('127.0.0.1', 0)) + def test_add_reader(self): self.loop._selector.get_key.side_effect = KeyError cb = lambda: True diff --git a/Misc/NEWS.d/next/Library/2022-02-22-15-08-30.bpo-46827.hvj38S.rst b/Misc/NEWS.d/next/Library/2022-02-22-15-08-30.bpo-46827.hvj38S.rst new file mode 100644 index 0000000000000..259686ab1ddda --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-22-15-08-30.bpo-46827.hvj38S.rst @@ -0,0 +1 @@ +Support UDP sockets in :meth:`asyncio.loop.sock_connect` for selector-based event loops. Patch by Thomas Grainger. From webhook-mailer at python.org Tue Feb 22 15:58:09 2022 From: webhook-mailer at python.org (ambv) Date: Tue, 22 Feb 2022 20:58:09 -0000 Subject: [Python-checkins] bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) Message-ID: https://github.com/python/cpython/commit/fdfd7a93540b0866ba42264ecb9b0a3c2286f654 commit: fdfd7a93540b0866ba42264ecb9b0a3c2286f654 branch: 3.8 author: Sebastian Pipping committer: ambv date: 2022-02-22T21:57:53+01:00 summary: bpo-46811: Make test suite support Expat >=2.4.5 (GH-31453) Curly brackets were never allowed in namespace URIs according to RFC 3986, and so-called namespace-validating XML parsers have the right to reject them a invalid URIs. libexpat >=2.4.5 has become strcter in that regard due to related security issues; with ET.XML instantiating a namespace-aware parser under the hood, this test has no future in CPython. References: - https://datatracker.ietf.org/doc/html/rfc3968 - https://www.w3.org/TR/xml-names/ Also, test_minidom.py: Support Expat >=2.4.5 (cherry picked from commit 2cae93832f46b245847bdc252456ddf7742ef45e) Co-authored-by: Sebastian Pipping files: A Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst M Lib/test/test_minidom.py M Lib/test/test_xml_etree.py diff --git a/Lib/test/test_minidom.py b/Lib/test/test_minidom.py index 70965854ed1b1..06c91079abdd9 100644 --- a/Lib/test/test_minidom.py +++ b/Lib/test/test_minidom.py @@ -6,10 +6,12 @@ from test import support import unittest +import pyexpat import xml.dom.minidom from xml.dom.minidom import parse, Node, Document, parseString from xml.dom.minidom import getDOMImplementation +from xml.parsers.expat import ExpatError tstfile = support.findfile("test.xml", subdir="xmltestdata") @@ -1147,7 +1149,13 @@ def testEncodings(self): # Verify that character decoding errors raise exceptions instead # of crashing - self.assertRaises(UnicodeDecodeError, parseString, + if pyexpat.version_info >= (2, 4, 5): + self.assertRaises(ExpatError, parseString, + b'') + self.assertRaises(ExpatError, parseString, + b'Comment \xe7a va ? Tr\xe8s bien ?') + else: + self.assertRaises(UnicodeDecodeError, parseString, b'Comment \xe7a va ? Tr\xe8s bien ?') doc.unlink() @@ -1593,7 +1601,12 @@ def testEmptyXMLNSValue(self): self.confirm(doc2.namespaceURI == xml.dom.EMPTY_NAMESPACE) def testExceptionOnSpacesInXMLNSValue(self): - with self.assertRaisesRegex(ValueError, 'Unsupported syntax'): + if pyexpat.version_info >= (2, 4, 5): + context = self.assertRaisesRegex(ExpatError, 'syntax error') + else: + context = self.assertRaisesRegex(ValueError, 'Unsupported syntax') + + with context: parseString('') def testDocRemoveChild(self): diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index d41ff4fd077e6..0a788477fc155 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -1968,12 +1968,6 @@ def test_issue6233(self): b"\n" b'tãg') - def test_issue3151(self): - e = ET.XML('') - self.assertEqual(e.tag, '{${stuff}}localname') - t = ET.ElementTree(e) - self.assertEqual(ET.tostring(e), b'') - def test_issue6565(self): elem = ET.XML("") self.assertEqual(summarize_list(elem), ['tag']) diff --git a/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst new file mode 100644 index 0000000000000..6969bd1898f65 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-20-21-03-31.bpo-46811.8BxgdQ.rst @@ -0,0 +1 @@ +Make test suite support Expat >=2.4.5 From webhook-mailer at python.org Tue Feb 22 16:04:20 2022 From: webhook-mailer at python.org (vstinner) Date: Tue, 22 Feb 2022 21:04:20 -0000 Subject: [Python-checkins] bpo-46659: Fix the MBCS codec alias on Windows (GH-31218) Message-ID: https://github.com/python/cpython/commit/ccbe8045faf6e63d36229ea4e1b9298572cda126 commit: ccbe8045faf6e63d36229ea4e1b9298572cda126 branch: main author: Victor Stinner committer: vstinner date: 2022-02-22T22:04:07+01:00 summary: bpo-46659: Fix the MBCS codec alias on Windows (GH-31218) files: M Lib/encodings/__init__.py M Lib/test/test_codecs.py diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py index dff22a4b27f45..f9075b8f0d98a 100644 --- a/Lib/encodings/__init__.py +++ b/Lib/encodings/__init__.py @@ -152,7 +152,14 @@ def search_function(encoding): # Return the registry entry return entry +# Register the search_function in the Python codec registry +codecs.register(search_function) + if sys.platform == 'win32': + # bpo-671666, bpo-46668: If Python does not implement a codec for current + # Windows ANSI code page, use the "mbcs" codec instead: + # WideCharToMultiByte() and MultiByteToWideChar() functions with CP_ACP. + # Python does not support custom code pages. def _alias_mbcs(encoding): try: import _winapi @@ -164,8 +171,4 @@ def _alias_mbcs(encoding): # Imports may fail while we are shutting down pass - # It must be registered before search_function() codecs.register(_alias_mbcs) - -# Register the search_function in the Python codec registry -codecs.register(search_function) diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index d30ff8f82db9a..8118ec6f4727b 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -3191,13 +3191,16 @@ def test_incremental(self): self.assertEqual(decoded, ('abc', 3)) def test_mbcs_alias(self): - # On Windows, the encoding name must be the ANSI code page - encoding = locale.getpreferredencoding(False) - self.assertTrue(encoding.startswith('cp'), encoding) - - # The encodings module create a "mbcs" alias to the ANSI code page - codec = codecs.lookup(encoding) - self.assertEqual(codec.name, "mbcs") + # Check that looking up our 'default' codepage will return + # mbcs when we don't have a more specific one available + code_page = 99_999 + name = f'cp{code_page}' + with mock.patch('_winapi.GetACP', return_value=code_page): + try: + codec = codecs.lookup(name) + self.assertEqual(codec.name, 'mbcs') + finally: + codecs.unregister(name) @support.bigmemtest(size=2**31, memuse=7, dry_run=False) def test_large_input(self, size): From webhook-mailer at python.org Tue Feb 22 16:06:50 2022 From: webhook-mailer at python.org (vstinner) Date: Tue, 22 Feb 2022 21:06:50 -0000 Subject: [Python-checkins] bpo-46659: Deprecate locale.getdefaultlocale() (GH-31206) Message-ID: https://github.com/python/cpython/commit/b899126094731bc49fecb61f2c1b7557d74ca839 commit: b899126094731bc49fecb61f2c1b7557d74ca839 branch: main author: Victor Stinner committer: vstinner date: 2022-02-22T22:06:43+01:00 summary: bpo-46659: Deprecate locale.getdefaultlocale() (GH-31206) The locale.getdefaultlocale() function is deprecated and will be removed in Python 3.13. Use locale.setlocale(), locale.getpreferredencoding(False) and locale.getlocale() functions instead. files: A Misc/NEWS.d/next/Library/2022-02-06-17-57-45.bpo-46659.zTmkoQ.rst M Doc/library/locale.rst M Doc/whatsnew/3.11.rst M Lib/locale.py M Lib/test/test_locale.py diff --git a/Doc/library/locale.rst b/Doc/library/locale.rst index 60d0c59d017c7..1b147342cef14 100644 --- a/Doc/library/locale.rst +++ b/Doc/library/locale.rst @@ -301,6 +301,8 @@ The :mod:`locale` module defines the following exception and functions: *language code* and *encoding* may be ``None`` if their values cannot be determined. + .. deprecated:: 3.11 3.13 + .. function:: getlocale(category=LC_CTYPE) diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 85f12fe8b4fc7..32f021f03ccce 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -486,6 +486,12 @@ Deprecated (Contributed by Hugo van Kemenade in :issue:`45173`.) +* The :func:`locale.getdefaultlocale` function is deprecated and will be + removed in Python 3.13. Use :func:`locale.setlocale`, + :func:`locale.getpreferredencoding(False) ` and + :func:`locale.getlocale` functions instead. + (Contributed by Victor Stinner in :issue:`46659`.) + Removed ======= diff --git a/Lib/locale.py b/Lib/locale.py index 4bd31c9fa2cdf..a710f27a807b0 100644 --- a/Lib/locale.py +++ b/Lib/locale.py @@ -555,6 +555,12 @@ def getdefaultlocale(envvars=('LC_ALL', 'LC_CTYPE', 'LANG', 'LANGUAGE')): """ + import warnings + warnings.warn( + "Use setlocale(), getpreferredencoding(False) and getlocale() instead", + DeprecationWarning, stacklevel=2 + ) + try: # check if it's supported by the _locale module import _locale diff --git a/Lib/test/test_locale.py b/Lib/test/test_locale.py index f844e62ca2e72..2a3b0acc6bd60 100644 --- a/Lib/test/test_locale.py +++ b/Lib/test/test_locale.py @@ -518,7 +518,8 @@ def test_defaults_UTF8(self): os.environ['LC_CTYPE'] = 'UTF-8' - self.assertEqual(locale.getdefaultlocale(), (None, 'UTF-8')) + with check_warnings(('', DeprecationWarning)): + self.assertEqual(locale.getdefaultlocale(), (None, 'UTF-8')) finally: for k in orig_env: diff --git a/Misc/NEWS.d/next/Library/2022-02-06-17-57-45.bpo-46659.zTmkoQ.rst b/Misc/NEWS.d/next/Library/2022-02-06-17-57-45.bpo-46659.zTmkoQ.rst new file mode 100644 index 0000000000000..6fd9a53c260f4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-06-17-57-45.bpo-46659.zTmkoQ.rst @@ -0,0 +1,4 @@ +The :func:`locale.getdefaultlocale` function is deprecated and will be removed +in Python 3.13. Use :func:`locale.setlocale`, +:func:`locale.getpreferredencoding(False) ` and +:func:`locale.getlocale` functions instead. Patch by Victor Stinner. From webhook-mailer at python.org Tue Feb 22 16:09:43 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 22 Feb 2022 21:09:43 -0000 Subject: [Python-checkins] bpo-46827: pass sock.type to getaddrinfo in sock_connect (GH-31499) Message-ID: https://github.com/python/cpython/commit/d327517b54c148eba499c777b99760356102cbe0 commit: d327517b54c148eba499c777b99760356102cbe0 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T13:09:35-08:00 summary: bpo-46827: pass sock.type to getaddrinfo in sock_connect (GH-31499) Co-authored-by: Thomas Grainger Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Andrew Svetlov (cherry picked from commit 8fb94893e4a870ed3533e80c4bc2f1ebf1cfa9e7) Co-authored-by: Thomas Grainger files: A Misc/NEWS.d/next/Library/2022-02-22-15-08-30.bpo-46827.hvj38S.rst M Lib/asyncio/selector_events.py M Lib/test/test_asyncio/test_selector_events.py diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index 6f764450ae3be..71080b8ad16c6 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -489,7 +489,9 @@ async def sock_connect(self, sock, address): if not hasattr(socket, 'AF_UNIX') or sock.family != socket.AF_UNIX: resolved = await self._ensure_resolved( - address, family=sock.family, proto=sock.proto, loop=self) + address, family=sock.family, type=sock.type, proto=sock.proto, + loop=self, + ) _, _, _, _, address = resolved[0] fut = self.create_future() diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py index 1613c753c26ee..ac221da4432f1 100644 --- a/Lib/test/test_asyncio/test_selector_events.py +++ b/Lib/test/test_asyncio/test_selector_events.py @@ -188,6 +188,24 @@ def test_write_to_self_exception(self): self.loop._csock.send.side_effect = RuntimeError() self.assertRaises(RuntimeError, self.loop._write_to_self) + @mock.patch('socket.getaddrinfo') + def test_sock_connect_resolve_using_socket_params(self, m_gai): + addr = ('need-resolution.com', 8080) + for sock_type in [socket.SOCK_STREAM, socket.SOCK_DGRAM]: + with self.subTest(sock_type): + sock = test_utils.mock_nonblocking_socket(type=sock_type) + + m_gai.side_effect = \ + lambda *args: [(None, None, None, None, ('127.0.0.1', 0))] + + con = self.loop.create_task(self.loop.sock_connect(sock, addr)) + self.loop.run_until_complete(con) + m_gai.assert_called_with( + addr[0], addr[1], sock.family, sock.type, sock.proto, 0) + + self.loop.run_until_complete(con) + sock.connect.assert_called_with(('127.0.0.1', 0)) + def test_add_reader(self): self.loop._selector.get_key.side_effect = KeyError cb = lambda: True diff --git a/Misc/NEWS.d/next/Library/2022-02-22-15-08-30.bpo-46827.hvj38S.rst b/Misc/NEWS.d/next/Library/2022-02-22-15-08-30.bpo-46827.hvj38S.rst new file mode 100644 index 0000000000000..259686ab1ddda --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-22-15-08-30.bpo-46827.hvj38S.rst @@ -0,0 +1 @@ +Support UDP sockets in :meth:`asyncio.loop.sock_connect` for selector-based event loops. Patch by Thomas Grainger. From webhook-mailer at python.org Tue Feb 22 17:11:59 2022 From: webhook-mailer at python.org (vstinner) Date: Tue, 22 Feb 2022 22:11:59 -0000 Subject: [Python-checkins] bpo-45459: Rename buffer.h to pybuffer.h (#31201) Message-ID: https://github.com/python/cpython/commit/66b3cd7063322a9f5c922a97bbd06fdb98309999 commit: 66b3cd7063322a9f5c922a97bbd06fdb98309999 branch: main author: Victor Stinner committer: vstinner date: 2022-02-22T23:11:48+01:00 summary: bpo-45459: Rename buffer.h to pybuffer.h (#31201) Rename Include/buffer.h header file to Include/pybuffer.h to avoid conflicts with projects having an existing "buffer.h" header file. * Incude pybuffer.h before object.h in Python.h. * Remove #include "buffer.h" from Include/cpython/object.h. * Add a forward declaration of the PyObject type in pybuffer.h to fix an inter-dependency issue. files: A Include/pybuffer.h A Misc/NEWS.d/next/C API/2022-02-07-18-47-00.bpo-45459.0FCWM8.rst D Include/buffer.h M Include/Python.h M Include/cpython/object.h M Makefile.pre.in M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters diff --git a/Include/Python.h b/Include/Python.h index 5416b04e4bfb3..5bc8cc656af0d 100644 --- a/Include/Python.h +++ b/Include/Python.h @@ -39,6 +39,7 @@ #include "pymacro.h" #include "pymath.h" #include "pymem.h" +#include "pybuffer.h" #include "object.h" #include "objimpl.h" #include "typeslots.h" @@ -50,7 +51,6 @@ #include "longobject.h" #include "cpython/longintrepr.h" #include "boolobject.h" -#include "buffer.h" #include "floatobject.h" #include "complexobject.h" #include "rangeobject.h" diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 2ee97ba0b40af..6cc3d72ca7084 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -2,8 +2,6 @@ # error "this header file must not be included directly" #endif -#include "buffer.h" // for Py_buffer, included after PyObject has been defined - PyAPI_FUNC(void) _Py_NewReference(PyObject *op); #ifdef Py_TRACE_REFS diff --git a/Include/buffer.h b/Include/pybuffer.h similarity index 96% rename from Include/buffer.h rename to Include/pybuffer.h index 6893505e66e3e..31795b780710f 100644 --- a/Include/buffer.h +++ b/Include/pybuffer.h @@ -17,6 +17,10 @@ extern "C" { * */ +// Forward declaration to be able to include pybuffer.h before object.h: +// pybuffer.h uses PyObject and object.h uses Py_buffer. +typedef struct _object PyObject; + typedef struct { void *buf; PyObject *obj; /* owned reference */ diff --git a/Makefile.pre.in b/Makefile.pre.in index 2464bbdf2ddd0..2a3e0fbceb4c0 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1439,7 +1439,6 @@ PYTHON_HEADERS= \ $(srcdir)/Include/abstract.h \ $(srcdir)/Include/bltinmodule.h \ $(srcdir)/Include/boolobject.h \ - $(srcdir)/Include/buffer.h \ $(srcdir)/Include/bytearrayobject.h \ $(srcdir)/Include/bytesobject.h \ $(srcdir)/Include/ceval.h \ @@ -1472,6 +1471,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/osdefs.h \ $(srcdir)/Include/osmodule.h \ $(srcdir)/Include/patchlevel.h \ + $(srcdir)/Include/pybuffer.h \ $(srcdir)/Include/pycapsule.h \ $(srcdir)/Include/pydtrace.h \ $(srcdir)/Include/pyerrors.h \ diff --git a/Misc/NEWS.d/next/C API/2022-02-07-18-47-00.bpo-45459.0FCWM8.rst b/Misc/NEWS.d/next/C API/2022-02-07-18-47-00.bpo-45459.0FCWM8.rst new file mode 100644 index 0000000000000..711c107746d64 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-02-07-18-47-00.bpo-45459.0FCWM8.rst @@ -0,0 +1,3 @@ +Rename ``Include/buffer.h`` header file to ``Include/pybuffer.h`` to avoid +conflits with projects having an existing ``buffer.h`` header file. Patch by +Victor Stinner. diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 8f9c4fe63b8dc..c2f1a016bc111 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -263,6 +263,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index dc3b554779486..b300103bbb420 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -147,6 +147,9 @@ Include + + Include + Include From webhook-mailer at python.org Tue Feb 22 18:39:33 2022 From: webhook-mailer at python.org (asvetlov) Date: Tue, 22 Feb 2022 23:39:33 -0000 Subject: [Python-checkins] Inherit asyncio proactor datagram transport from asyncio.DatagramTransport (#31512) Message-ID: https://github.com/python/cpython/commit/cff4d5c5d29528299ec1ac5b3b3a6f7735577c01 commit: cff4d5c5d29528299ec1ac5b3b3a6f7735577c01 branch: main author: Andrew Svetlov committer: asvetlov date: 2022-02-23T01:39:24+02:00 summary: Inherit asyncio proactor datagram transport from asyncio.DatagramTransport (#31512) files: A Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst M Lib/asyncio/proactor_events.py M Lib/test/test_asyncio/test_proactor_events.py diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py index ae59f30db1c3c..43d5e70b79cac 100644 --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -452,7 +452,8 @@ def _pipe_closed(self, fut): self.close() -class _ProactorDatagramTransport(_ProactorBasePipeTransport): +class _ProactorDatagramTransport(_ProactorBasePipeTransport, + transports.DatagramTransport): max_size = 256 * 1024 def __init__(self, loop, sock, protocol, address=None, waiter=None, extra=None): diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py index 45af8101a0093..67573823ecb22 100644 --- a/Lib/test/test_asyncio/test_proactor_events.py +++ b/Lib/test/test_asyncio/test_proactor_events.py @@ -839,6 +839,7 @@ def datagram_transport(self): def test_make_datagram_transport(self): tr = self.datagram_transport() self.assertIsInstance(tr, _ProactorDatagramTransport) + self.assertIsInstance(tr, asyncio.DatagramTransport) close_transport(tr) def test_datagram_loop_writing(self): diff --git a/Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst b/Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst new file mode 100644 index 0000000000000..be223ddd58ba1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst @@ -0,0 +1,2 @@ +Inherit asyncio proactor datagram transport from +:class:`asyncio.DatagramTransport`. From webhook-mailer at python.org Tue Feb 22 19:08:52 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 23 Feb 2022 00:08:52 -0000 Subject: [Python-checkins] Inherit asyncio proactor datagram transport from asyncio.DatagramTransport (GH-31512) Message-ID: https://github.com/python/cpython/commit/f3bb8fe4e831ef452a503b3db2297f6628c63288 commit: f3bb8fe4e831ef452a503b3db2297f6628c63288 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T16:08:43-08:00 summary: Inherit asyncio proactor datagram transport from asyncio.DatagramTransport (GH-31512) (cherry picked from commit cff4d5c5d29528299ec1ac5b3b3a6f7735577c01) Co-authored-by: Andrew Svetlov files: A Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst M Lib/asyncio/proactor_events.py M Lib/test/test_asyncio/test_proactor_events.py diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py index b4cd414b82d72..6762253abc967 100644 --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -450,7 +450,8 @@ def _pipe_closed(self, fut): self.close() -class _ProactorDatagramTransport(_ProactorBasePipeTransport): +class _ProactorDatagramTransport(_ProactorBasePipeTransport, + transports.DatagramTransport): max_size = 256 * 1024 def __init__(self, loop, sock, protocol, address=None, waiter=None, extra=None): diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py index 59d5e5b6ccb63..1bbb487626e89 100644 --- a/Lib/test/test_asyncio/test_proactor_events.py +++ b/Lib/test/test_asyncio/test_proactor_events.py @@ -823,6 +823,7 @@ def datagram_transport(self): def test_make_datagram_transport(self): tr = self.datagram_transport() self.assertIsInstance(tr, _ProactorDatagramTransport) + self.assertIsInstance(tr, asyncio.DatagramTransport) close_transport(tr) def test_datagram_loop_writing(self): diff --git a/Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst b/Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst new file mode 100644 index 0000000000000..be223ddd58ba1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst @@ -0,0 +1,2 @@ +Inherit asyncio proactor datagram transport from +:class:`asyncio.DatagramTransport`. From webhook-mailer at python.org Tue Feb 22 19:24:18 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Wed, 23 Feb 2022 00:24:18 -0000 Subject: [Python-checkins] bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) Message-ID: https://github.com/python/cpython/commit/1f455361ecfb1892e375bdbee813cdf095b6cfb8 commit: 1f455361ecfb1892e375bdbee813cdf095b6cfb8 branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-22T17:23:51-07:00 summary: bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366) https://bugs.python.org/issue46765 files: M Include/internal/pycore_global_strings.h M Include/internal/pycore_runtime_init.h M Include/internal/pycore_symtable.h M Modules/_io/_iomodule.c M Modules/_io/_iomodule.h M Modules/_io/bufferedio.c M Modules/_io/iobase.c M Modules/_io/stringio.c M Modules/_io/textio.c M Modules/_tracemalloc.c M Objects/boolobject.c M Objects/classobject.c M Objects/codeobject.c M Objects/listobject.c M Python/_warnings.c M Python/ast_unparse.c M Python/compile.c M Python/pylifecycle.c M Python/symtable.c M Python/sysmodule.c M Tools/c-analyzer/cpython/globals-to-fix.tsv M Tools/c-analyzer/cpython/ignored.tsv diff --git a/Include/internal/pycore_global_strings.h b/Include/internal/pycore_global_strings.h index aa597bc8281a5..2a42dc16ce1b5 100644 --- a/Include/internal/pycore_global_strings.h +++ b/Include/internal/pycore_global_strings.h @@ -35,17 +35,26 @@ struct _Py_global_strings { STRUCT_FOR_STR(anon_module, "") STRUCT_FOR_STR(anon_setcomp, "") STRUCT_FOR_STR(anon_string, "") + STRUCT_FOR_STR(anon_unknown, "") + STRUCT_FOR_STR(close_br, "}") STRUCT_FOR_STR(comma_sep, ", ") + STRUCT_FOR_STR(dbl_close_br, "}}") + STRUCT_FOR_STR(dbl_open_br, "{{") STRUCT_FOR_STR(dbl_percent, "%%") STRUCT_FOR_STR(dot, ".") STRUCT_FOR_STR(dot_locals, ".") STRUCT_FOR_STR(empty, "") + STRUCT_FOR_STR(list_err, "list index out of range") + STRUCT_FOR_STR(newline, "\n") + STRUCT_FOR_STR(open_br, "{") STRUCT_FOR_STR(percent, "%") } literals; struct { + STRUCT_FOR_ID(False) STRUCT_FOR_ID(Py_Repr) STRUCT_FOR_ID(TextIOWrapper) + STRUCT_FOR_ID(True) STRUCT_FOR_ID(WarningMessage) STRUCT_FOR_ID(_) STRUCT_FOR_ID(__IOBase_closed) @@ -194,7 +203,9 @@ struct _Py_global_strings { STRUCT_FOR_ID(__weakref__) STRUCT_FOR_ID(__xor__) STRUCT_FOR_ID(_abc_impl) + STRUCT_FOR_ID(_annotation) STRUCT_FOR_ID(_blksize) + STRUCT_FOR_ID(_bootstrap) STRUCT_FOR_ID(_dealloc_warn) STRUCT_FOR_ID(_finalizing) STRUCT_FOR_ID(_find_and_load) @@ -216,23 +227,31 @@ struct _Py_global_strings { STRUCT_FOR_ID(big) STRUCT_FOR_ID(buffer) STRUCT_FOR_ID(builtins) + STRUCT_FOR_ID(c_call) + STRUCT_FOR_ID(c_exception) + STRUCT_FOR_ID(c_return) + STRUCT_FOR_ID(call) STRUCT_FOR_ID(clear) STRUCT_FOR_ID(close) + STRUCT_FOR_ID(closed) STRUCT_FOR_ID(code) STRUCT_FOR_ID(copy) STRUCT_FOR_ID(copyreg) STRUCT_FOR_ID(decode) STRUCT_FOR_ID(default) STRUCT_FOR_ID(defaultaction) + STRUCT_FOR_ID(dictcomp) STRUCT_FOR_ID(difference_update) STRUCT_FOR_ID(dispatch_table) STRUCT_FOR_ID(displayhook) STRUCT_FOR_ID(enable) + STRUCT_FOR_ID(encode) STRUCT_FOR_ID(encoding) STRUCT_FOR_ID(end_lineno) STRUCT_FOR_ID(end_offset) STRUCT_FOR_ID(errors) STRUCT_FOR_ID(excepthook) + STRUCT_FOR_ID(exception) STRUCT_FOR_ID(extend) STRUCT_FOR_ID(filename) STRUCT_FOR_ID(fileno) @@ -240,22 +259,30 @@ struct _Py_global_strings { STRUCT_FOR_ID(filters) STRUCT_FOR_ID(find_class) STRUCT_FOR_ID(flush) + STRUCT_FOR_ID(genexpr) STRUCT_FOR_ID(get) STRUCT_FOR_ID(get_source) STRUCT_FOR_ID(getattr) + STRUCT_FOR_ID(getstate) STRUCT_FOR_ID(ignore) STRUCT_FOR_ID(importlib) + STRUCT_FOR_ID(inf) STRUCT_FOR_ID(intersection) STRUCT_FOR_ID(isatty) STRUCT_FOR_ID(items) STRUCT_FOR_ID(iter) + STRUCT_FOR_ID(join) STRUCT_FOR_ID(keys) + STRUCT_FOR_ID(lambda) STRUCT_FOR_ID(last_traceback) STRUCT_FOR_ID(last_type) STRUCT_FOR_ID(last_value) STRUCT_FOR_ID(latin1) + STRUCT_FOR_ID(line) STRUCT_FOR_ID(lineno) + STRUCT_FOR_ID(listcomp) STRUCT_FOR_ID(little) + STRUCT_FOR_ID(locale) STRUCT_FOR_ID(match) STRUCT_FOR_ID(metaclass) STRUCT_FOR_ID(mode) @@ -266,9 +293,11 @@ struct _Py_global_strings { STRUCT_FOR_ID(n_sequence_fields) STRUCT_FOR_ID(n_unnamed_fields) STRUCT_FOR_ID(name) + STRUCT_FOR_ID(newlines) STRUCT_FOR_ID(obj) STRUCT_FOR_ID(offset) STRUCT_FOR_ID(onceregistry) + STRUCT_FOR_ID(opcode) STRUCT_FOR_ID(open) STRUCT_FOR_ID(parent) STRUCT_FOR_ID(partial) @@ -296,6 +325,7 @@ struct _Py_global_strings { STRUCT_FOR_ID(seek) STRUCT_FOR_ID(seekable) STRUCT_FOR_ID(send) + STRUCT_FOR_ID(setcomp) STRUCT_FOR_ID(setstate) STRUCT_FOR_ID(sort) STRUCT_FOR_ID(stderr) @@ -307,6 +337,8 @@ struct _Py_global_strings { STRUCT_FOR_ID(text) STRUCT_FOR_ID(threading) STRUCT_FOR_ID(throw) + STRUCT_FOR_ID(top) + STRUCT_FOR_ID(truncate) STRUCT_FOR_ID(unraisablehook) STRUCT_FOR_ID(values) STRUCT_FOR_ID(version) diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h index 04c1e671235ea..8f62343443351 100644 --- a/Include/internal/pycore_runtime_init.h +++ b/Include/internal/pycore_runtime_init.h @@ -651,16 +651,25 @@ extern "C" { INIT_STR(anon_module, ""), \ INIT_STR(anon_setcomp, ""), \ INIT_STR(anon_string, ""), \ + INIT_STR(anon_unknown, ""), \ + INIT_STR(close_br, "}"), \ INIT_STR(comma_sep, ", "), \ + INIT_STR(dbl_close_br, "}}"), \ + INIT_STR(dbl_open_br, "{{"), \ INIT_STR(dbl_percent, "%%"), \ INIT_STR(dot, "."), \ INIT_STR(dot_locals, "."), \ INIT_STR(empty, ""), \ + INIT_STR(list_err, "list index out of range"), \ + INIT_STR(newline, "\n"), \ + INIT_STR(open_br, "{"), \ INIT_STR(percent, "%"), \ }, \ .identifiers = { \ + INIT_ID(False), \ INIT_ID(Py_Repr), \ INIT_ID(TextIOWrapper), \ + INIT_ID(True), \ INIT_ID(WarningMessage), \ INIT_ID(_), \ INIT_ID(__IOBase_closed), \ @@ -809,7 +818,9 @@ extern "C" { INIT_ID(__weakref__), \ INIT_ID(__xor__), \ INIT_ID(_abc_impl), \ + INIT_ID(_annotation), \ INIT_ID(_blksize), \ + INIT_ID(_bootstrap), \ INIT_ID(_dealloc_warn), \ INIT_ID(_finalizing), \ INIT_ID(_find_and_load), \ @@ -831,23 +842,31 @@ extern "C" { INIT_ID(big), \ INIT_ID(buffer), \ INIT_ID(builtins), \ + INIT_ID(c_call), \ + INIT_ID(c_exception), \ + INIT_ID(c_return), \ + INIT_ID(call), \ INIT_ID(clear), \ INIT_ID(close), \ + INIT_ID(closed), \ INIT_ID(code), \ INIT_ID(copy), \ INIT_ID(copyreg), \ INIT_ID(decode), \ INIT_ID(default), \ INIT_ID(defaultaction), \ + INIT_ID(dictcomp), \ INIT_ID(difference_update), \ INIT_ID(dispatch_table), \ INIT_ID(displayhook), \ INIT_ID(enable), \ + INIT_ID(encode), \ INIT_ID(encoding), \ INIT_ID(end_lineno), \ INIT_ID(end_offset), \ INIT_ID(errors), \ INIT_ID(excepthook), \ + INIT_ID(exception), \ INIT_ID(extend), \ INIT_ID(filename), \ INIT_ID(fileno), \ @@ -855,22 +874,30 @@ extern "C" { INIT_ID(filters), \ INIT_ID(find_class), \ INIT_ID(flush), \ + INIT_ID(genexpr), \ INIT_ID(get), \ INIT_ID(get_source), \ INIT_ID(getattr), \ + INIT_ID(getstate), \ INIT_ID(ignore), \ INIT_ID(importlib), \ + INIT_ID(inf), \ INIT_ID(intersection), \ INIT_ID(isatty), \ INIT_ID(items), \ INIT_ID(iter), \ + INIT_ID(join), \ INIT_ID(keys), \ + INIT_ID(lambda), \ INIT_ID(last_traceback), \ INIT_ID(last_type), \ INIT_ID(last_value), \ INIT_ID(latin1), \ + INIT_ID(line), \ INIT_ID(lineno), \ + INIT_ID(listcomp), \ INIT_ID(little), \ + INIT_ID(locale), \ INIT_ID(match), \ INIT_ID(metaclass), \ INIT_ID(mode), \ @@ -881,9 +908,11 @@ extern "C" { INIT_ID(n_sequence_fields), \ INIT_ID(n_unnamed_fields), \ INIT_ID(name), \ + INIT_ID(newlines), \ INIT_ID(obj), \ INIT_ID(offset), \ INIT_ID(onceregistry), \ + INIT_ID(opcode), \ INIT_ID(open), \ INIT_ID(parent), \ INIT_ID(partial), \ @@ -911,6 +940,7 @@ extern "C" { INIT_ID(seek), \ INIT_ID(seekable), \ INIT_ID(send), \ + INIT_ID(setcomp), \ INIT_ID(setstate), \ INIT_ID(sort), \ INIT_ID(stderr), \ @@ -922,6 +952,8 @@ extern "C" { INIT_ID(text), \ INIT_ID(threading), \ INIT_ID(throw), \ + INIT_ID(top), \ + INIT_ID(truncate), \ INIT_ID(unraisablehook), \ INIT_ID(values), \ INIT_ID(version), \ diff --git a/Include/internal/pycore_symtable.h b/Include/internal/pycore_symtable.h index 4ecfab5585032..28935f4ed5501 100644 --- a/Include/internal/pycore_symtable.h +++ b/Include/internal/pycore_symtable.h @@ -128,8 +128,6 @@ extern struct symtable* _Py_SymtableStringObjectFlags( int start, PyCompilerFlags *flags); -extern void _PySymtable_Fini(void); - #ifdef __cplusplus } #endif diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c index 9ecc6718cb1fd..7f029f26078b8 100644 --- a/Modules/_io/_iomodule.c +++ b/Modules/_io/_iomodule.c @@ -24,38 +24,6 @@ #include #endif -/* Various interned strings */ - -PyObject *_PyIO_str_close = NULL; -PyObject *_PyIO_str_closed = NULL; -PyObject *_PyIO_str_decode = NULL; -PyObject *_PyIO_str_encode = NULL; -PyObject *_PyIO_str_fileno = NULL; -PyObject *_PyIO_str_flush = NULL; -PyObject *_PyIO_str_getstate = NULL; -PyObject *_PyIO_str_isatty = NULL; -PyObject *_PyIO_str_locale = NULL; -PyObject *_PyIO_str_newlines = NULL; -PyObject *_PyIO_str_nl = NULL; -PyObject *_PyIO_str_peek = NULL; -PyObject *_PyIO_str_read = NULL; -PyObject *_PyIO_str_read1 = NULL; -PyObject *_PyIO_str_readable = NULL; -PyObject *_PyIO_str_readall = NULL; -PyObject *_PyIO_str_readinto = NULL; -PyObject *_PyIO_str_readline = NULL; -PyObject *_PyIO_str_reset = NULL; -PyObject *_PyIO_str_seek = NULL; -PyObject *_PyIO_str_seekable = NULL; -PyObject *_PyIO_str_setstate = NULL; -PyObject *_PyIO_str_tell = NULL; -PyObject *_PyIO_str_truncate = NULL; -PyObject *_PyIO_str_writable = NULL; -PyObject *_PyIO_str_write = NULL; - -PyObject *_PyIO_empty_str = NULL; -PyObject *_PyIO_empty_bytes = NULL; - PyDoc_STRVAR(module_doc, "The io module provides the Python interfaces to stream handling. The\n" "builtin open function is defined in this module.\n" @@ -511,8 +479,7 @@ _io_text_encoding_impl(PyObject *module, PyObject *encoding, int stacklevel) return NULL; } } - Py_INCREF(_PyIO_str_locale); - return _PyIO_str_locale; + return &_Py_ID(locale); } Py_INCREF(encoding); return encoding; @@ -699,41 +666,6 @@ _PyIO_Fini(void) PyTypeObject *exc = static_types[i]; _PyStaticType_Dealloc(exc); } - - /* Interned strings */ -#define CLEAR_INTERNED(name) \ - Py_CLEAR(_PyIO_str_ ## name) - - CLEAR_INTERNED(close); - CLEAR_INTERNED(closed); - CLEAR_INTERNED(decode); - CLEAR_INTERNED(encode); - CLEAR_INTERNED(fileno); - CLEAR_INTERNED(flush); - CLEAR_INTERNED(getstate); - CLEAR_INTERNED(isatty); - CLEAR_INTERNED(locale); - CLEAR_INTERNED(newlines); - CLEAR_INTERNED(peek); - CLEAR_INTERNED(read); - CLEAR_INTERNED(read1); - CLEAR_INTERNED(readable); - CLEAR_INTERNED(readall); - CLEAR_INTERNED(readinto); - CLEAR_INTERNED(readline); - CLEAR_INTERNED(reset); - CLEAR_INTERNED(seek); - CLEAR_INTERNED(seekable); - CLEAR_INTERNED(setstate); - CLEAR_INTERNED(tell); - CLEAR_INTERNED(truncate); - CLEAR_INTERNED(write); - CLEAR_INTERNED(writable); -#undef CLEAR_INTERNED - - Py_CLEAR(_PyIO_str_nl); - Py_CLEAR(_PyIO_empty_str); - Py_CLEAR(_PyIO_empty_bytes); } @@ -797,50 +729,6 @@ PyInit__io(void) } } - /* Interned strings */ -#define ADD_INTERNED(name) \ - if (!_PyIO_str_ ## name && \ - !(_PyIO_str_ ## name = PyUnicode_InternFromString(# name))) \ - goto fail; - - ADD_INTERNED(close) - ADD_INTERNED(closed) - ADD_INTERNED(decode) - ADD_INTERNED(encode) - ADD_INTERNED(fileno) - ADD_INTERNED(flush) - ADD_INTERNED(getstate) - ADD_INTERNED(isatty) - ADD_INTERNED(locale) - ADD_INTERNED(newlines) - ADD_INTERNED(peek) - ADD_INTERNED(read) - ADD_INTERNED(read1) - ADD_INTERNED(readable) - ADD_INTERNED(readall) - ADD_INTERNED(readinto) - ADD_INTERNED(readline) - ADD_INTERNED(reset) - ADD_INTERNED(seek) - ADD_INTERNED(seekable) - ADD_INTERNED(setstate) - ADD_INTERNED(tell) - ADD_INTERNED(truncate) - ADD_INTERNED(write) - ADD_INTERNED(writable) -#undef ADD_INTERNED - - if (!_PyIO_str_nl && - !(_PyIO_str_nl = PyUnicode_InternFromString("\n"))) - goto fail; - - if (!_PyIO_empty_str && - !(_PyIO_empty_str = PyUnicode_FromStringAndSize(NULL, 0))) - goto fail; - if (!_PyIO_empty_bytes && - !(_PyIO_empty_bytes = PyBytes_FromStringAndSize(NULL, 0))) - goto fail; - state->initialized = 1; return m; diff --git a/Modules/_io/_iomodule.h b/Modules/_io/_iomodule.h index 638797fd35736..c260080f0e348 100644 --- a/Modules/_io/_iomodule.h +++ b/Modules/_io/_iomodule.h @@ -155,33 +155,4 @@ extern _PyIO_State *_PyIO_get_module_state(void); extern char _PyIO_get_console_type(PyObject *); #endif -extern PyObject *_PyIO_str_close; -extern PyObject *_PyIO_str_closed; -extern PyObject *_PyIO_str_decode; -extern PyObject *_PyIO_str_encode; -extern PyObject *_PyIO_str_fileno; -extern PyObject *_PyIO_str_flush; -extern PyObject *_PyIO_str_getstate; -extern PyObject *_PyIO_str_isatty; -extern PyObject *_PyIO_str_newlines; -extern PyObject *_PyIO_str_nl; -extern PyObject *_PyIO_str_peek; -extern PyObject *_PyIO_str_read; -extern PyObject *_PyIO_str_read1; -extern PyObject *_PyIO_str_readable; -extern PyObject *_PyIO_str_readall; -extern PyObject *_PyIO_str_readinto; -extern PyObject *_PyIO_str_readline; -extern PyObject *_PyIO_str_reset; -extern PyObject *_PyIO_str_seek; -extern PyObject *_PyIO_str_seekable; -extern PyObject *_PyIO_str_setstate; -extern PyObject *_PyIO_str_tell; -extern PyObject *_PyIO_str_truncate; -extern PyObject *_PyIO_str_writable; -extern PyObject *_PyIO_str_write; - -extern PyObject *_PyIO_empty_str; -extern PyObject *_PyIO_empty_bytes; - extern Py_EXPORTED_SYMBOL PyTypeObject _PyBytesIOBuffer_Type; diff --git a/Modules/_io/bufferedio.c b/Modules/_io/bufferedio.c index bb93ae361f892..ac30d1d632bff 100644 --- a/Modules/_io/bufferedio.c +++ b/Modules/_io/bufferedio.c @@ -443,7 +443,7 @@ static PyObject * buffered_simple_flush(buffered *self, PyObject *args) { CHECK_INITIALIZED(self) - return PyObject_CallMethodNoArgs(self->raw, _PyIO_str_flush); + return PyObject_CallMethodNoArgs(self->raw, &_Py_ID(flush)); } static int @@ -452,7 +452,7 @@ buffered_closed(buffered *self) int closed; PyObject *res; CHECK_INITIALIZED_INT(self) - res = PyObject_GetAttr(self->raw, _PyIO_str_closed); + res = PyObject_GetAttr(self->raw, &_Py_ID(closed)); if (res == NULL) return -1; closed = PyObject_IsTrue(res); @@ -464,7 +464,7 @@ static PyObject * buffered_closed_get(buffered *self, void *context) { CHECK_INITIALIZED(self) - return PyObject_GetAttr(self->raw, _PyIO_str_closed); + return PyObject_GetAttr(self->raw, &_Py_ID(closed)); } static PyObject * @@ -495,7 +495,7 @@ buffered_close(buffered *self, PyObject *args) } /* flush() will most probably re-take the lock, so drop it first */ LEAVE_BUFFERED(self) - res = PyObject_CallMethodNoArgs((PyObject *)self, _PyIO_str_flush); + res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(flush)); if (!ENTER_BUFFERED(self)) return NULL; if (res == NULL) @@ -503,7 +503,7 @@ buffered_close(buffered *self, PyObject *args) else Py_DECREF(res); - res = PyObject_CallMethodNoArgs(self->raw, _PyIO_str_close); + res = PyObject_CallMethodNoArgs(self->raw, &_Py_ID(close)); if (self->buffer) { PyMem_Free(self->buffer); @@ -530,7 +530,7 @@ buffered_detach(buffered *self, PyObject *Py_UNUSED(ignored)) { PyObject *raw, *res; CHECK_INITIALIZED(self) - res = PyObject_CallMethodNoArgs((PyObject *)self, _PyIO_str_flush); + res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(flush)); if (res == NULL) return NULL; Py_DECREF(res); @@ -547,21 +547,21 @@ static PyObject * buffered_seekable(buffered *self, PyObject *Py_UNUSED(ignored)) { CHECK_INITIALIZED(self) - return PyObject_CallMethodNoArgs(self->raw, _PyIO_str_seekable); + return PyObject_CallMethodNoArgs(self->raw, &_Py_ID(seekable)); } static PyObject * buffered_readable(buffered *self, PyObject *Py_UNUSED(ignored)) { CHECK_INITIALIZED(self) - return PyObject_CallMethodNoArgs(self->raw, _PyIO_str_readable); + return PyObject_CallMethodNoArgs(self->raw, &_Py_ID(readable)); } static PyObject * buffered_writable(buffered *self, PyObject *Py_UNUSED(ignored)) { CHECK_INITIALIZED(self) - return PyObject_CallMethodNoArgs(self->raw, _PyIO_str_writable); + return PyObject_CallMethodNoArgs(self->raw, &_Py_ID(writable)); } static PyObject * @@ -584,14 +584,14 @@ static PyObject * buffered_fileno(buffered *self, PyObject *Py_UNUSED(ignored)) { CHECK_INITIALIZED(self) - return PyObject_CallMethodNoArgs(self->raw, _PyIO_str_fileno); + return PyObject_CallMethodNoArgs(self->raw, &_Py_ID(fileno)); } static PyObject * buffered_isatty(buffered *self, PyObject *Py_UNUSED(ignored)) { CHECK_INITIALIZED(self) - return PyObject_CallMethodNoArgs(self->raw, _PyIO_str_isatty); + return PyObject_CallMethodNoArgs(self->raw, &_Py_ID(isatty)); } /* Forward decls */ @@ -655,7 +655,7 @@ _buffered_raw_tell(buffered *self) { Py_off_t n; PyObject *res; - res = PyObject_CallMethodNoArgs(self->raw, _PyIO_str_tell); + res = PyObject_CallMethodNoArgs(self->raw, &_Py_ID(tell)); if (res == NULL) return -1; n = PyNumber_AsOff_t(res, PyExc_ValueError); @@ -685,7 +685,7 @@ _buffered_raw_seek(buffered *self, Py_off_t target, int whence) Py_DECREF(posobj); return -1; } - res = PyObject_CallMethodObjArgs(self->raw, _PyIO_str_seek, + res = PyObject_CallMethodObjArgs(self->raw, &_Py_ID(seek), posobj, whenceobj, NULL); Py_DECREF(posobj); Py_DECREF(whenceobj); @@ -1152,7 +1152,7 @@ _buffered_readline(buffered *self, Py_ssize_t limit) Py_CLEAR(res); goto end; } - Py_XSETREF(res, _PyBytes_Join(_PyIO_empty_bytes, chunks)); + Py_XSETREF(res, _PyBytes_Join((PyObject *)&_Py_SINGLETON(bytes_empty), chunks)); end: LEAVE_BUFFERED(self) @@ -1311,7 +1311,7 @@ _io__Buffered_truncate_impl(buffered *self, PyObject *pos) } Py_CLEAR(res); - res = PyObject_CallMethodOneArg(self->raw, _PyIO_str_truncate, pos); + res = PyObject_CallMethodOneArg(self->raw, &_Py_ID(truncate), pos); if (res == NULL) goto end; /* Reset cached position */ @@ -1339,7 +1339,7 @@ buffered_iternext(buffered *self) } else { line = PyObject_CallMethodNoArgs((PyObject *)self, - _PyIO_str_readline); + &_Py_ID(readline)); if (line && !PyBytes_Check(line)) { PyErr_Format(PyExc_OSError, "readline() should have returned a bytes object, " @@ -1457,7 +1457,7 @@ _bufferedreader_raw_read(buffered *self, char *start, Py_ssize_t len) raised (see issue #10956). */ do { - res = PyObject_CallMethodOneArg(self->raw, _PyIO_str_readinto, memobj); + res = PyObject_CallMethodOneArg(self->raw, &_Py_ID(readinto), memobj); } while (res == NULL && _PyIO_trap_eintr()); Py_DECREF(memobj); if (res == NULL) @@ -1530,7 +1530,7 @@ _bufferedreader_read_all(buffered *self) } _bufferedreader_reset_buf(self); - if (_PyObject_LookupAttr(self->raw, _PyIO_str_readall, &readall) < 0) { + if (_PyObject_LookupAttr(self->raw, &_Py_ID(readall), &readall) < 0) { goto cleanup; } if (readall) { @@ -1565,7 +1565,7 @@ _bufferedreader_read_all(buffered *self) } /* Read until EOF or until read() would block. */ - data = PyObject_CallMethodNoArgs(self->raw, _PyIO_str_read); + data = PyObject_CallMethodNoArgs(self->raw, &_Py_ID(read)); if (data == NULL) goto cleanup; if (data != Py_None && !PyBytes_Check(data)) { @@ -1578,7 +1578,7 @@ _bufferedreader_read_all(buffered *self) goto cleanup; } else { - tmp = _PyBytes_Join(_PyIO_empty_bytes, chunks); + tmp = _PyBytes_Join((PyObject *)&_Py_SINGLETON(bytes_empty), chunks); res = tmp; goto cleanup; } @@ -1814,7 +1814,7 @@ _bufferedwriter_raw_write(buffered *self, char *start, Py_ssize_t len) */ do { errno = 0; - res = PyObject_CallMethodOneArg(self->raw, _PyIO_str_write, memobj); + res = PyObject_CallMethodOneArg(self->raw, &_Py_ID(write), memobj); errnum = errno; } while (res == NULL && _PyIO_trap_eintr()); Py_DECREF(memobj); @@ -2251,7 +2251,7 @@ bufferedrwpair_closed_get(rwpair *self, void *context) "the BufferedRWPair object is being garbage-collected"); return NULL; } - return PyObject_GetAttr((PyObject *) self->writer, _PyIO_str_closed); + return PyObject_GetAttr((PyObject *) self->writer, &_Py_ID(closed)); } diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c index 74663e007b1e5..b00b6b983ee0b 100644 --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -181,7 +181,7 @@ iobase_check_closed(PyObject *self) int closed; /* This gets the derived attribute, which is *not* __IOBase_closed in most cases! */ - closed = _PyObject_LookupAttr(self, _PyIO_str_closed, &res); + closed = _PyObject_LookupAttr(self, &_Py_ID(closed), &res); if (closed > 0) { closed = PyObject_IsTrue(res); Py_DECREF(res); @@ -231,7 +231,7 @@ _io__IOBase_close_impl(PyObject *self) Py_RETURN_NONE; } - res = PyObject_CallMethodNoArgs(self, _PyIO_str_flush); + res = PyObject_CallMethodNoArgs(self, &_Py_ID(flush)); PyErr_Fetch(&exc, &val, &tb); rc = PyObject_SetAttr(self, &_Py_ID(__IOBase_closed), Py_True); @@ -261,7 +261,7 @@ iobase_finalize(PyObject *self) /* If `closed` doesn't exist or can't be evaluated as bool, then the object is probably in an unusable state, so ignore. */ - if (_PyObject_LookupAttr(self, _PyIO_str_closed, &res) <= 0) { + if (_PyObject_LookupAttr(self, &_Py_ID(closed), &res) <= 0) { PyErr_Clear(); closed = -1; } @@ -276,7 +276,7 @@ iobase_finalize(PyObject *self) finalization process. */ if (PyObject_SetAttr(self, &_Py_ID(_finalizing), Py_True)) PyErr_Clear(); - res = PyObject_CallMethodNoArgs((PyObject *)self, _PyIO_str_close); + res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(close)); /* Silencing I/O errors is bad, but printing spurious tracebacks is equally as bad, and potentially more frequent (because of shutdown issues). */ @@ -377,7 +377,7 @@ _io__IOBase_seekable_impl(PyObject *self) PyObject * _PyIOBase_check_seekable(PyObject *self, PyObject *args) { - PyObject *res = PyObject_CallMethodNoArgs(self, _PyIO_str_seekable); + PyObject *res = PyObject_CallMethodNoArgs(self, &_Py_ID(seekable)); if (res == NULL) return NULL; if (res != Py_True) { @@ -410,7 +410,7 @@ _io__IOBase_readable_impl(PyObject *self) PyObject * _PyIOBase_check_readable(PyObject *self, PyObject *args) { - PyObject *res = PyObject_CallMethodNoArgs(self, _PyIO_str_readable); + PyObject *res = PyObject_CallMethodNoArgs(self, &_Py_ID(readable)); if (res == NULL) return NULL; if (res != Py_True) { @@ -443,7 +443,7 @@ _io__IOBase_writable_impl(PyObject *self) PyObject * _PyIOBase_check_writable(PyObject *self, PyObject *args) { - PyObject *res = PyObject_CallMethodNoArgs(self, _PyIO_str_writable); + PyObject *res = PyObject_CallMethodNoArgs(self, &_Py_ID(writable)); if (res == NULL) return NULL; if (res != Py_True) { @@ -472,7 +472,7 @@ iobase_enter(PyObject *self, PyObject *args) static PyObject * iobase_exit(PyObject *self, PyObject *args) { - return PyObject_CallMethodNoArgs(self, _PyIO_str_close); + return PyObject_CallMethodNoArgs(self, &_Py_ID(close)); } /* Lower-level APIs */ @@ -536,7 +536,7 @@ _io__IOBase_readline_impl(PyObject *self, Py_ssize_t limit) PyObject *peek, *buffer, *result; Py_ssize_t old_size = -1; - if (_PyObject_LookupAttr(self, _PyIO_str_peek, &peek) < 0) { + if (_PyObject_LookupAttr(self, &_Py_ID(peek), &peek) < 0) { return NULL; } @@ -650,7 +650,7 @@ iobase_iter(PyObject *self) static PyObject * iobase_iternext(PyObject *self) { - PyObject *line = PyObject_CallMethodNoArgs(self, _PyIO_str_readline); + PyObject *line = PyObject_CallMethodNoArgs(self, &_Py_ID(readline)); if (line == NULL) return NULL; @@ -776,7 +776,7 @@ _io__IOBase_writelines(PyObject *self, PyObject *lines) res = NULL; do { - res = PyObject_CallMethodObjArgs(self, _PyIO_str_write, line, NULL); + res = PyObject_CallMethodObjArgs(self, &_Py_ID(write), line, NULL); } while (res == NULL && _PyIO_trap_eintr()); Py_DECREF(line); if (res == NULL) { @@ -920,7 +920,7 @@ _io__RawIOBase_read_impl(PyObject *self, Py_ssize_t n) if (b == NULL) return NULL; - res = PyObject_CallMethodObjArgs(self, _PyIO_str_readinto, b, NULL); + res = PyObject_CallMethodObjArgs(self, &_Py_ID(readinto), b, NULL); if (res == NULL || res == Py_None) { Py_DECREF(b); return res; @@ -994,7 +994,7 @@ _io__RawIOBase_readall_impl(PyObject *self) return NULL; } } - result = _PyBytes_Join(_PyIO_empty_bytes, chunks); + result = _PyBytes_Join((PyObject *)&_Py_SINGLETON(bytes_empty), chunks); Py_DECREF(chunks); return result; } diff --git a/Modules/_io/stringio.c b/Modules/_io/stringio.c index e76152e617bdc..3fe02d35924f8 100644 --- a/Modules/_io/stringio.c +++ b/Modules/_io/stringio.c @@ -192,7 +192,7 @@ write_str(stringio *self, PyObject *obj) } if (self->writenl) { PyObject *translated = PyUnicode_Replace( - decoded, _PyIO_str_nl, self->writenl, -1); + decoded, &_Py_STR(newline), self->writenl, -1); Py_DECREF(decoded); decoded = translated; } @@ -409,7 +409,7 @@ stringio_iternext(stringio *self) else { /* XXX is subclassing StringIO really supported? */ line = PyObject_CallMethodNoArgs((PyObject *)self, - _PyIO_str_readline); + &_Py_ID(readline)); if (line && !PyUnicode_Check(line)) { PyErr_Format(PyExc_OSError, "readline() should have returned a str object, " @@ -964,7 +964,7 @@ stringio_newlines(stringio *self, void *context) CHECK_CLOSED(self); if (self->decoder == NULL) Py_RETURN_NONE; - return PyObject_GetAttr(self->decoder, _PyIO_str_newlines); + return PyObject_GetAttr(self->decoder, &_Py_ID(newlines)); } #include "clinic/stringio.c.h" diff --git a/Modules/_io/textio.c b/Modules/_io/textio.c index d5ceec7981f3a..ca59a4ef27135 100644 --- a/Modules/_io/textio.c +++ b/Modules/_io/textio.c @@ -298,7 +298,7 @@ _PyIncrementalNewlineDecoder_decode(PyObject *myself, /* decode input (with the eventual \r from a previous pass) */ if (self->decoder != Py_None) { output = PyObject_CallMethodObjArgs(self->decoder, - _PyIO_str_decode, input, final ? Py_True : Py_False, NULL); + &_Py_ID(decode), input, final ? Py_True : Py_False, NULL); } else { output = input; @@ -509,7 +509,7 @@ _io_IncrementalNewlineDecoder_getstate_impl(nldecoder_object *self) if (self->decoder != Py_None) { PyObject *state = PyObject_CallMethodNoArgs(self->decoder, - _PyIO_str_getstate); + &_Py_ID(getstate)); if (state == NULL) return NULL; if (!PyTuple_Check(state)) { @@ -584,7 +584,7 @@ _io_IncrementalNewlineDecoder_reset_impl(nldecoder_object *self) self->seennl = 0; self->pendingcr = 0; if (self->decoder != Py_None) - return PyObject_CallMethodNoArgs(self->decoder, _PyIO_str_reset); + return PyObject_CallMethodNoArgs(self->decoder, &_Py_ID(reset)); else Py_RETURN_NONE; } @@ -883,7 +883,7 @@ _textiowrapper_decode(PyObject *decoder, PyObject *bytes, int eof) if (Py_IS_TYPE(decoder, &PyIncrementalNewlineDecoder_Type)) chars = _PyIncrementalNewlineDecoder_decode(decoder, bytes, eof); else - chars = PyObject_CallMethodObjArgs(decoder, _PyIO_str_decode, bytes, + chars = PyObject_CallMethodObjArgs(decoder, &_Py_ID(decode), bytes, eof ? Py_True : Py_False, NULL); if (check_decoded(chars) < 0) @@ -947,7 +947,7 @@ _textiowrapper_fix_encoder_state(textio *self) self->encoding_start_of_stream = 1; PyObject *cookieObj = PyObject_CallMethodNoArgs( - self->buffer, _PyIO_str_tell); + self->buffer, &_Py_ID(tell)); if (cookieObj == NULL) { return -1; } @@ -961,7 +961,7 @@ _textiowrapper_fix_encoder_state(textio *self) if (cmp == 0) { self->encoding_start_of_stream = 0; PyObject *res = PyObject_CallMethodOneArg( - self->encoder, _PyIO_str_setstate, _PyLong_GetZero()); + self->encoder, &_Py_ID(setstate), _PyLong_GetZero()); if (res == NULL) { return -1; } @@ -1225,7 +1225,7 @@ _io_TextIOWrapper___init___impl(textio *self, PyObject *buffer, goto error; self->seekable = self->telling = r; - r = _PyObject_LookupAttr(buffer, _PyIO_str_read1, &res); + r = _PyObject_LookupAttr(buffer, &_Py_ID(read1), &res); if (r < 0) { goto error; } @@ -1358,7 +1358,7 @@ _io_TextIOWrapper_reconfigure_impl(textio *self, PyObject *encoding, return NULL; } - PyObject *res = PyObject_CallMethodNoArgs((PyObject *)self, _PyIO_str_flush); + PyObject *res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(flush)); if (res == NULL) { return NULL; } @@ -1497,7 +1497,7 @@ _io_TextIOWrapper_detach_impl(textio *self) { PyObject *buffer, *res; CHECK_ATTACHED(self); - res = PyObject_CallMethodNoArgs((PyObject *)self, _PyIO_str_flush); + res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(flush)); if (res == NULL) return NULL; Py_DECREF(res); @@ -1569,7 +1569,7 @@ _textiowrapper_writeflush(textio *self) PyObject *ret; do { - ret = PyObject_CallMethodOneArg(self->buffer, _PyIO_str_write, b); + ret = PyObject_CallMethodOneArg(self->buffer, &_Py_ID(write), b); } while (ret == NULL && _PyIO_trap_eintr()); Py_DECREF(b); // NOTE: We cleared buffer but we don't know how many bytes are actually written @@ -1644,7 +1644,7 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) self->encoding_start_of_stream = 0; } else { - b = PyObject_CallMethodOneArg(self->encoder, _PyIO_str_encode, text); + b = PyObject_CallMethodOneArg(self->encoder, &_Py_ID(encode), text); } Py_DECREF(text); @@ -1704,7 +1704,7 @@ _io_TextIOWrapper_write_impl(textio *self, PyObject *text) } if (needflush) { - ret = PyObject_CallMethodNoArgs(self->buffer, _PyIO_str_flush); + ret = PyObject_CallMethodNoArgs(self->buffer, &_Py_ID(flush)); if (ret == NULL) return NULL; Py_DECREF(ret); @@ -1795,7 +1795,7 @@ textiowrapper_read_chunk(textio *self, Py_ssize_t size_hint) * where the decoder's input buffer is empty. */ PyObject *state = PyObject_CallMethodNoArgs(self->decoder, - _PyIO_str_getstate); + &_Py_ID(getstate)); if (state == NULL) return -1; /* Given this, we know there was a valid snapshot point @@ -1836,7 +1836,7 @@ textiowrapper_read_chunk(textio *self, Py_ssize_t size_hint) goto fail; input_chunk = PyObject_CallMethodOneArg(self->buffer, - (self->has_read1 ? _PyIO_str_read1: _PyIO_str_read), + (self->has_read1 ? &_Py_ID(read1): &_Py_ID(read)), chunk_size); Py_DECREF(chunk_size); if (input_chunk == NULL) @@ -1928,7 +1928,7 @@ _io_TextIOWrapper_read_impl(textio *self, Py_ssize_t n) bytes, 1); else decoded = PyObject_CallMethodObjArgs( - self->decoder, _PyIO_str_decode, bytes, Py_True, NULL); + self->decoder, &_Py_ID(decode), bytes, Py_True, NULL); Py_DECREF(bytes); if (check_decoded(decoded) < 0) goto fail; @@ -1989,7 +1989,7 @@ _io_TextIOWrapper_read_impl(textio *self, Py_ssize_t n) if (chunks != NULL) { if (result != NULL && PyList_Append(chunks, result) < 0) goto fail; - Py_XSETREF(result, PyUnicode_Join(_PyIO_empty_str, chunks)); + Py_XSETREF(result, PyUnicode_Join(&_Py_STR(empty), chunks)); if (result == NULL) goto fail; Py_CLEAR(chunks); @@ -2254,14 +2254,13 @@ _textiowrapper_readline(textio *self, Py_ssize_t limit) goto error; Py_DECREF(line); } - line = PyUnicode_Join(_PyIO_empty_str, chunks); + line = PyUnicode_Join(&_Py_STR(empty), chunks); if (line == NULL) goto error; Py_CLEAR(chunks); } if (line == NULL) { - Py_INCREF(_PyIO_empty_str); - line = _PyIO_empty_str; + line = &_Py_STR(empty); } return line; @@ -2379,7 +2378,7 @@ _textiowrapper_decoder_setstate(textio *self, cookie_type *cookie) utf-16, that we are expecting a BOM). */ if (cookie->start_pos == 0 && cookie->dec_flags == 0) { - res = PyObject_CallMethodNoArgs(self->decoder, _PyIO_str_reset); + res = PyObject_CallMethodNoArgs(self->decoder, &_Py_ID(reset)); } else { res = _PyObject_CallMethod(self->decoder, &_Py_ID(setstate), @@ -2397,11 +2396,11 @@ _textiowrapper_encoder_reset(textio *self, int start_of_stream) { PyObject *res; if (start_of_stream) { - res = PyObject_CallMethodNoArgs(self->encoder, _PyIO_str_reset); + res = PyObject_CallMethodNoArgs(self->encoder, &_Py_ID(reset)); self->encoding_start_of_stream = 1; } else { - res = PyObject_CallMethodOneArg(self->encoder, _PyIO_str_setstate, + res = PyObject_CallMethodOneArg(self->encoder, &_Py_ID(setstate), _PyLong_GetZero()); self->encoding_start_of_stream = 0; } @@ -2528,7 +2527,7 @@ _io_TextIOWrapper_seek_impl(textio *self, PyObject *cookieObj, int whence) goto fail; } - res = PyObject_CallMethodNoArgs((PyObject *)self, _PyIO_str_flush); + res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(flush)); if (res == NULL) goto fail; Py_DECREF(res); @@ -2543,7 +2542,7 @@ _io_TextIOWrapper_seek_impl(textio *self, PyObject *cookieObj, int whence) posobj = PyLong_FromOff_t(cookie.start_pos); if (posobj == NULL) goto fail; - res = PyObject_CallMethodOneArg(self->buffer, _PyIO_str_seek, posobj); + res = PyObject_CallMethodOneArg(self->buffer, &_Py_ID(seek), posobj); Py_DECREF(posobj); if (res == NULL) goto fail; @@ -2692,14 +2691,14 @@ _io_TextIOWrapper_tell_impl(textio *self) /* Decoder state will be restored at the end */ saved_state = PyObject_CallMethodNoArgs(self->decoder, - _PyIO_str_getstate); + &_Py_ID(getstate)); if (saved_state == NULL) goto fail; #define DECODER_GETSTATE() do { \ PyObject *dec_buffer; \ PyObject *_state = PyObject_CallMethodNoArgs(self->decoder, \ - _PyIO_str_getstate); \ + &_Py_ID(getstate)); \ if (_state == NULL) \ goto fail; \ if (!PyTuple_Check(_state)) { \ @@ -2863,12 +2862,12 @@ _io_TextIOWrapper_truncate_impl(textio *self, PyObject *pos) CHECK_ATTACHED(self) - res = PyObject_CallMethodNoArgs((PyObject *)self, _PyIO_str_flush); + res = PyObject_CallMethodNoArgs((PyObject *)self, &_Py_ID(flush)); if (res == NULL) return NULL; Py_DECREF(res); - return PyObject_CallMethodOneArg(self->buffer, _PyIO_str_truncate, pos); + return PyObject_CallMethodOneArg(self->buffer, &_Py_ID(truncate), pos); } static PyObject * @@ -3077,7 +3076,7 @@ textiowrapper_iternext(textio *self) } else { line = PyObject_CallMethodNoArgs((PyObject *)self, - _PyIO_str_readline); + &_Py_ID(readline)); if (line && !PyUnicode_Check(line)) { PyErr_Format(PyExc_OSError, "readline() should have returned a str object, " @@ -3112,7 +3111,7 @@ static PyObject * textiowrapper_closed_get(textio *self, void *context) { CHECK_ATTACHED(self); - return PyObject_GetAttr(self->buffer, _PyIO_str_closed); + return PyObject_GetAttr(self->buffer, &_Py_ID(closed)); } static PyObject * @@ -3121,7 +3120,7 @@ textiowrapper_newlines_get(textio *self, void *context) PyObject *res; CHECK_ATTACHED(self); if (self->decoder == NULL || - _PyObject_LookupAttr(self->decoder, _PyIO_str_newlines, &res) == 0) + _PyObject_LookupAttr(self->decoder, &_Py_ID(newlines), &res) == 0) { Py_RETURN_NONE; } diff --git a/Modules/_tracemalloc.c b/Modules/_tracemalloc.c index 14bad00e4c629..53e329989abf2 100644 --- a/Modules/_tracemalloc.c +++ b/Modules/_tracemalloc.c @@ -3,6 +3,7 @@ #include "pycore_gc.h" // PyGC_Head #include "pycore_hashtable.h" // _Py_hashtable_t #include "pycore_pymem.h" // _Py_tracemalloc_config +#include "pycore_runtime.h" // _Py_ID() #include "pycore_traceback.h" #include @@ -15,6 +16,8 @@ module _tracemalloc [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=708a98302fc46e5f]*/ +_Py_DECLARE_STR(anon_unknown, ""); + /* Trace memory blocks allocated by PyMem_RawMalloc() */ #define TRACE_RAW_MALLOC @@ -91,7 +94,6 @@ typedef struct { static const unsigned long MAX_NFRAME = Py_MIN(UINT16_MAX, ((SIZE_MAX - sizeof(traceback_t)) / sizeof(frame_t) + 1)); -static PyObject *unknown_filename = NULL; static traceback_t tracemalloc_empty_traceback; /* Trace of a memory block */ @@ -305,7 +307,7 @@ hashtable_compare_traceback(const void *key1, const void *key2) static void tracemalloc_get_frame(InterpreterFrame *pyframe, frame_t *frame) { - frame->filename = unknown_filename; + frame->filename = &_Py_STR(anon_unknown); int lineno = PyCode_Addr2Line(pyframe->f_code, pyframe->f_lasti*sizeof(_Py_CODEUNIT)); if (lineno < 0) { lineno = 0; @@ -905,15 +907,10 @@ tracemalloc_init(void) return -1; } - unknown_filename = PyUnicode_FromString(""); - if (unknown_filename == NULL) - return -1; - PyUnicode_InternInPlace(&unknown_filename); - tracemalloc_empty_traceback.nframe = 1; tracemalloc_empty_traceback.total_nframe = 1; /* borrowed reference */ - tracemalloc_empty_traceback.frames[0].filename = unknown_filename; + tracemalloc_empty_traceback.frames[0].filename = &_Py_STR(anon_unknown); tracemalloc_empty_traceback.frames[0].lineno = 0; tracemalloc_empty_traceback.hash = traceback_hash(&tracemalloc_empty_traceback); @@ -947,8 +944,6 @@ tracemalloc_deinit(void) #ifdef REENTRANT_THREADLOCAL PyThread_tss_delete(&tracemalloc_reentrant_key); #endif - - Py_XDECREF(unknown_filename); } diff --git a/Objects/boolobject.c b/Objects/boolobject.c index 53f8192605797..d86958aff9ccb 100644 --- a/Objects/boolobject.c +++ b/Objects/boolobject.c @@ -1,26 +1,15 @@ /* Boolean type, a subtype of int */ #include "Python.h" +#include "pycore_runtime.h" // _Py_ID() #include "pycore_pyerrors.h" // _Py_FatalRefcountError() /* We define bool_repr to return "False" or "True" */ -static PyObject *false_str = NULL; -static PyObject *true_str = NULL; - static PyObject * bool_repr(PyObject *self) { - PyObject *s; - - if (self == Py_True) - s = true_str ? true_str : - (true_str = PyUnicode_InternFromString("True")); - else - s = false_str ? false_str : - (false_str = PyUnicode_InternFromString("False")); - Py_XINCREF(s); - return s; + return self == Py_True ? &_Py_ID(True) : &_Py_ID(False); } /* Function to return a bool from a C long */ diff --git a/Objects/classobject.c b/Objects/classobject.c index d7ccf31244e8b..3b1c25394f152 100644 --- a/Objects/classobject.c +++ b/Objects/classobject.c @@ -157,13 +157,7 @@ static PyMemberDef method_memberlist[] = { static PyObject * method_get_doc(PyMethodObject *im, void *context) { - static PyObject *docstr; - if (docstr == NULL) { - docstr= PyUnicode_InternFromString("__doc__"); - if (docstr == NULL) - return NULL; - } - return PyObject_GetAttr(im->im_func, docstr); + return PyObject_GetAttr(im->im_func, &_Py_ID(__doc__)); } static PyGetSetDef method_getset[] = { @@ -405,13 +399,8 @@ static PyMemberDef instancemethod_memberlist[] = { static PyObject * instancemethod_get_doc(PyObject *self, void *context) { - static PyObject *docstr; - if (docstr == NULL) { - docstr = PyUnicode_InternFromString("__doc__"); - if (docstr == NULL) - return NULL; - } - return PyObject_GetAttr(PyInstanceMethod_GET_FUNCTION(self), docstr); + return PyObject_GetAttr(PyInstanceMethod_GET_FUNCTION(self), + &_Py_ID(__doc__)); } static PyGetSetDef instancemethod_getset[] = { diff --git a/Objects/codeobject.c b/Objects/codeobject.c index 9a27f1295b189..f8ef1e6b40cc1 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -553,16 +553,11 @@ PyCode_New(int argcount, int kwonlyargcount, PyCodeObject * PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno) { - PyObject *emptystring = NULL; PyObject *nulltuple = NULL; PyObject *filename_ob = NULL; PyObject *funcname_ob = NULL; PyCodeObject *result = NULL; - emptystring = PyBytes_FromString(""); - if (emptystring == NULL) { - goto failed; - } nulltuple = PyTuple_New(0); if (nulltuple == NULL) { goto failed; @@ -576,6 +571,7 @@ PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno) goto failed; } +#define emptystring (PyObject *)&_Py_SINGLETON(bytes_empty) struct _PyCodeConstructor con = { .filename = filename_ob, .name = funcname_ob, @@ -594,7 +590,6 @@ PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno) result = _PyCode_New(&con); failed: - Py_XDECREF(emptystring); Py_XDECREF(nulltuple); Py_XDECREF(funcname_ob); Py_XDECREF(filename_ob); diff --git a/Objects/listobject.c b/Objects/listobject.c index 1ba1c1b053143..783ae88a17f3b 100644 --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -15,7 +15,7 @@ class list "PyListObject *" "&PyList_Type" #include "clinic/listobject.c.h" -static PyObject *indexerr = NULL; +_Py_DECLARE_STR(list_err, "list index out of range"); #if PyList_MAXFREELIST > 0 static struct _Py_list_state * @@ -125,10 +125,6 @@ _PyList_Fini(PyInterpreterState *interp) struct _Py_list_state *state = &interp->list; state->numfree = -1; #endif - - if (_Py_IsMainInterpreter(interp)) { - Py_CLEAR(indexerr); - } } /* Print summary info about the state of the optimized allocator */ @@ -238,13 +234,8 @@ PyList_GetItem(PyObject *op, Py_ssize_t i) return NULL; } if (!valid_index(i, Py_SIZE(op))) { - if (indexerr == NULL) { - indexerr = PyUnicode_FromString( - "list index out of range"); - if (indexerr == NULL) - return NULL; - } - PyErr_SetObject(PyExc_IndexError, indexerr); + _Py_DECLARE_STR(list_err, "list index out of range"); + PyErr_SetObject(PyExc_IndexError, &_Py_STR(list_err)); return NULL; } return ((PyListObject *)op) -> ob_item[i]; @@ -452,13 +443,7 @@ static PyObject * list_item(PyListObject *a, Py_ssize_t i) { if (!valid_index(i, Py_SIZE(a))) { - if (indexerr == NULL) { - indexerr = PyUnicode_FromString( - "list index out of range"); - if (indexerr == NULL) - return NULL; - } - PyErr_SetObject(PyExc_IndexError, indexerr); + PyErr_SetObject(PyExc_IndexError, &_Py_STR(list_err)); return NULL; } Py_INCREF(a->ob_item[i]); diff --git a/Python/_warnings.c b/Python/_warnings.c index 03e6ffcee0ac2..be962e76cd801 100644 --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -768,25 +768,6 @@ warn_explicit(PyThreadState *tstate, PyObject *category, PyObject *message, static int is_internal_frame(PyFrameObject *frame) { - static PyObject *importlib_string = NULL; - static PyObject *bootstrap_string = NULL; - int contains; - - if (importlib_string == NULL) { - importlib_string = PyUnicode_FromString("importlib"); - if (importlib_string == NULL) { - return 0; - } - - bootstrap_string = PyUnicode_FromString("_bootstrap"); - if (bootstrap_string == NULL) { - Py_DECREF(importlib_string); - return 0; - } - Py_INCREF(importlib_string); - Py_INCREF(bootstrap_string); - } - if (frame == NULL) { return 0; } @@ -802,12 +783,12 @@ is_internal_frame(PyFrameObject *frame) return 0; } - contains = PyUnicode_Contains(filename, importlib_string); + int contains = PyUnicode_Contains(filename, &_Py_ID(importlib)); if (contains < 0) { return 0; } else if (contains > 0) { - contains = PyUnicode_Contains(filename, bootstrap_string); + contains = PyUnicode_Contains(filename, &_Py_ID(_bootstrap)); if (contains < 0) { return 0; } diff --git a/Python/ast_unparse.c b/Python/ast_unparse.c index 126e9047d58d6..f148e99865259 100644 --- a/Python/ast_unparse.c +++ b/Python/ast_unparse.c @@ -1,5 +1,6 @@ #include "Python.h" #include "pycore_ast.h" // expr_ty +#include "pycore_runtime.h" // _Py_ID() #include // DBL_MAX_10_EXP #include @@ -8,11 +9,10 @@ * See ast.unparse for a full unparser (written in Python) */ -static PyObject *_str_open_br; -static PyObject *_str_dbl_open_br; -static PyObject *_str_close_br; -static PyObject *_str_dbl_close_br; -static PyObject *_str_inf; +_Py_DECLARE_STR(open_br, "{"); +_Py_DECLARE_STR(dbl_open_br, "{{"); +_Py_DECLARE_STR(close_br, "}"); +_Py_DECLARE_STR(dbl_close_br, "}}"); static PyObject *_str_replace_inf; /* Forward declarations for recursion via helper functions. */ @@ -80,7 +80,7 @@ append_repr(_PyUnicodeWriter *writer, PyObject *obj) { PyObject *new_repr = PyUnicode_Replace( repr, - _str_inf, + &_Py_ID(inf), _str_replace_inf, -1 ); @@ -575,11 +575,11 @@ escape_braces(PyObject *orig) { PyObject *temp; PyObject *result; - temp = PyUnicode_Replace(orig, _str_open_br, _str_dbl_open_br, -1); + temp = PyUnicode_Replace(orig, &_Py_STR(open_br), &_Py_STR(dbl_open_br), -1); if (!temp) { return NULL; } - result = PyUnicode_Replace(temp, _str_close_br, _str_dbl_close_br, -1); + result = PyUnicode_Replace(temp, &_Py_STR(close_br), &_Py_STR(dbl_close_br), -1); Py_DECREF(temp); return result; } @@ -673,7 +673,7 @@ append_formattedvalue(_PyUnicodeWriter *writer, expr_ty e) if (!temp_fv_str) { return -1; } - if (PyUnicode_Find(temp_fv_str, _str_open_br, 0, 1, 1) == 0) { + if (PyUnicode_Find(temp_fv_str, &_Py_STR(open_br), 0, 1, 1) == 0) { /* Expression starts with a brace, split it with a space from the outer one. */ outer_brace = "{ "; @@ -927,26 +927,6 @@ append_ast_expr(_PyUnicodeWriter *writer, expr_ty e, int level) static int maybe_init_static_strings(void) { - if (!_str_open_br && - !(_str_open_br = PyUnicode_InternFromString("{"))) { - return -1; - } - if (!_str_dbl_open_br && - !(_str_dbl_open_br = PyUnicode_InternFromString("{{"))) { - return -1; - } - if (!_str_close_br && - !(_str_close_br = PyUnicode_InternFromString("}"))) { - return -1; - } - if (!_str_dbl_close_br && - !(_str_dbl_close_br = PyUnicode_InternFromString("}}"))) { - return -1; - } - if (!_str_inf && - !(_str_inf = PyUnicode_FromString("inf"))) { - return -1; - } if (!_str_replace_inf && !(_str_replace_inf = PyUnicode_FromFormat("1e%d", 1 + DBL_MAX_10_EXP))) { return -1; diff --git a/Python/compile.c b/Python/compile.c index 7f0a6f096d723..cb6e5e3550fdf 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -2522,7 +2522,6 @@ static int compiler_class(struct compiler *c, stmt_ty s) { PyCodeObject *co; - PyObject *str; int i, firstlineno; asdl_expr_seq *decos = s->v.ClassDef.decorator_list; @@ -2557,30 +2556,21 @@ compiler_class(struct compiler *c, stmt_ty s) Py_INCREF(s->v.ClassDef.name); Py_XSETREF(c->u->u_private, s->v.ClassDef.name); /* load (global) __name__ ... */ - str = PyUnicode_InternFromString("__name__"); - if (!str || !compiler_nameop(c, str, Load)) { - Py_XDECREF(str); + if (!compiler_nameop(c, &_Py_ID(__name__), Load)) { compiler_exit_scope(c); return 0; } - Py_DECREF(str); /* ... and store it as __module__ */ - str = PyUnicode_InternFromString("__module__"); - if (!str || !compiler_nameop(c, str, Store)) { - Py_XDECREF(str); + if (!compiler_nameop(c, &_Py_ID(__module__), Store)) { compiler_exit_scope(c); return 0; } - Py_DECREF(str); assert(c->u->u_qualname); ADDOP_LOAD_CONST(c, c->u->u_qualname); - str = PyUnicode_InternFromString("__qualname__"); - if (!str || !compiler_nameop(c, str, Store)) { - Py_XDECREF(str); + if (!compiler_nameop(c, &_Py_ID(__qualname__), Store)) { compiler_exit_scope(c); return 0; } - Py_DECREF(str); /* compile the body proper */ if (!compiler_body(c, s->v.ClassDef.body)) { compiler_exit_scope(c); @@ -2591,13 +2581,7 @@ compiler_class(struct compiler *c, stmt_ty s) /* Return __classcell__ if it is referenced, otherwise return None */ if (c->u->u_ste->ste_needs_class_closure) { /* Store __classcell__ into class namespace & return it */ - str = PyUnicode_InternFromString("__class__"); - if (str == NULL) { - compiler_exit_scope(c); - return 0; - } - i = compiler_lookup_arg(c->u->u_cellvars, str); - Py_DECREF(str); + i = compiler_lookup_arg(c->u->u_cellvars, &_Py_ID(__class__)); if (i < 0) { compiler_exit_scope(c); return 0; @@ -2606,13 +2590,10 @@ compiler_class(struct compiler *c, stmt_ty s) ADDOP_I(c, LOAD_CLOSURE, i); ADDOP_I(c, COPY, 1); - str = PyUnicode_InternFromString("__classcell__"); - if (!str || !compiler_nameop(c, str, Store)) { - Py_XDECREF(str); + if (!compiler_nameop(c, &_Py_ID(__classcell__), Store)) { compiler_exit_scope(c); return 0; } - Py_DECREF(str); } else { /* No methods referenced __class__, so just return None */ @@ -4741,13 +4722,8 @@ compiler_joined_str(struct compiler *c, expr_ty e) Py_ssize_t value_count = asdl_seq_LEN(e->v.JoinedStr.values); if (value_count > STACK_USE_GUIDELINE) { - ADDOP_LOAD_CONST_NEW(c, _PyUnicode_FromASCII("", 0)); - PyObject *join = _PyUnicode_FromASCII("join", 4); - if (join == NULL) { - return 0; - } - ADDOP_NAME(c, LOAD_METHOD, join, names); - Py_DECREF(join); + ADDOP_LOAD_CONST_NEW(c, &_Py_STR(empty)); + ADDOP_NAME(c, LOAD_METHOD, &_Py_ID(join), names); ADDOP_I(c, BUILD_LIST, 0); for (Py_ssize_t i = 0; i < asdl_seq_LEN(e->v.JoinedStr.values); i++) { VISIT(c, expr, asdl_seq_GET(e->v.JoinedStr.values, i)); diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 4a3a1abb3a4f0..b6310c9aeb808 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -24,7 +24,6 @@ #include "pycore_runtime.h" // _Py_ID() #include "pycore_runtime_init.h" // _PyRuntimeState_INIT #include "pycore_sliceobject.h" // _PySlice_Fini() -#include "pycore_symtable.h" // _PySymtable_Fini() #include "pycore_sysmodule.h" // _PySys_ClearAuditHooks() #include "pycore_traceback.h" // _Py_DumpTracebackThreads() #include "pycore_tuple.h" // _PyTuple_InitTypes() @@ -1690,9 +1689,6 @@ finalize_interp_clear(PyThreadState *tstate) int is_main_interp = _Py_IsMainInterpreter(tstate->interp); _PyExc_ClearExceptionGroupType(tstate->interp); - if (is_main_interp) { - _PySymtable_Fini(); - } /* Clear interpreter state and all thread states */ _PyInterpreterState_Clear(tstate); diff --git a/Python/symtable.c b/Python/symtable.c index e9bdff3eba109..0b259b08b61f9 100644 --- a/Python/symtable.c +++ b/Python/symtable.c @@ -230,13 +230,6 @@ static int symtable_raise_if_annotation_block(struct symtable *st, const char *, static int symtable_raise_if_comprehension_block(struct symtable *st, expr_ty); -static identifier top = NULL, lambda = NULL, genexpr = NULL, - listcomp = NULL, setcomp = NULL, dictcomp = NULL, - __class__ = NULL, _annotation = NULL; - -#define GET_IDENTIFIER(VAR) \ - ((VAR) ? (VAR) : ((VAR) = PyUnicode_InternFromString(# VAR))) - #define DUPLICATE_ARGUMENT \ "duplicate argument '%U' in function definition" @@ -313,8 +306,7 @@ _PySymtable_Build(mod_ty mod, PyObject *filename, PyFutureFeatures *future) recursion_limit * COMPILER_STACK_FRAME_SCALE : recursion_limit; /* Make the initial symbol information gathering pass */ - if (!GET_IDENTIFIER(top) || - !symtable_enter_block(st, top, ModuleBlock, (void *)mod, 0, 0, 0, 0)) { + if (!symtable_enter_block(st, &_Py_ID(top), ModuleBlock, (void *)mod, 0, 0, 0, 0)) { _PySymtable_Free(st); return NULL; } @@ -619,9 +611,7 @@ static int drop_class_free(PySTEntryObject *ste, PyObject *free) { int res; - if (!GET_IDENTIFIER(__class__)) - return 0; - res = PySet_Discard(free, __class__); + res = PySet_Discard(free, &_Py_ID(__class__)); if (res < 0) return 0; if (res) @@ -834,9 +824,7 @@ analyze_block(PySTEntryObject *ste, PyObject *bound, PyObject *free, } else { /* Special-case __class__ */ - if (!GET_IDENTIFIER(__class__)) - goto error; - if (PySet_Add(newbound, __class__) < 0) + if (PySet_Add(newbound, &_Py_ID(__class__)) < 0) goto error; } @@ -1610,13 +1598,11 @@ symtable_visit_expr(struct symtable *st, expr_ty e) VISIT(st, expr, e->v.UnaryOp.operand); break; case Lambda_kind: { - if (!GET_IDENTIFIER(lambda)) - VISIT_QUIT(st, 0); if (e->v.Lambda.args->defaults) VISIT_SEQ(st, expr, e->v.Lambda.args->defaults); if (e->v.Lambda.args->kw_defaults) VISIT_SEQ_WITH_NULL(st, expr, e->v.Lambda.args->kw_defaults); - if (!symtable_enter_block(st, lambda, + if (!symtable_enter_block(st, &_Py_ID(lambda), FunctionBlock, (void *)e, e->lineno, e->col_offset, e->end_lineno, e->end_col_offset)) @@ -1730,8 +1716,7 @@ symtable_visit_expr(struct symtable *st, expr_ty e) if (e->v.Name.ctx == Load && st->st_cur->ste_type == FunctionBlock && _PyUnicode_EqualToASCIIString(e->v.Name.id, "super")) { - if (!GET_IDENTIFIER(__class__) || - !symtable_add_def(st, __class__, USE, LOCATION(e))) + if (!symtable_add_def(st, &_Py_ID(__class__), USE, LOCATION(e))) VISIT_QUIT(st, 0); } break; @@ -1832,7 +1817,7 @@ symtable_visit_annotation(struct symtable *st, expr_ty annotation) { int future_annotations = st->st_future->ff_features & CO_FUTURE_ANNOTATIONS; if (future_annotations && - !symtable_enter_block(st, GET_IDENTIFIER(_annotation), AnnotationBlock, + !symtable_enter_block(st, &_Py_ID(_annotation), AnnotationBlock, (void *)annotation, annotation->lineno, annotation->col_offset, annotation->end_lineno, annotation->end_col_offset)) { @@ -1867,7 +1852,7 @@ symtable_visit_annotations(struct symtable *st, stmt_ty o, arguments_ty a, expr_ { int future_annotations = st->st_future->ff_features & CO_FUTURE_ANNOTATIONS; if (future_annotations && - !symtable_enter_block(st, GET_IDENTIFIER(_annotation), AnnotationBlock, + !symtable_enter_block(st, &_Py_ID(_annotation), AnnotationBlock, (void *)o, o->lineno, o->col_offset, o->end_lineno, o->end_col_offset)) { VISIT_QUIT(st, 0); @@ -2085,7 +2070,7 @@ symtable_handle_comprehension(struct symtable *st, expr_ty e, static int symtable_visit_genexp(struct symtable *st, expr_ty e) { - return symtable_handle_comprehension(st, e, GET_IDENTIFIER(genexpr), + return symtable_handle_comprehension(st, e, &_Py_ID(genexpr), e->v.GeneratorExp.generators, e->v.GeneratorExp.elt, NULL); } @@ -2093,7 +2078,7 @@ symtable_visit_genexp(struct symtable *st, expr_ty e) static int symtable_visit_listcomp(struct symtable *st, expr_ty e) { - return symtable_handle_comprehension(st, e, GET_IDENTIFIER(listcomp), + return symtable_handle_comprehension(st, e, &_Py_ID(listcomp), e->v.ListComp.generators, e->v.ListComp.elt, NULL); } @@ -2101,7 +2086,7 @@ symtable_visit_listcomp(struct symtable *st, expr_ty e) static int symtable_visit_setcomp(struct symtable *st, expr_ty e) { - return symtable_handle_comprehension(st, e, GET_IDENTIFIER(setcomp), + return symtable_handle_comprehension(st, e, &_Py_ID(setcomp), e->v.SetComp.generators, e->v.SetComp.elt, NULL); } @@ -2109,7 +2094,7 @@ symtable_visit_setcomp(struct symtable *st, expr_ty e) static int symtable_visit_dictcomp(struct symtable *st, expr_ty e) { - return symtable_handle_comprehension(st, e, GET_IDENTIFIER(dictcomp), + return symtable_handle_comprehension(st, e, &_Py_ID(dictcomp), e->v.DictComp.generators, e->v.DictComp.key, e->v.DictComp.value); @@ -2173,16 +2158,3 @@ _Py_SymtableStringObjectFlags(const char *str, PyObject *filename, _PyArena_Free(arena); return st; } - -void -_PySymtable_Fini(void) -{ - Py_CLEAR(top); - Py_CLEAR(lambda); - Py_CLEAR(genexpr); - Py_CLEAR(listcomp); - Py_CLEAR(setcomp); - Py_CLEAR(dictcomp); - Py_CLEAR(__class__); - Py_CLEAR(_annotation); -} diff --git a/Python/sysmodule.c b/Python/sysmodule.c index 57bf04dd306ae..e23b879c1ab2b 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -706,7 +706,6 @@ sys_displayhook(PyObject *module, PyObject *o) { PyObject *outf; PyObject *builtins; - static PyObject *newline = NULL; PyThreadState *tstate = _PyThreadState_GET(); builtins = PyImport_GetModule(&_Py_ID(builtins)); @@ -747,12 +746,8 @@ sys_displayhook(PyObject *module, PyObject *o) return NULL; } } - if (newline == NULL) { - newline = PyUnicode_FromString("\n"); - if (newline == NULL) - return NULL; - } - if (PyFile_WriteObject(newline, outf, Py_PRINT_RAW) != 0) + _Py_DECLARE_STR(newline, "\n"); + if (PyFile_WriteObject(&_Py_STR(newline), outf, Py_PRINT_RAW) != 0) return NULL; if (PyObject_SetAttr(builtins, &_Py_ID(_), o) != 0) return NULL; @@ -946,30 +941,18 @@ sys_intern_impl(PyObject *module, PyObject *s) /* * Cached interned string objects used for calling the profile and - * trace functions. Initialized by trace_init(). + * trace functions. */ -static PyObject *whatstrings[8] = {NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL}; - -static int -trace_init(void) -{ - static const char * const whatnames[8] = { - "call", "exception", "line", "return", - "c_call", "c_exception", "c_return", - "opcode" - }; - PyObject *name; - int i; - for (i = 0; i < 8; ++i) { - if (whatstrings[i] == NULL) { - name = PyUnicode_InternFromString(whatnames[i]); - if (name == NULL) - return -1; - whatstrings[i] = name; - } - } - return 0; -} +static PyObject *whatstrings[8] = { + &_Py_ID(call), + &_Py_ID(exception), + &_Py_ID(line), + &_Py_ID(return), + &_Py_ID(c_call), + &_Py_ID(c_exception), + &_Py_ID(c_return), + &_Py_ID(opcode), +}; static PyObject * @@ -1050,10 +1033,6 @@ trace_trampoline(PyObject *self, PyFrameObject *frame, static PyObject * sys_settrace(PyObject *self, PyObject *args) { - if (trace_init() == -1) { - return NULL; - } - PyThreadState *tstate = _PyThreadState_GET(); if (args == Py_None) { if (_PyEval_SetTrace(tstate, NULL, NULL) < 0) { @@ -1099,10 +1078,6 @@ sys_gettrace_impl(PyObject *module) static PyObject * sys_setprofile(PyObject *self, PyObject *args) { - if (trace_init() == -1) { - return NULL; - } - PyThreadState *tstate = _PyThreadState_GET(); if (args == Py_None) { if (_PyEval_SetProfile(tstate, NULL, NULL) < 0) { diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 4ef0a6c848b7e..7e939277f1e05 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -280,66 +280,8 @@ Objects/sliceobject.c - _Py_EllipsisObject - # cached - initialized once # manually cached PyUnicodeObject -Objects/boolobject.c - false_str - -Objects/boolobject.c - true_str - -Objects/classobject.c method_get_doc docstr - -Objects/classobject.c instancemethod_get_doc docstr - -Objects/codeobject.c PyCode_NewEmpty emptystring - -Objects/exceptions.c _check_for_legacy_statements print_prefix - -Objects/exceptions.c _check_for_legacy_statements exec_prefix - -Objects/funcobject.c PyFunction_NewWithQualName __name__ - -Objects/listobject.c - indexerr - -Objects/typeobject.c object___reduce_ex___impl objreduce - -# XXX This should have been found by the analyzer but wasn't: -Python/_warnings.c is_internal_frame bootstrap_string - -# XXX This should have been found by the analyzer but wasn't: -Python/_warnings.c is_internal_frame importlib_string - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_close_br - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_dbl_close_br - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_dbl_open_br - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_inf - -# XXX This should have been found by the analyzer but wasn't: -Python/ast_unparse.c - _str_open_br - # XXX This should have been found by the analyzer but wasn't: Python/ast_unparse.c - _str_replace_inf - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c - __annotations__ - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c - __doc__ - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_dictcomp name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_from_import empty_string - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_genexp name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_lambda name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_listcomp name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_setcomp name - -# XXX This should have been found by the analyzer but wasn't: -Python/compile.c compiler_visit_annotations return_str - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c PyImport_Import builtins_str - -# XXX This should have been found by the analyzer but wasn't: -Python/import.c PyImport_Import import_str - -Python/symtable.c - __class__ - -Python/symtable.c - _annotation - -Python/symtable.c - dictcomp - -Python/symtable.c - genexpr - -Python/symtable.c - lambda - -Python/symtable.c - listcomp - -Python/symtable.c - setcomp - -Python/symtable.c - top - -# XXX This should have been found by the analyzer but wasn't: -Python/sysmodule.c - whatstrings - -# XXX This should have been found by the analyzer but wasn't: -Python/sysmodule.c sys_displayhook newline - # _PyArg_Parser (holds tuple of strings) Objects/clinic/bytearrayobject.c.h bytearray___init__ _parser - @@ -402,6 +344,7 @@ Objects/typeobject.c - slotdefs - # other Objects/typeobject.c - method_cache - +Objects/typeobject.c object___reduce_ex___impl objreduce - Objects/unicodeobject.c - _string_module - Objects/unicodeobject.c - interned - Objects/unicodeobject.c - static_strings - @@ -666,37 +609,6 @@ Modules/signalmodule.c - ItimerError - #----------------------- # cached - initialized once -# manually cached PyUnicodeOjbect -Modules/_io/_iomodule.c - _PyIO_str_close - -Modules/_io/_iomodule.c - _PyIO_str_closed - -Modules/_io/_iomodule.c - _PyIO_str_decode - -Modules/_io/_iomodule.c - _PyIO_str_encode - -Modules/_io/_iomodule.c - _PyIO_str_fileno - -Modules/_io/_iomodule.c - _PyIO_str_flush - -Modules/_io/_iomodule.c - _PyIO_str_getstate - -Modules/_io/_iomodule.c - _PyIO_str_isatty - -Modules/_io/_iomodule.c - _PyIO_str_locale - -Modules/_io/_iomodule.c - _PyIO_str_newlines - -Modules/_io/_iomodule.c - _PyIO_str_nl - -Modules/_io/_iomodule.c - _PyIO_str_peek - -Modules/_io/_iomodule.c - _PyIO_str_read - -Modules/_io/_iomodule.c - _PyIO_str_read1 - -Modules/_io/_iomodule.c - _PyIO_str_readable - -Modules/_io/_iomodule.c - _PyIO_str_readall - -Modules/_io/_iomodule.c - _PyIO_str_readinto - -Modules/_io/_iomodule.c - _PyIO_str_readline - -Modules/_io/_iomodule.c - _PyIO_str_reset - -Modules/_io/_iomodule.c - _PyIO_str_seek - -Modules/_io/_iomodule.c - _PyIO_str_seekable - -Modules/_io/_iomodule.c - _PyIO_str_setstate - -Modules/_io/_iomodule.c - _PyIO_str_tell - -Modules/_io/_iomodule.c - _PyIO_str_truncate - -Modules/_io/_iomodule.c - _PyIO_str_writable - -Modules/_io/_iomodule.c - _PyIO_str_write - -Modules/_io/_iomodule.c - _PyIO_empty_str - -Modules/_threadmodule.c - str_dict - -Modules/_tracemalloc.c - unknown_filename - - # _PyArg_Parser Modules/clinic/_codecsmodule.c.h _codecs_decode _parser - Modules/clinic/_codecsmodule.c.h _codecs_encode _parser - diff --git a/Tools/c-analyzer/cpython/ignored.tsv b/Tools/c-analyzer/cpython/ignored.tsv index c945542e29aa5..63d0695bfb4a6 100644 --- a/Tools/c-analyzer/cpython/ignored.tsv +++ b/Tools/c-analyzer/cpython/ignored.tsv @@ -1517,3 +1517,4 @@ Python/pylifecycle.c - _TARGET_LOCALES - Python/specialize.c - adaptive_opcodes - Python/specialize.c - cache_requirements - Python/specialize.c - compare_masks - +Python/sysmodule.c - whatstrings - From webhook-mailer at python.org Tue Feb 22 19:25:05 2022 From: webhook-mailer at python.org (asvetlov) Date: Wed, 23 Feb 2022 00:25:05 -0000 Subject: [Python-checkins] bpo-46522: fix concurrent.futures and io AttributeError messages (GH-30887) Message-ID: https://github.com/python/cpython/commit/9b12b1b803d7b73640ab637a74a6f35f3fe9db21 commit: 9b12b1b803d7b73640ab637a74a6f35f3fe9db21 branch: main author: Thomas Grainger committer: asvetlov date: 2022-02-23T02:25:00+02:00 summary: bpo-46522: fix concurrent.futures and io AttributeError messages (GH-30887) Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Andrew Svetlov files: A Misc/NEWS.d/next/Library/2022-01-25-15-31-04.bpo-46522.tYAlX4.rst M Lib/_pyio.py M Lib/concurrent/futures/__init__.py M Lib/io.py diff --git a/Lib/_pyio.py b/Lib/_pyio.py index d7119742b9d22..8f20c5ed2abd5 100644 --- a/Lib/_pyio.py +++ b/Lib/_pyio.py @@ -312,7 +312,7 @@ def __getattr__(name): global OpenWrapper OpenWrapper = open return OpenWrapper - raise AttributeError(name) + raise AttributeError(f"module {__name__!r} has no attribute {name!r}") # In normal operation, both `UnsupportedOperation`s should be bound to the diff --git a/Lib/concurrent/futures/__init__.py b/Lib/concurrent/futures/__init__.py index d746aeac50a99..292e886d5a88a 100644 --- a/Lib/concurrent/futures/__init__.py +++ b/Lib/concurrent/futures/__init__.py @@ -50,4 +50,4 @@ def __getattr__(name): ThreadPoolExecutor = te return te - raise AttributeError(f"module {__name__} has no attribute {name}") + raise AttributeError(f"module {__name__!r} has no attribute {name!r}") diff --git a/Lib/io.py b/Lib/io.py index 2a6140c3dd509..a205e00575f7e 100644 --- a/Lib/io.py +++ b/Lib/io.py @@ -70,7 +70,7 @@ def __getattr__(name): global OpenWrapper OpenWrapper = open return OpenWrapper - raise AttributeError(name) + raise AttributeError("module {__name__!r} has no attribute {name!r}") # Pretend this exception was created here. diff --git a/Misc/NEWS.d/next/Library/2022-01-25-15-31-04.bpo-46522.tYAlX4.rst b/Misc/NEWS.d/next/Library/2022-01-25-15-31-04.bpo-46522.tYAlX4.rst new file mode 100644 index 0000000000000..999863adb9b31 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-01-25-15-31-04.bpo-46522.tYAlX4.rst @@ -0,0 +1 @@ +Make various module ``__getattr__`` AttributeErrors more closely match a typical AttributeError From webhook-mailer at python.org Tue Feb 22 20:40:46 2022 From: webhook-mailer at python.org (corona10) Date: Wed, 23 Feb 2022 01:40:46 -0000 Subject: [Python-checkins] bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) Message-ID: https://github.com/python/cpython/commit/1935e1cc284942bec8006287c939e295e1a7bf13 commit: 1935e1cc284942bec8006287c939e295e1a7bf13 branch: main author: Dong-hee Na committer: corona10 date: 2022-02-23T10:40:30+09:00 summary: bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst M Modules/expat/expat.h M Modules/expat/xmlparse.c M Modules/expat/xmlrole.c M Modules/expat/xmltok.c M Modules/expat/xmltok_impl.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst new file mode 100644 index 0000000000000..127387d32cb7a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst @@ -0,0 +1 @@ +Bump up the libexpat version into 2.4.6 diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index 4c5704fd9336b..46a0e1bcd22de 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -1041,7 +1041,7 @@ XML_SetBillionLaughsAttackProtectionActivationThreshold( */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 4 -#define XML_MICRO_VERSION 4 +#define XML_MICRO_VERSION 6 #ifdef __cplusplus } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 4b43e61321691..7db28d07acbcd 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 2e2c8ce5f11a473d65ec313ab20ceee6afefb355f5405afc06e7204e2e41c8c0 (2.4.4+) +/* a30d2613dcfdef81475a9d1a349134d2d42722172fdaa7d5bb12ed2aa74b9596 (2.4.6+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -11,7 +11,7 @@ Copyright (c) 2000-2006 Fred L. Drake, Jr. Copyright (c) 2001-2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016 Eric Rahm Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Gaurav @@ -718,8 +718,7 @@ XML_ParserCreate(const XML_Char *encodingName) { XML_Parser XMLCALL XML_ParserCreateNS(const XML_Char *encodingName, XML_Char nsSep) { - XML_Char tmp[2]; - *tmp = nsSep; + XML_Char tmp[2] = {nsSep, 0}; return XML_ParserCreate_MM(encodingName, NULL, tmp); } @@ -1344,8 +1343,7 @@ XML_ExternalEntityParserCreate(XML_Parser oldParser, const XML_Char *context, would be otherwise. */ if (parser->m_ns) { - XML_Char tmp[2]; - *tmp = parser->m_namespaceSeparator; + XML_Char tmp[2] = {parser->m_namespaceSeparator, 0}; parser = parserCreate(encodingName, &parser->m_mem, tmp, newDtd); } else { parser = parserCreate(encodingName, &parser->m_mem, NULL, newDtd); @@ -2563,6 +2561,7 @@ storeRawNames(XML_Parser parser) { while (tag) { int bufSize; int nameLen = sizeof(XML_Char) * (tag->name.strLen + 1); + size_t rawNameLen; char *rawNameBuf = tag->buf + nameLen; /* Stop if already stored. Since m_tagStack is a stack, we can stop at the first entry that has already been copied; everything @@ -2574,7 +2573,11 @@ storeRawNames(XML_Parser parser) { /* For re-use purposes we need to ensure that the size of tag->buf is a multiple of sizeof(XML_Char). */ - bufSize = nameLen + ROUND_UP(tag->rawNameLength, sizeof(XML_Char)); + rawNameLen = ROUND_UP(tag->rawNameLength, sizeof(XML_Char)); + /* Detect and prevent integer overflow. */ + if (rawNameLen > (size_t)INT_MAX - nameLen) + return XML_FALSE; + bufSize = nameLen + (int)rawNameLen; if (bufSize > tag->bufEnd - tag->buf) { char *temp = (char *)REALLOC(parser, tag->buf, bufSize); if (temp == NULL) @@ -3756,6 +3759,17 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (! mustBeXML && isXMLNS && (len > xmlnsLen || uri[len] != xmlnsNamespace[len])) isXMLNS = XML_FALSE; + + // NOTE: While Expat does not validate namespace URIs against RFC 3986, + // we have to at least make sure that the XML processor on top of + // Expat (that is splitting tag names by namespace separator into + // 2- or 3-tuples (uri-local or uri-local-prefix)) cannot be confused + // by an attacker putting additional namespace separator characters + // into namespace declarations. That would be ambiguous and not to + // be expected. + if (parser->m_ns && (uri[len] == parser->m_namespaceSeparator)) { + return XML_ERROR_SYNTAX; + } } isXML = isXML && len == xmlLen; isXMLNS = isXMLNS && len == xmlnsLen; @@ -7317,44 +7331,15 @@ nextScaffoldPart(XML_Parser parser) { return next; } -static void -build_node(XML_Parser parser, int src_node, XML_Content *dest, - XML_Content **contpos, XML_Char **strpos) { - DTD *const dtd = parser->m_dtd; /* save one level of indirection */ - dest->type = dtd->scaffold[src_node].type; - dest->quant = dtd->scaffold[src_node].quant; - if (dest->type == XML_CTYPE_NAME) { - const XML_Char *src; - dest->name = *strpos; - src = dtd->scaffold[src_node].name; - for (;;) { - *(*strpos)++ = *src; - if (! *src) - break; - src++; - } - dest->numchildren = 0; - dest->children = NULL; - } else { - unsigned int i; - int cn; - dest->numchildren = dtd->scaffold[src_node].childcnt; - dest->children = *contpos; - *contpos += dest->numchildren; - for (i = 0, cn = dtd->scaffold[src_node].firstchild; i < dest->numchildren; - i++, cn = dtd->scaffold[cn].nextsib) { - build_node(parser, cn, &(dest->children[i]), contpos, strpos); - } - dest->name = NULL; - } -} - static XML_Content * build_model(XML_Parser parser) { + /* Function build_model transforms the existing parser->m_dtd->scaffold + * array of CONTENT_SCAFFOLD tree nodes into a new array of + * XML_Content tree nodes followed by a gapless list of zero-terminated + * strings. */ DTD *const dtd = parser->m_dtd; /* save one level of indirection */ XML_Content *ret; - XML_Content *cpos; - XML_Char *str; + XML_Char *str; /* the current string writing location */ /* Detect and prevent integer overflow. * The preprocessor guard addresses the "always false" warning @@ -7380,10 +7365,96 @@ build_model(XML_Parser parser) { if (! ret) return NULL; - str = (XML_Char *)(&ret[dtd->scaffCount]); - cpos = &ret[1]; + /* What follows is an iterative implementation (of what was previously done + * recursively in a dedicated function called "build_node". The old recursive + * build_node could be forced into stack exhaustion from input as small as a + * few megabyte, and so that was a security issue. Hence, a function call + * stack is avoided now by resolving recursion.) + * + * The iterative approach works as follows: + * + * - We have two writing pointers, both walking up the result array; one does + * the work, the other creates "jobs" for its colleague to do, and leads + * the way: + * + * - The faster one, pointer jobDest, always leads and writes "what job + * to do" by the other, once they reach that place in the + * array: leader "jobDest" stores the source node array index (relative + * to array dtd->scaffold) in field "numchildren". + * + * - The slower one, pointer dest, looks at the value stored in the + * "numchildren" field (which actually holds a source node array index + * at that time) and puts the real data from dtd->scaffold in. + * + * - Before the loop starts, jobDest writes source array index 0 + * (where the root node is located) so that dest will have something to do + * when it starts operation. + * + * - Whenever nodes with children are encountered, jobDest appends + * them as new jobs, in order. As a result, tree node siblings are + * adjacent in the resulting array, for example: + * + * [0] root, has two children + * [1] first child of 0, has three children + * [3] first child of 1, does not have children + * [4] second child of 1, does not have children + * [5] third child of 1, does not have children + * [2] second child of 0, does not have children + * + * Or (the same data) presented in flat array view: + * + * [0] root, has two children + * + * [1] first child of 0, has three children + * [2] second child of 0, does not have children + * + * [3] first child of 1, does not have children + * [4] second child of 1, does not have children + * [5] third child of 1, does not have children + * + * - The algorithm repeats until all target array indices have been processed. + */ + XML_Content *dest = ret; /* tree node writing location, moves upwards */ + XML_Content *const destLimit = &ret[dtd->scaffCount]; + XML_Content *jobDest = ret; /* next free writing location in target array */ + str = (XML_Char *)&ret[dtd->scaffCount]; + + /* Add the starting job, the root node (index 0) of the source tree */ + (jobDest++)->numchildren = 0; + + for (; dest < destLimit; dest++) { + /* Retrieve source tree array index from job storage */ + const int src_node = (int)dest->numchildren; + + /* Convert item */ + dest->type = dtd->scaffold[src_node].type; + dest->quant = dtd->scaffold[src_node].quant; + if (dest->type == XML_CTYPE_NAME) { + const XML_Char *src; + dest->name = str; + src = dtd->scaffold[src_node].name; + for (;;) { + *str++ = *src; + if (! *src) + break; + src++; + } + dest->numchildren = 0; + dest->children = NULL; + } else { + unsigned int i; + int cn; + dest->name = NULL; + dest->numchildren = dtd->scaffold[src_node].childcnt; + dest->children = jobDest; + + /* Append scaffold indices of children to array */ + for (i = 0, cn = dtd->scaffold[src_node].firstchild; + i < dest->numchildren; i++, cn = dtd->scaffold[cn].nextsib) + (jobDest++)->numchildren = (unsigned int)cn; + } + } - build_node(parser, 0, ret, &cpos, &str); return ret; } @@ -7412,7 +7483,7 @@ getElementType(XML_Parser parser, const ENCODING *enc, const char *ptr, static XML_Char * copyString(const XML_Char *s, const XML_Memory_Handling_Suite *memsuite) { - int charsRequired = 0; + size_t charsRequired = 0; XML_Char *result; /* First determine how long the string is */ diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 77746ee42d10a..3f0f5c150c627 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -11,7 +11,7 @@ Copyright (c) 2002 Greg Stein Copyright (c) 2002-2006 Karl Waclawek Copyright (c) 2002-2003 Fred L. Drake, Jr. - Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016-2021 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2019 David Loffredo diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index 502ca1adc33b9..c659983b4008b 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -11,8 +11,8 @@ Copyright (c) 2001-2003 Fred L. Drake, Jr. Copyright (c) 2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2005-2009 Steven Solie - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Pascal Cuoq Copyright (c) 2016 Don Lewis Copyright (c) 2017 Rhodri James @@ -98,11 +98,6 @@ + ((((byte)[1]) & 3) << 1) + ((((byte)[2]) >> 5) & 1)] \ & (1u << (((byte)[2]) & 0x1F))) -#define UTF8_GET_NAMING(pages, p, n) \ - ((n) == 2 \ - ? UTF8_GET_NAMING2(pages, (const unsigned char *)(p)) \ - : ((n) == 3 ? UTF8_GET_NAMING3(pages, (const unsigned char *)(p)) : 0)) - /* Detection of invalid UTF-8 sequences is based on Table 3.1B of Unicode 3.2: http://www.unicode.org/unicode/reports/tr28/ with the additional restriction of not allowing the Unicode diff --git a/Modules/expat/xmltok_impl.c b/Modules/expat/xmltok_impl.c index 0430591b42636..4072b06497d1c 100644 --- a/Modules/expat/xmltok_impl.c +++ b/Modules/expat/xmltok_impl.c @@ -10,7 +10,7 @@ Copyright (c) 2000 Clark Cooper Copyright (c) 2002 Fred L. Drake, Jr. Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2018 Benjamin Peterson Copyright (c) 2018 Anton Maklakov @@ -69,7 +69,7 @@ case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ - if (! IS_NAME_CHAR(enc, ptr, n)) { \ + if (IS_INVALID_CHAR(enc, ptr, n) || ! IS_NAME_CHAR(enc, ptr, n)) { \ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ @@ -98,7 +98,7 @@ case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ - if (! IS_NMSTRT_CHAR(enc, ptr, n)) { \ + if (IS_INVALID_CHAR(enc, ptr, n) || ! IS_NMSTRT_CHAR(enc, ptr, n)) { \ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ @@ -1142,6 +1142,10 @@ PREFIX(prologTok)(const ENCODING *enc, const char *ptr, const char *end, case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ + if (IS_INVALID_CHAR(enc, ptr, n)) { \ + *nextTokPtr = ptr; \ + return XML_TOK_INVALID; \ + } \ if (IS_NMSTRT_CHAR(enc, ptr, n)) { \ ptr += n; \ tok = XML_TOK_NAME; \ @@ -1270,7 +1274,7 @@ PREFIX(attributeValueTok)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1339,7 +1343,7 @@ PREFIX(entityValueTok)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1518,7 +1522,7 @@ PREFIX(getAtts)(const ENCODING *enc, const char *ptr, int attsMax, state = inName; \ } # define LEAD_CASE(n) \ - case BT_LEAD##n: \ + case BT_LEAD##n: /* NOTE: The encoding has already been validated. */ \ START_NAME ptr += (n - MINBPC(enc)); \ break; LEAD_CASE(2) @@ -1730,7 +1734,7 @@ PREFIX(nameLength)(const ENCODING *enc, const char *ptr) { switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1775,7 +1779,7 @@ PREFIX(updatePosition)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ pos->columnNumber++; \ break; LEAD_CASE(2) From webhook-mailer at python.org Tue Feb 22 21:50:45 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 23 Feb 2022 02:50:45 -0000 Subject: [Python-checkins] bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) Message-ID: https://github.com/python/cpython/commit/4955a9ed14c681ed835bc8902a9db0bcc728bdee commit: 4955a9ed14c681ed835bc8902a9db0bcc728bdee branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T18:50:38-08:00 summary: bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) (cherry picked from commit 1935e1cc284942bec8006287c939e295e1a7bf13) Co-authored-by: Dong-hee Na files: A Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst M Modules/expat/expat.h M Modules/expat/xmlparse.c M Modules/expat/xmlrole.c M Modules/expat/xmltok.c M Modules/expat/xmltok_impl.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst new file mode 100644 index 0000000000000..127387d32cb7a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst @@ -0,0 +1 @@ +Bump up the libexpat version into 2.4.6 diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index 4c5704fd9336b..46a0e1bcd22de 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -1041,7 +1041,7 @@ XML_SetBillionLaughsAttackProtectionActivationThreshold( */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 4 -#define XML_MICRO_VERSION 4 +#define XML_MICRO_VERSION 6 #ifdef __cplusplus } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 4b43e61321691..7db28d07acbcd 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 2e2c8ce5f11a473d65ec313ab20ceee6afefb355f5405afc06e7204e2e41c8c0 (2.4.4+) +/* a30d2613dcfdef81475a9d1a349134d2d42722172fdaa7d5bb12ed2aa74b9596 (2.4.6+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -11,7 +11,7 @@ Copyright (c) 2000-2006 Fred L. Drake, Jr. Copyright (c) 2001-2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016 Eric Rahm Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Gaurav @@ -718,8 +718,7 @@ XML_ParserCreate(const XML_Char *encodingName) { XML_Parser XMLCALL XML_ParserCreateNS(const XML_Char *encodingName, XML_Char nsSep) { - XML_Char tmp[2]; - *tmp = nsSep; + XML_Char tmp[2] = {nsSep, 0}; return XML_ParserCreate_MM(encodingName, NULL, tmp); } @@ -1344,8 +1343,7 @@ XML_ExternalEntityParserCreate(XML_Parser oldParser, const XML_Char *context, would be otherwise. */ if (parser->m_ns) { - XML_Char tmp[2]; - *tmp = parser->m_namespaceSeparator; + XML_Char tmp[2] = {parser->m_namespaceSeparator, 0}; parser = parserCreate(encodingName, &parser->m_mem, tmp, newDtd); } else { parser = parserCreate(encodingName, &parser->m_mem, NULL, newDtd); @@ -2563,6 +2561,7 @@ storeRawNames(XML_Parser parser) { while (tag) { int bufSize; int nameLen = sizeof(XML_Char) * (tag->name.strLen + 1); + size_t rawNameLen; char *rawNameBuf = tag->buf + nameLen; /* Stop if already stored. Since m_tagStack is a stack, we can stop at the first entry that has already been copied; everything @@ -2574,7 +2573,11 @@ storeRawNames(XML_Parser parser) { /* For re-use purposes we need to ensure that the size of tag->buf is a multiple of sizeof(XML_Char). */ - bufSize = nameLen + ROUND_UP(tag->rawNameLength, sizeof(XML_Char)); + rawNameLen = ROUND_UP(tag->rawNameLength, sizeof(XML_Char)); + /* Detect and prevent integer overflow. */ + if (rawNameLen > (size_t)INT_MAX - nameLen) + return XML_FALSE; + bufSize = nameLen + (int)rawNameLen; if (bufSize > tag->bufEnd - tag->buf) { char *temp = (char *)REALLOC(parser, tag->buf, bufSize); if (temp == NULL) @@ -3756,6 +3759,17 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (! mustBeXML && isXMLNS && (len > xmlnsLen || uri[len] != xmlnsNamespace[len])) isXMLNS = XML_FALSE; + + // NOTE: While Expat does not validate namespace URIs against RFC 3986, + // we have to at least make sure that the XML processor on top of + // Expat (that is splitting tag names by namespace separator into + // 2- or 3-tuples (uri-local or uri-local-prefix)) cannot be confused + // by an attacker putting additional namespace separator characters + // into namespace declarations. That would be ambiguous and not to + // be expected. + if (parser->m_ns && (uri[len] == parser->m_namespaceSeparator)) { + return XML_ERROR_SYNTAX; + } } isXML = isXML && len == xmlLen; isXMLNS = isXMLNS && len == xmlnsLen; @@ -7317,44 +7331,15 @@ nextScaffoldPart(XML_Parser parser) { return next; } -static void -build_node(XML_Parser parser, int src_node, XML_Content *dest, - XML_Content **contpos, XML_Char **strpos) { - DTD *const dtd = parser->m_dtd; /* save one level of indirection */ - dest->type = dtd->scaffold[src_node].type; - dest->quant = dtd->scaffold[src_node].quant; - if (dest->type == XML_CTYPE_NAME) { - const XML_Char *src; - dest->name = *strpos; - src = dtd->scaffold[src_node].name; - for (;;) { - *(*strpos)++ = *src; - if (! *src) - break; - src++; - } - dest->numchildren = 0; - dest->children = NULL; - } else { - unsigned int i; - int cn; - dest->numchildren = dtd->scaffold[src_node].childcnt; - dest->children = *contpos; - *contpos += dest->numchildren; - for (i = 0, cn = dtd->scaffold[src_node].firstchild; i < dest->numchildren; - i++, cn = dtd->scaffold[cn].nextsib) { - build_node(parser, cn, &(dest->children[i]), contpos, strpos); - } - dest->name = NULL; - } -} - static XML_Content * build_model(XML_Parser parser) { + /* Function build_model transforms the existing parser->m_dtd->scaffold + * array of CONTENT_SCAFFOLD tree nodes into a new array of + * XML_Content tree nodes followed by a gapless list of zero-terminated + * strings. */ DTD *const dtd = parser->m_dtd; /* save one level of indirection */ XML_Content *ret; - XML_Content *cpos; - XML_Char *str; + XML_Char *str; /* the current string writing location */ /* Detect and prevent integer overflow. * The preprocessor guard addresses the "always false" warning @@ -7380,10 +7365,96 @@ build_model(XML_Parser parser) { if (! ret) return NULL; - str = (XML_Char *)(&ret[dtd->scaffCount]); - cpos = &ret[1]; + /* What follows is an iterative implementation (of what was previously done + * recursively in a dedicated function called "build_node". The old recursive + * build_node could be forced into stack exhaustion from input as small as a + * few megabyte, and so that was a security issue. Hence, a function call + * stack is avoided now by resolving recursion.) + * + * The iterative approach works as follows: + * + * - We have two writing pointers, both walking up the result array; one does + * the work, the other creates "jobs" for its colleague to do, and leads + * the way: + * + * - The faster one, pointer jobDest, always leads and writes "what job + * to do" by the other, once they reach that place in the + * array: leader "jobDest" stores the source node array index (relative + * to array dtd->scaffold) in field "numchildren". + * + * - The slower one, pointer dest, looks at the value stored in the + * "numchildren" field (which actually holds a source node array index + * at that time) and puts the real data from dtd->scaffold in. + * + * - Before the loop starts, jobDest writes source array index 0 + * (where the root node is located) so that dest will have something to do + * when it starts operation. + * + * - Whenever nodes with children are encountered, jobDest appends + * them as new jobs, in order. As a result, tree node siblings are + * adjacent in the resulting array, for example: + * + * [0] root, has two children + * [1] first child of 0, has three children + * [3] first child of 1, does not have children + * [4] second child of 1, does not have children + * [5] third child of 1, does not have children + * [2] second child of 0, does not have children + * + * Or (the same data) presented in flat array view: + * + * [0] root, has two children + * + * [1] first child of 0, has three children + * [2] second child of 0, does not have children + * + * [3] first child of 1, does not have children + * [4] second child of 1, does not have children + * [5] third child of 1, does not have children + * + * - The algorithm repeats until all target array indices have been processed. + */ + XML_Content *dest = ret; /* tree node writing location, moves upwards */ + XML_Content *const destLimit = &ret[dtd->scaffCount]; + XML_Content *jobDest = ret; /* next free writing location in target array */ + str = (XML_Char *)&ret[dtd->scaffCount]; + + /* Add the starting job, the root node (index 0) of the source tree */ + (jobDest++)->numchildren = 0; + + for (; dest < destLimit; dest++) { + /* Retrieve source tree array index from job storage */ + const int src_node = (int)dest->numchildren; + + /* Convert item */ + dest->type = dtd->scaffold[src_node].type; + dest->quant = dtd->scaffold[src_node].quant; + if (dest->type == XML_CTYPE_NAME) { + const XML_Char *src; + dest->name = str; + src = dtd->scaffold[src_node].name; + for (;;) { + *str++ = *src; + if (! *src) + break; + src++; + } + dest->numchildren = 0; + dest->children = NULL; + } else { + unsigned int i; + int cn; + dest->name = NULL; + dest->numchildren = dtd->scaffold[src_node].childcnt; + dest->children = jobDest; + + /* Append scaffold indices of children to array */ + for (i = 0, cn = dtd->scaffold[src_node].firstchild; + i < dest->numchildren; i++, cn = dtd->scaffold[cn].nextsib) + (jobDest++)->numchildren = (unsigned int)cn; + } + } - build_node(parser, 0, ret, &cpos, &str); return ret; } @@ -7412,7 +7483,7 @@ getElementType(XML_Parser parser, const ENCODING *enc, const char *ptr, static XML_Char * copyString(const XML_Char *s, const XML_Memory_Handling_Suite *memsuite) { - int charsRequired = 0; + size_t charsRequired = 0; XML_Char *result; /* First determine how long the string is */ diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 77746ee42d10a..3f0f5c150c627 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -11,7 +11,7 @@ Copyright (c) 2002 Greg Stein Copyright (c) 2002-2006 Karl Waclawek Copyright (c) 2002-2003 Fred L. Drake, Jr. - Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016-2021 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2019 David Loffredo diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index 502ca1adc33b9..c659983b4008b 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -11,8 +11,8 @@ Copyright (c) 2001-2003 Fred L. Drake, Jr. Copyright (c) 2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2005-2009 Steven Solie - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Pascal Cuoq Copyright (c) 2016 Don Lewis Copyright (c) 2017 Rhodri James @@ -98,11 +98,6 @@ + ((((byte)[1]) & 3) << 1) + ((((byte)[2]) >> 5) & 1)] \ & (1u << (((byte)[2]) & 0x1F))) -#define UTF8_GET_NAMING(pages, p, n) \ - ((n) == 2 \ - ? UTF8_GET_NAMING2(pages, (const unsigned char *)(p)) \ - : ((n) == 3 ? UTF8_GET_NAMING3(pages, (const unsigned char *)(p)) : 0)) - /* Detection of invalid UTF-8 sequences is based on Table 3.1B of Unicode 3.2: http://www.unicode.org/unicode/reports/tr28/ with the additional restriction of not allowing the Unicode diff --git a/Modules/expat/xmltok_impl.c b/Modules/expat/xmltok_impl.c index 0430591b42636..4072b06497d1c 100644 --- a/Modules/expat/xmltok_impl.c +++ b/Modules/expat/xmltok_impl.c @@ -10,7 +10,7 @@ Copyright (c) 2000 Clark Cooper Copyright (c) 2002 Fred L. Drake, Jr. Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2018 Benjamin Peterson Copyright (c) 2018 Anton Maklakov @@ -69,7 +69,7 @@ case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ - if (! IS_NAME_CHAR(enc, ptr, n)) { \ + if (IS_INVALID_CHAR(enc, ptr, n) || ! IS_NAME_CHAR(enc, ptr, n)) { \ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ @@ -98,7 +98,7 @@ case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ - if (! IS_NMSTRT_CHAR(enc, ptr, n)) { \ + if (IS_INVALID_CHAR(enc, ptr, n) || ! IS_NMSTRT_CHAR(enc, ptr, n)) { \ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ @@ -1142,6 +1142,10 @@ PREFIX(prologTok)(const ENCODING *enc, const char *ptr, const char *end, case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ + if (IS_INVALID_CHAR(enc, ptr, n)) { \ + *nextTokPtr = ptr; \ + return XML_TOK_INVALID; \ + } \ if (IS_NMSTRT_CHAR(enc, ptr, n)) { \ ptr += n; \ tok = XML_TOK_NAME; \ @@ -1270,7 +1274,7 @@ PREFIX(attributeValueTok)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1339,7 +1343,7 @@ PREFIX(entityValueTok)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1518,7 +1522,7 @@ PREFIX(getAtts)(const ENCODING *enc, const char *ptr, int attsMax, state = inName; \ } # define LEAD_CASE(n) \ - case BT_LEAD##n: \ + case BT_LEAD##n: /* NOTE: The encoding has already been validated. */ \ START_NAME ptr += (n - MINBPC(enc)); \ break; LEAD_CASE(2) @@ -1730,7 +1734,7 @@ PREFIX(nameLength)(const ENCODING *enc, const char *ptr) { switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1775,7 +1779,7 @@ PREFIX(updatePosition)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ pos->columnNumber++; \ break; LEAD_CASE(2) From webhook-mailer at python.org Tue Feb 22 21:51:09 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 23 Feb 2022 02:51:09 -0000 Subject: [Python-checkins] bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) Message-ID: https://github.com/python/cpython/commit/87cebb1e69758aa8b79f8e15187b976d62cba36a commit: 87cebb1e69758aa8b79f8e15187b976d62cba36a branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T18:50:57-08:00 summary: bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) (cherry picked from commit 1935e1cc284942bec8006287c939e295e1a7bf13) Co-authored-by: Dong-hee Na files: A Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst M Modules/expat/expat.h M Modules/expat/xmlparse.c M Modules/expat/xmlrole.c M Modules/expat/xmltok.c M Modules/expat/xmltok_impl.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst new file mode 100644 index 0000000000000..127387d32cb7a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst @@ -0,0 +1 @@ +Bump up the libexpat version into 2.4.6 diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index 4c5704fd9336b..46a0e1bcd22de 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -1041,7 +1041,7 @@ XML_SetBillionLaughsAttackProtectionActivationThreshold( */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 4 -#define XML_MICRO_VERSION 4 +#define XML_MICRO_VERSION 6 #ifdef __cplusplus } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 4b43e61321691..7db28d07acbcd 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 2e2c8ce5f11a473d65ec313ab20ceee6afefb355f5405afc06e7204e2e41c8c0 (2.4.4+) +/* a30d2613dcfdef81475a9d1a349134d2d42722172fdaa7d5bb12ed2aa74b9596 (2.4.6+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -11,7 +11,7 @@ Copyright (c) 2000-2006 Fred L. Drake, Jr. Copyright (c) 2001-2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016 Eric Rahm Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Gaurav @@ -718,8 +718,7 @@ XML_ParserCreate(const XML_Char *encodingName) { XML_Parser XMLCALL XML_ParserCreateNS(const XML_Char *encodingName, XML_Char nsSep) { - XML_Char tmp[2]; - *tmp = nsSep; + XML_Char tmp[2] = {nsSep, 0}; return XML_ParserCreate_MM(encodingName, NULL, tmp); } @@ -1344,8 +1343,7 @@ XML_ExternalEntityParserCreate(XML_Parser oldParser, const XML_Char *context, would be otherwise. */ if (parser->m_ns) { - XML_Char tmp[2]; - *tmp = parser->m_namespaceSeparator; + XML_Char tmp[2] = {parser->m_namespaceSeparator, 0}; parser = parserCreate(encodingName, &parser->m_mem, tmp, newDtd); } else { parser = parserCreate(encodingName, &parser->m_mem, NULL, newDtd); @@ -2563,6 +2561,7 @@ storeRawNames(XML_Parser parser) { while (tag) { int bufSize; int nameLen = sizeof(XML_Char) * (tag->name.strLen + 1); + size_t rawNameLen; char *rawNameBuf = tag->buf + nameLen; /* Stop if already stored. Since m_tagStack is a stack, we can stop at the first entry that has already been copied; everything @@ -2574,7 +2573,11 @@ storeRawNames(XML_Parser parser) { /* For re-use purposes we need to ensure that the size of tag->buf is a multiple of sizeof(XML_Char). */ - bufSize = nameLen + ROUND_UP(tag->rawNameLength, sizeof(XML_Char)); + rawNameLen = ROUND_UP(tag->rawNameLength, sizeof(XML_Char)); + /* Detect and prevent integer overflow. */ + if (rawNameLen > (size_t)INT_MAX - nameLen) + return XML_FALSE; + bufSize = nameLen + (int)rawNameLen; if (bufSize > tag->bufEnd - tag->buf) { char *temp = (char *)REALLOC(parser, tag->buf, bufSize); if (temp == NULL) @@ -3756,6 +3759,17 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (! mustBeXML && isXMLNS && (len > xmlnsLen || uri[len] != xmlnsNamespace[len])) isXMLNS = XML_FALSE; + + // NOTE: While Expat does not validate namespace URIs against RFC 3986, + // we have to at least make sure that the XML processor on top of + // Expat (that is splitting tag names by namespace separator into + // 2- or 3-tuples (uri-local or uri-local-prefix)) cannot be confused + // by an attacker putting additional namespace separator characters + // into namespace declarations. That would be ambiguous and not to + // be expected. + if (parser->m_ns && (uri[len] == parser->m_namespaceSeparator)) { + return XML_ERROR_SYNTAX; + } } isXML = isXML && len == xmlLen; isXMLNS = isXMLNS && len == xmlnsLen; @@ -7317,44 +7331,15 @@ nextScaffoldPart(XML_Parser parser) { return next; } -static void -build_node(XML_Parser parser, int src_node, XML_Content *dest, - XML_Content **contpos, XML_Char **strpos) { - DTD *const dtd = parser->m_dtd; /* save one level of indirection */ - dest->type = dtd->scaffold[src_node].type; - dest->quant = dtd->scaffold[src_node].quant; - if (dest->type == XML_CTYPE_NAME) { - const XML_Char *src; - dest->name = *strpos; - src = dtd->scaffold[src_node].name; - for (;;) { - *(*strpos)++ = *src; - if (! *src) - break; - src++; - } - dest->numchildren = 0; - dest->children = NULL; - } else { - unsigned int i; - int cn; - dest->numchildren = dtd->scaffold[src_node].childcnt; - dest->children = *contpos; - *contpos += dest->numchildren; - for (i = 0, cn = dtd->scaffold[src_node].firstchild; i < dest->numchildren; - i++, cn = dtd->scaffold[cn].nextsib) { - build_node(parser, cn, &(dest->children[i]), contpos, strpos); - } - dest->name = NULL; - } -} - static XML_Content * build_model(XML_Parser parser) { + /* Function build_model transforms the existing parser->m_dtd->scaffold + * array of CONTENT_SCAFFOLD tree nodes into a new array of + * XML_Content tree nodes followed by a gapless list of zero-terminated + * strings. */ DTD *const dtd = parser->m_dtd; /* save one level of indirection */ XML_Content *ret; - XML_Content *cpos; - XML_Char *str; + XML_Char *str; /* the current string writing location */ /* Detect and prevent integer overflow. * The preprocessor guard addresses the "always false" warning @@ -7380,10 +7365,96 @@ build_model(XML_Parser parser) { if (! ret) return NULL; - str = (XML_Char *)(&ret[dtd->scaffCount]); - cpos = &ret[1]; + /* What follows is an iterative implementation (of what was previously done + * recursively in a dedicated function called "build_node". The old recursive + * build_node could be forced into stack exhaustion from input as small as a + * few megabyte, and so that was a security issue. Hence, a function call + * stack is avoided now by resolving recursion.) + * + * The iterative approach works as follows: + * + * - We have two writing pointers, both walking up the result array; one does + * the work, the other creates "jobs" for its colleague to do, and leads + * the way: + * + * - The faster one, pointer jobDest, always leads and writes "what job + * to do" by the other, once they reach that place in the + * array: leader "jobDest" stores the source node array index (relative + * to array dtd->scaffold) in field "numchildren". + * + * - The slower one, pointer dest, looks at the value stored in the + * "numchildren" field (which actually holds a source node array index + * at that time) and puts the real data from dtd->scaffold in. + * + * - Before the loop starts, jobDest writes source array index 0 + * (where the root node is located) so that dest will have something to do + * when it starts operation. + * + * - Whenever nodes with children are encountered, jobDest appends + * them as new jobs, in order. As a result, tree node siblings are + * adjacent in the resulting array, for example: + * + * [0] root, has two children + * [1] first child of 0, has three children + * [3] first child of 1, does not have children + * [4] second child of 1, does not have children + * [5] third child of 1, does not have children + * [2] second child of 0, does not have children + * + * Or (the same data) presented in flat array view: + * + * [0] root, has two children + * + * [1] first child of 0, has three children + * [2] second child of 0, does not have children + * + * [3] first child of 1, does not have children + * [4] second child of 1, does not have children + * [5] third child of 1, does not have children + * + * - The algorithm repeats until all target array indices have been processed. + */ + XML_Content *dest = ret; /* tree node writing location, moves upwards */ + XML_Content *const destLimit = &ret[dtd->scaffCount]; + XML_Content *jobDest = ret; /* next free writing location in target array */ + str = (XML_Char *)&ret[dtd->scaffCount]; + + /* Add the starting job, the root node (index 0) of the source tree */ + (jobDest++)->numchildren = 0; + + for (; dest < destLimit; dest++) { + /* Retrieve source tree array index from job storage */ + const int src_node = (int)dest->numchildren; + + /* Convert item */ + dest->type = dtd->scaffold[src_node].type; + dest->quant = dtd->scaffold[src_node].quant; + if (dest->type == XML_CTYPE_NAME) { + const XML_Char *src; + dest->name = str; + src = dtd->scaffold[src_node].name; + for (;;) { + *str++ = *src; + if (! *src) + break; + src++; + } + dest->numchildren = 0; + dest->children = NULL; + } else { + unsigned int i; + int cn; + dest->name = NULL; + dest->numchildren = dtd->scaffold[src_node].childcnt; + dest->children = jobDest; + + /* Append scaffold indices of children to array */ + for (i = 0, cn = dtd->scaffold[src_node].firstchild; + i < dest->numchildren; i++, cn = dtd->scaffold[cn].nextsib) + (jobDest++)->numchildren = (unsigned int)cn; + } + } - build_node(parser, 0, ret, &cpos, &str); return ret; } @@ -7412,7 +7483,7 @@ getElementType(XML_Parser parser, const ENCODING *enc, const char *ptr, static XML_Char * copyString(const XML_Char *s, const XML_Memory_Handling_Suite *memsuite) { - int charsRequired = 0; + size_t charsRequired = 0; XML_Char *result; /* First determine how long the string is */ diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 77746ee42d10a..3f0f5c150c627 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -11,7 +11,7 @@ Copyright (c) 2002 Greg Stein Copyright (c) 2002-2006 Karl Waclawek Copyright (c) 2002-2003 Fred L. Drake, Jr. - Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016-2021 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2019 David Loffredo diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index 502ca1adc33b9..c659983b4008b 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -11,8 +11,8 @@ Copyright (c) 2001-2003 Fred L. Drake, Jr. Copyright (c) 2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2005-2009 Steven Solie - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Pascal Cuoq Copyright (c) 2016 Don Lewis Copyright (c) 2017 Rhodri James @@ -98,11 +98,6 @@ + ((((byte)[1]) & 3) << 1) + ((((byte)[2]) >> 5) & 1)] \ & (1u << (((byte)[2]) & 0x1F))) -#define UTF8_GET_NAMING(pages, p, n) \ - ((n) == 2 \ - ? UTF8_GET_NAMING2(pages, (const unsigned char *)(p)) \ - : ((n) == 3 ? UTF8_GET_NAMING3(pages, (const unsigned char *)(p)) : 0)) - /* Detection of invalid UTF-8 sequences is based on Table 3.1B of Unicode 3.2: http://www.unicode.org/unicode/reports/tr28/ with the additional restriction of not allowing the Unicode diff --git a/Modules/expat/xmltok_impl.c b/Modules/expat/xmltok_impl.c index 0430591b42636..4072b06497d1c 100644 --- a/Modules/expat/xmltok_impl.c +++ b/Modules/expat/xmltok_impl.c @@ -10,7 +10,7 @@ Copyright (c) 2000 Clark Cooper Copyright (c) 2002 Fred L. Drake, Jr. Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2018 Benjamin Peterson Copyright (c) 2018 Anton Maklakov @@ -69,7 +69,7 @@ case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ - if (! IS_NAME_CHAR(enc, ptr, n)) { \ + if (IS_INVALID_CHAR(enc, ptr, n) || ! IS_NAME_CHAR(enc, ptr, n)) { \ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ @@ -98,7 +98,7 @@ case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ - if (! IS_NMSTRT_CHAR(enc, ptr, n)) { \ + if (IS_INVALID_CHAR(enc, ptr, n) || ! IS_NMSTRT_CHAR(enc, ptr, n)) { \ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ @@ -1142,6 +1142,10 @@ PREFIX(prologTok)(const ENCODING *enc, const char *ptr, const char *end, case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ + if (IS_INVALID_CHAR(enc, ptr, n)) { \ + *nextTokPtr = ptr; \ + return XML_TOK_INVALID; \ + } \ if (IS_NMSTRT_CHAR(enc, ptr, n)) { \ ptr += n; \ tok = XML_TOK_NAME; \ @@ -1270,7 +1274,7 @@ PREFIX(attributeValueTok)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1339,7 +1343,7 @@ PREFIX(entityValueTok)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1518,7 +1522,7 @@ PREFIX(getAtts)(const ENCODING *enc, const char *ptr, int attsMax, state = inName; \ } # define LEAD_CASE(n) \ - case BT_LEAD##n: \ + case BT_LEAD##n: /* NOTE: The encoding has already been validated. */ \ START_NAME ptr += (n - MINBPC(enc)); \ break; LEAD_CASE(2) @@ -1730,7 +1734,7 @@ PREFIX(nameLength)(const ENCODING *enc, const char *ptr) { switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1775,7 +1779,7 @@ PREFIX(updatePosition)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ pos->columnNumber++; \ break; LEAD_CASE(2) From webhook-mailer at python.org Tue Feb 22 23:34:25 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Wed, 23 Feb 2022 04:34:25 -0000 Subject: [Python-checkins] Minor fixes to C API docs (GH-31501) Message-ID: https://github.com/python/cpython/commit/43cf44ddcce6b225f959ea2a53e4817244ca6054 commit: 43cf44ddcce6b225f959ea2a53e4817244ca6054 branch: main author: Jelle Zijlstra committer: JelleZijlstra date: 2022-02-22T20:34:17-08:00 summary: Minor fixes to C API docs (GH-31501) * C API docs: move PyErr_SetImportErrorSubclass docs It was in the section about warnings, but it makes more sense to put it with PyErr_SetImportError. * C API docs: document closeit argument to PyRun_AnyFileExFlags It was already documented for PyRun_SimpleFileExFlags. * textual fixes to unicode docs * Move paragraph about tp_dealloc into tp_dealloc section * __aiter__ returns an async iterator, not an awaitable files: M Doc/c-api/exceptions.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/c-api/veryhigh.rst diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 27feab92dede6..f1807f913aace 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -253,6 +253,14 @@ For convenience, some of these functions will always return a .. versionadded:: 3.3 +.. c:function:: PyObject* PyErr_SetImportErrorSubclass(PyObject *exception, PyObject *msg, PyObject *name, PyObject *path) + + Much like :c:func:`PyErr_SetImportError` but this function allows for + specifying a subclass of :exc:`ImportError` to raise. + + .. versionadded:: 3.6 + + .. c:function:: void PyErr_SyntaxLocationObject(PyObject *filename, int lineno, int col_offset) Set file, line, and offset information for the current exception. If the @@ -320,13 +328,6 @@ an error value). :mod:`warnings` module and the :option:`-W` option in the command line documentation. There is no C API for warning control. -.. c:function:: PyObject* PyErr_SetImportErrorSubclass(PyObject *exception, PyObject *msg, PyObject *name, PyObject *path) - - Much like :c:func:`PyErr_SetImportError` but this function allows for - specifying a subclass of :exc:`ImportError` to raise. - - .. versionadded:: 3.6 - .. c:function:: int PyErr_WarnExplicitObject(PyObject *category, PyObject *message, PyObject *filename, int lineno, PyObject *module, PyObject *registry) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index cd8723efef6bb..3630990368cdf 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -476,7 +476,7 @@ PyObject Slots -------------- The type object structure extends the :c:type:`PyVarObject` structure. The -:attr:`ob_size` field is used for dynamic types (created by :func:`type_new`, +:attr:`ob_size` field is used for dynamic types (created by :func:`type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. type objects) *must* have the :attr:`ob_size` field. @@ -2000,6 +2000,17 @@ and :c:type:`PyType_Type` effectively act as defaults.) For this field to be taken into account (even through inheritance), you must also set the :const:`Py_TPFLAGS_HAVE_FINALIZE` flags bit. + Also, note that, in a garbage collected Python, + :c:member:`~PyTypeObject.tp_dealloc` may be called from + any Python thread, not just the thread which created the object (if the object + becomes part of a refcount cycle, that cycle might be collected by a garbage + collection on any thread). This is not a problem for Python API calls, since + the thread on which tp_dealloc is called will own the Global Interpreter Lock + (GIL). However, if the object being destroyed in turn destroys objects from some + other C or C++ library, care should be taken to ensure that destroying those + objects on the thread which called tp_dealloc will not violate any assumptions + of the library. + **Inheritance:** This field is inherited by subtypes. @@ -2024,17 +2035,6 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionadded:: 3.9 (the field exists since 3.8 but it's only used since 3.9) -Also, note that, in a garbage collected Python, :c:member:`~PyTypeObject.tp_dealloc` may be called from -any Python thread, not just the thread which created the object (if the object -becomes part of a refcount cycle, that cycle might be collected by a garbage -collection on any thread). This is not a problem for Python API calls, since -the thread on which tp_dealloc is called will own the Global Interpreter Lock -(GIL). However, if the object being destroyed in turn destroys objects from some -other C or C++ library, care should be taken to ensure that destroying those -objects on the thread which called tp_dealloc will not violate any assumptions -of the library. - - .. _static-types: Static Types @@ -2440,7 +2440,8 @@ Async Object Structures PyObject *am_aiter(PyObject *self); - Must return an :term:`awaitable` object. See :meth:`__anext__` for details. + Must return an :term:`asynchronous iterator` object. + See :meth:`__anext__` for details. This slot may be set to ``NULL`` if an object does not implement asynchronous iteration protocol. diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index cb7f0a7b85acf..11aa751cdad15 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1003,7 +1003,7 @@ Error handling is set by errors which may also be set to ``NULL`` meaning to use the default handling defined for the codec. Default error handling for all built-in codecs is "strict" (:exc:`ValueError` is raised). -The codecs all use a similar interface. Only deviation from the following +The codecs all use a similar interface. Only deviations from the following generic ones are documented for simplicity. @@ -1171,7 +1171,7 @@ These are the UTF-16 codec APIs: ``1``, any byte order mark is copied to the output (where it will result in either a ``\ufeff`` or a ``\ufffe`` character). - After completion, *\*byteorder* is set to the current byte order at the end + After completion, ``*byteorder`` is set to the current byte order at the end of input data. If *byteorder* is ``NULL``, the codec starts in native order mode. @@ -1302,7 +1302,7 @@ Character Map Codecs This codec is special in that it can be used to implement many different codecs (and this is in fact what was done to obtain most of the standard codecs -included in the :mod:`encodings` package). The codec uses mapping to encode and +included in the :mod:`encodings` package). The codec uses mappings to encode and decode characters. The mapping objects provided must support the :meth:`__getitem__` mapping interface; dictionaries and sequences work well. @@ -1426,7 +1426,7 @@ They all return ``NULL`` or ``-1`` if an exception occurs. .. c:function:: PyObject* PyUnicode_Splitlines(PyObject *s, int keepend) Split a Unicode string at line breaks, returning a list of Unicode strings. - CRLF is considered to be one line break. If *keepend* is ``0``, the Line break + CRLF is considered to be one line break. If *keepend* is ``0``, the line break characters are not included in the resulting strings. diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 3354a2b976f34..5b8735de75e9d 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -75,6 +75,8 @@ the same library that the Python runtime is using. :c:func:`PyRun_SimpleFile`. *filename* is decoded from the filesystem encoding (:func:`sys.getfilesystemencoding`). If *filename* is ``NULL``, this function uses ``"???"`` as the filename. + If *closeit* is true, the file is closed before + ``PyRun_SimpleFileExFlags()`` returns. .. c:function:: int PyRun_SimpleString(const char *command) From webhook-mailer at python.org Tue Feb 22 23:57:19 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 23 Feb 2022 04:57:19 -0000 Subject: [Python-checkins] Minor fixes to C API docs (GH-31501) Message-ID: https://github.com/python/cpython/commit/28d9353305e865f97216140c9d8db764563ac058 commit: 28d9353305e865f97216140c9d8db764563ac058 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-22T20:57:02-08:00 summary: Minor fixes to C API docs (GH-31501) * C API docs: move PyErr_SetImportErrorSubclass docs It was in the section about warnings, but it makes more sense to put it with PyErr_SetImportError. * C API docs: document closeit argument to PyRun_AnyFileExFlags It was already documented for PyRun_SimpleFileExFlags. * textual fixes to unicode docs * Move paragraph about tp_dealloc into tp_dealloc section * __aiter__ returns an async iterator, not an awaitable (cherry picked from commit 43cf44ddcce6b225f959ea2a53e4817244ca6054) Co-authored-by: Jelle Zijlstra files: M Doc/c-api/exceptions.rst M Doc/c-api/typeobj.rst M Doc/c-api/unicode.rst M Doc/c-api/veryhigh.rst diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index dc05659007470..bcd9621681c2e 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -253,6 +253,14 @@ For convenience, some of these functions will always return a .. versionadded:: 3.3 +.. c:function:: PyObject* PyErr_SetImportErrorSubclass(PyObject *exception, PyObject *msg, PyObject *name, PyObject *path) + + Much like :c:func:`PyErr_SetImportError` but this function allows for + specifying a subclass of :exc:`ImportError` to raise. + + .. versionadded:: 3.6 + + .. c:function:: void PyErr_SyntaxLocationObject(PyObject *filename, int lineno, int col_offset) Set file, line, and offset information for the current exception. If the @@ -320,13 +328,6 @@ an error value). :mod:`warnings` module and the :option:`-W` option in the command line documentation. There is no C API for warning control. -.. c:function:: PyObject* PyErr_SetImportErrorSubclass(PyObject *exception, PyObject *msg, PyObject *name, PyObject *path) - - Much like :c:func:`PyErr_SetImportError` but this function allows for - specifying a subclass of :exc:`ImportError` to raise. - - .. versionadded:: 3.6 - .. c:function:: int PyErr_WarnExplicitObject(PyObject *category, PyObject *message, PyObject *filename, int lineno, PyObject *module, PyObject *registry) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index e491c26f981b5..f0dd3923e8085 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -476,7 +476,7 @@ PyObject Slots -------------- The type object structure extends the :c:type:`PyVarObject` structure. The -:attr:`ob_size` field is used for dynamic types (created by :func:`type_new`, +:attr:`ob_size` field is used for dynamic types (created by :func:`type_new`, usually called from a class statement). Note that :c:data:`PyType_Type` (the metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e. type objects) *must* have the :attr:`ob_size` field. @@ -1999,6 +1999,17 @@ and :c:type:`PyType_Type` effectively act as defaults.) For this field to be taken into account (even through inheritance), you must also set the :const:`Py_TPFLAGS_HAVE_FINALIZE` flags bit. + Also, note that, in a garbage collected Python, + :c:member:`~PyTypeObject.tp_dealloc` may be called from + any Python thread, not just the thread which created the object (if the object + becomes part of a refcount cycle, that cycle might be collected by a garbage + collection on any thread). This is not a problem for Python API calls, since + the thread on which tp_dealloc is called will own the Global Interpreter Lock + (GIL). However, if the object being destroyed in turn destroys objects from some + other C or C++ library, care should be taken to ensure that destroying those + objects on the thread which called tp_dealloc will not violate any assumptions + of the library. + **Inheritance:** This field is inherited by subtypes. @@ -2023,17 +2034,6 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionadded:: 3.9 (the field exists since 3.8 but it's only used since 3.9) -Also, note that, in a garbage collected Python, :c:member:`~PyTypeObject.tp_dealloc` may be called from -any Python thread, not just the thread which created the object (if the object -becomes part of a refcount cycle, that cycle might be collected by a garbage -collection on any thread). This is not a problem for Python API calls, since -the thread on which tp_dealloc is called will own the Global Interpreter Lock -(GIL). However, if the object being destroyed in turn destroys objects from some -other C or C++ library, care should be taken to ensure that destroying those -objects on the thread which called tp_dealloc will not violate any assumptions -of the library. - - .. _static-types: Static Types @@ -2439,7 +2439,8 @@ Async Object Structures PyObject *am_aiter(PyObject *self); - Must return an :term:`awaitable` object. See :meth:`__anext__` for details. + Must return an :term:`asynchronous iterator` object. + See :meth:`__anext__` for details. This slot may be set to ``NULL`` if an object does not implement asynchronous iteration protocol. diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index 343eaf1444df6..a08cebcc000d0 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1008,7 +1008,7 @@ Error handling is set by errors which may also be set to ``NULL`` meaning to use the default handling defined for the codec. Default error handling for all built-in codecs is "strict" (:exc:`ValueError` is raised). -The codecs all use a similar interface. Only deviation from the following +The codecs all use a similar interface. Only deviations from the following generic ones are documented for simplicity. @@ -1225,7 +1225,7 @@ These are the UTF-16 codec APIs: ``1``, any byte order mark is copied to the output (where it will result in either a ``\ufeff`` or a ``\ufffe`` character). - After completion, *\*byteorder* is set to the current byte order at the end + After completion, ``*byteorder`` is set to the current byte order at the end of input data. If *byteorder* is ``NULL``, the codec starts in native order mode. @@ -1443,7 +1443,7 @@ Character Map Codecs This codec is special in that it can be used to implement many different codecs (and this is in fact what was done to obtain most of the standard codecs -included in the :mod:`encodings` package). The codec uses mapping to encode and +included in the :mod:`encodings` package). The codec uses mappings to encode and decode characters. The mapping objects provided must support the :meth:`__getitem__` mapping interface; dictionaries and sequences work well. @@ -1605,7 +1605,7 @@ They all return ``NULL`` or ``-1`` if an exception occurs. .. c:function:: PyObject* PyUnicode_Splitlines(PyObject *s, int keepend) Split a Unicode string at line breaks, returning a list of Unicode strings. - CRLF is considered to be one line break. If *keepend* is ``0``, the Line break + CRLF is considered to be one line break. If *keepend* is ``0``, the line break characters are not included in the resulting strings. diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 3354a2b976f34..5b8735de75e9d 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -75,6 +75,8 @@ the same library that the Python runtime is using. :c:func:`PyRun_SimpleFile`. *filename* is decoded from the filesystem encoding (:func:`sys.getfilesystemencoding`). If *filename* is ``NULL``, this function uses ``"???"`` as the filename. + If *closeit* is true, the file is closed before + ``PyRun_SimpleFileExFlags()`` returns. .. c:function:: int PyRun_SimpleString(const char *command) From webhook-mailer at python.org Wed Feb 23 00:14:43 2022 From: webhook-mailer at python.org (ericvsmith) Date: Wed, 23 Feb 2022 05:14:43 -0000 Subject: [Python-checkins] bpo-46757: Add a test to verify dataclass's __post_init__ isn't being automatically added. (GH-31523) Message-ID: https://github.com/python/cpython/commit/288af845a32fd2a92e3b49738faf8f2de6a7bf7c commit: 288af845a32fd2a92e3b49738faf8f2de6a7bf7c branch: main author: Eric V. Smith committer: ericvsmith date: 2022-02-23T00:14:35-05:00 summary: bpo-46757: Add a test to verify dataclass's __post_init__ isn't being automatically added. (GH-31523) files: M Lib/test/test_dataclasses.py diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index 69e76850830091..2f37ecdfca6b4d 100644 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -1016,6 +1016,65 @@ def __post_init__(cls): self.assertEqual((c.x, c.y), (3, 4)) self.assertTrue(C.flag) + def test_post_init_not_auto_added(self): + # See bpo-46757, which had proposed always adding __post_init__. As + # Raymond Hettinger pointed out, that would be a breaking change. So, + # add a test to make sure that the current behavior doesn't change. + + @dataclass + class A0: + pass + + @dataclass + class B0: + b_called: bool = False + def __post_init__(self): + self.b_called = True + + @dataclass + class C0(A0, B0): + c_called: bool = False + def __post_init__(self): + super().__post_init__() + self.c_called = True + + # Since A0 has no __post_init__, and one wasn't automatically added + # (because that's the rule: it's never added by @dataclass, it's only + # the class author that can add it), then B0.__post_init__ is called. + # Verify that. + c = C0() + self.assertTrue(c.b_called) + self.assertTrue(c.c_called) + + ###################################### + # Now, the same thing, except A1 defines __post_init__. + @dataclass + class A1: + def __post_init__(self): + pass + + @dataclass + class B1: + b_called: bool = False + def __post_init__(self): + self.b_called = True + + @dataclass + class C1(A1, B1): + c_called: bool = False + def __post_init__(self): + super().__post_init__() + self.c_called = True + + # This time, B1.__post_init__ isn't being called. This mimics what + # would happen if A1.__post_init__ had been automatically added, + # instead of manually added as we see here. This test isn't really + # needed, but I'm including it just to demonstrate the changed + # behavior when A1 does define __post_init__. + c = C1() + self.assertFalse(c.b_called) + self.assertTrue(c.c_called) + def test_class_var(self): # Make sure ClassVars are ignored in __init__, __repr__, etc. @dataclass From webhook-mailer at python.org Wed Feb 23 06:01:05 2022 From: webhook-mailer at python.org (markshannon) Date: Wed, 23 Feb 2022 11:01:05 -0000 Subject: [Python-checkins] bpo-46329: Fix test failure when `Py_STATS` is enabled (GH-31511) Message-ID: https://github.com/python/cpython/commit/424023efee5b21567b4725015ef143b627112e3c commit: 424023efee5b21567b4725015ef143b627112e3c branch: main author: Brandt Bucher committer: markshannon date: 2022-02-23T11:00:28Z summary: bpo-46329: Fix test failure when `Py_STATS` is enabled (GH-31511) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-22-14-03-56.bpo-46329.RX_AzJ.rst M Python/specialize.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-22-14-03-56.bpo-46329.RX_AzJ.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-14-03-56.bpo-46329.RX_AzJ.rst new file mode 100644 index 0000000000000..8d1d5027ca9cd --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-14-03-56.bpo-46329.RX_AzJ.rst @@ -0,0 +1 @@ +Fix specialization stats gathering for :opcode:`PRECALL` instructions. diff --git a/Python/specialize.c b/Python/specialize.c index b46f701428923..91010a5be3c06 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -160,6 +160,7 @@ _Py_GetSpecializationStats(void) { err += add_stat_dict(stats, BINARY_OP, "binary_op"); err += add_stat_dict(stats, COMPARE_OP, "compare_op"); err += add_stat_dict(stats, UNPACK_SEQUENCE, "unpack_sequence"); + err += add_stat_dict(stats, PRECALL, "precall"); if (err < 0) { Py_DECREF(stats); return NULL; @@ -180,8 +181,6 @@ print_spec_stats(FILE *out, OpcodeStats *stats) /* Mark some opcodes as specializable for stats, * even though we don't specialize them yet. */ fprintf(out, "opcode[%d].specializable : 1\n", FOR_ITER); - fprintf(out, "opcode[%d].specializable : 1\n", PRECALL); - fprintf(out, "opcode[%d].specializable : 1\n", UNPACK_SEQUENCE); for (int i = 0; i < 256; i++) { if (adaptive_opcodes[i]) { fprintf(out, "opcode[%d].specializable : 1\n", i); From webhook-mailer at python.org Wed Feb 23 06:06:36 2022 From: webhook-mailer at python.org (markshannon) Date: Wed, 23 Feb 2022 11:06:36 -0000 Subject: [Python-checkins] bpo-45885: Don't un-adapt `COMPARE_OP` when collecting stats (GH-31516) Message-ID: https://github.com/python/cpython/commit/375a56bd4015596c0cf44129c8842a1fe7199785 commit: 375a56bd4015596c0cf44129c8842a1fe7199785 branch: main author: Brandt Bucher committer: markshannon date: 2022-02-23T11:06:25Z summary: bpo-45885: Don't un-adapt `COMPARE_OP` when collecting stats (GH-31516) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-22-15-48-32.bpo-45885.W2vkaI.rst M Python/specialize.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-22-15-48-32.bpo-45885.W2vkaI.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-15-48-32.bpo-45885.W2vkaI.rst new file mode 100644 index 0000000000000..4339f501fd3fb --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-15-48-32.bpo-45885.W2vkaI.rst @@ -0,0 +1 @@ +Don't un-adapt :opcode:`COMPARE_OP` when collecting specialization stats. diff --git a/Python/specialize.c b/Python/specialize.c index 91010a5be3c06..0fc992d30bdc8 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -2013,9 +2013,15 @@ _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, int op = adaptive->original_oparg; int next_opcode = _Py_OPCODE(instr[1]); if (next_opcode != POP_JUMP_IF_FALSE && next_opcode != POP_JUMP_IF_TRUE) { - // Can't ever combine, so don't don't bother being adaptive. - SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_COMPARE_OP_NOT_FOLLOWED_BY_COND_JUMP); + // Can't ever combine, so don't don't bother being adaptive (unless + // we're collecting stats, where it's more important to get accurate hit + // counts for the unadaptive version and each of the different failure + // types): +#ifndef Py_STATS *instr = _Py_MAKECODEUNIT(COMPARE_OP, adaptive->original_oparg); + return; +#endif + SPECIALIZATION_FAIL(COMPARE_OP, SPEC_FAIL_COMPARE_OP_NOT_FOLLOWED_BY_COND_JUMP); goto failure; } assert(op <= Py_GE); From webhook-mailer at python.org Wed Feb 23 07:05:26 2022 From: webhook-mailer at python.org (asvetlov) Date: Wed, 23 Feb 2022 12:05:26 -0000 Subject: [Python-checkins] Inherit asyncio proactor datagram transport from asyncio.DatagramTransport (GH-31512) (GH-31514) Message-ID: https://github.com/python/cpython/commit/6717edcf84077440a8839f2086cc5c5e2fc779c5 commit: 6717edcf84077440a8839f2086cc5c5e2fc779c5 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: asvetlov date: 2022-02-23T14:04:49+02:00 summary: Inherit asyncio proactor datagram transport from asyncio.DatagramTransport (GH-31512) (GH-31514) (cherry picked from commit cff4d5c5d29528299ec1ac5b3b3a6f7735577c01) Co-authored-by: Andrew Svetlov Co-authored-by: Andrew Svetlov files: A Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst M Lib/asyncio/proactor_events.py M Lib/test/test_asyncio/test_proactor_events.py diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py index 45c11ee4b487e..e551896395b84 100644 --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -452,7 +452,8 @@ def _pipe_closed(self, fut): self.close() -class _ProactorDatagramTransport(_ProactorBasePipeTransport): +class _ProactorDatagramTransport(_ProactorBasePipeTransport, + transports.DatagramTransport): max_size = 256 * 1024 def __init__(self, loop, sock, protocol, address=None, waiter=None, extra=None): diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py index 4c8906d531ce5..c024477e3baba 100644 --- a/Lib/test/test_asyncio/test_proactor_events.py +++ b/Lib/test/test_asyncio/test_proactor_events.py @@ -838,6 +838,7 @@ def datagram_transport(self): def test_make_datagram_transport(self): tr = self.datagram_transport() self.assertIsInstance(tr, _ProactorDatagramTransport) + self.assertIsInstance(tr, asyncio.DatagramTransport) close_transport(tr) def test_datagram_loop_writing(self): diff --git a/Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst b/Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst new file mode 100644 index 0000000000000..be223ddd58ba1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-23-00-55-59.bpo-44886.I40Mbr.rst @@ -0,0 +1,2 @@ +Inherit asyncio proactor datagram transport from +:class:`asyncio.DatagramTransport`. From webhook-mailer at python.org Wed Feb 23 12:16:41 2022 From: webhook-mailer at python.org (vstinner) Date: Wed, 23 Feb 2022 17:16:41 -0000 Subject: [Python-checkins] bpo-45412: Add _PY_SHORT_FLOAT_REPR macro (GH-31171) Message-ID: https://github.com/python/cpython/commit/9bbdde218005f552304d9954bb97e3f9185edded commit: 9bbdde218005f552304d9954bb97e3f9185edded branch: main author: Victor Stinner committer: vstinner date: 2022-02-23T18:16:23+01:00 summary: bpo-45412: Add _PY_SHORT_FLOAT_REPR macro (GH-31171) Remove the HAVE_PY_SET_53BIT_PRECISION macro (moved to the internal C API). * Move HAVE_PY_SET_53BIT_PRECISION macro to pycore_pymath.h. * Replace PY_NO_SHORT_FLOAT_REPR macro with _PY_SHORT_FLOAT_REPR macro which is always defined. gcc -Wundef emits a warning when using _PY_SHORT_FLOAT_REPR but the macro is not defined, if pycore_pymath.h include was forgotten. files: A Misc/NEWS.d/next/C API/2022-02-06-20-14-21.bpo-45412.XJVaGW.rst M Doc/whatsnew/3.11.rst M Include/internal/pycore_dtoa.h M Include/internal/pycore_pymath.h M Include/pyport.h M Modules/cmathmodule.c M Modules/mathmodule.c M Objects/floatobject.c M Python/dtoa.c M Python/pystrtod.c M Python/sysmodule.c diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 32f021f03ccce..34642e320dd18 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -1019,3 +1019,7 @@ Removed public C API by mistake, it must only be used by Python internally. Use the ``PyTypeObject.tp_members`` member instead. (Contributed by Victor Stinner in :issue:`40170`.) + +* Remove the ``HAVE_PY_SET_53BIT_PRECISION`` macro (moved to the internal C + API). + (Contributed by Victor Stinner in :issue:`45412`.) diff --git a/Include/internal/pycore_dtoa.h b/Include/internal/pycore_dtoa.h index 3faf8cf6b2eef..c77cf6e46cc3c 100644 --- a/Include/internal/pycore_dtoa.h +++ b/Include/internal/pycore_dtoa.h @@ -1,4 +1,3 @@ -#ifndef PY_NO_SHORT_FLOAT_REPR #ifdef __cplusplus extern "C" { #endif @@ -7,6 +6,11 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +#include "pycore_pymath.h" // _PY_SHORT_FLOAT_REPR + + +#if _PY_SHORT_FLOAT_REPR == 1 + /* These functions are used by modules compiled as C extension like math: they must be exported. */ @@ -17,7 +21,8 @@ PyAPI_FUNC(void) _Py_dg_freedtoa(char *s); PyAPI_FUNC(double) _Py_dg_stdnan(int sign); PyAPI_FUNC(double) _Py_dg_infinity(int sign); +#endif // _PY_SHORT_FLOAT_REPR == 1 + #ifdef __cplusplus } #endif -#endif /* !PY_NO_SHORT_FLOAT_REPR */ diff --git a/Include/internal/pycore_pymath.h b/Include/internal/pycore_pymath.h index 395b71452e40a..1f54b3d81325d 100644 --- a/Include/internal/pycore_pymath.h +++ b/Include/internal/pycore_pymath.h @@ -85,19 +85,34 @@ static inline void _Py_ADJUST_ERANGE2(double x, double y) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type)) -//--- Implementation of the HAVE_PY_SET_53BIT_PRECISION macro ------------- -//--- defined in pyport.h ------------------------------------------------- +//--- HAVE_PY_SET_53BIT_PRECISION macro ------------------------------------ // -// Give appropriate definitions for the following three macros: +// The functions _Py_dg_strtod() and _Py_dg_dtoa() in Python/dtoa.c (which are +// required to support the short float repr introduced in Python 3.1) require +// that the floating-point unit that's being used for arithmetic operations on +// C doubles is set to use 53-bit precision. It also requires that the FPU +// rounding mode is round-half-to-even, but that's less often an issue. // -// _Py_SET_53BIT_PRECISION_HEADER : any variable declarations needed to -// use the two macros below. -// _Py_SET_53BIT_PRECISION_START : store original FPU settings, and -// set FPU to 53-bit precision/round-half-to-even -// _Py_SET_53BIT_PRECISION_END : restore original FPU settings +// If your FPU isn't already set to 53-bit precision/round-half-to-even, and +// you want to make use of _Py_dg_strtod() and _Py_dg_dtoa(), then you should: +// +// #define HAVE_PY_SET_53BIT_PRECISION 1 +// +// and also give appropriate definitions for the following three macros: +// +// * _Py_SET_53BIT_PRECISION_HEADER: any variable declarations needed to +// use the two macros below. +// * _Py_SET_53BIT_PRECISION_START: store original FPU settings, and +// set FPU to 53-bit precision/round-half-to-even +// * _Py_SET_53BIT_PRECISION_END: restore original FPU settings +// +// The macros are designed to be used within a single C function: see +// Python/pystrtod.c for an example of their use. + // Get and set x87 control word for gcc/x86 #ifdef HAVE_GCC_ASM_FOR_X87 +#define HAVE_PY_SET_53BIT_PRECISION 1 // Functions defined in Python/pymath.c extern unsigned short _Py_get_387controlword(void); @@ -124,6 +139,7 @@ extern void _Py_set_387controlword(unsigned short); // Get and set x87 control word for VisualStudio/x86. // x87 is not supported in 64-bit or ARM. #if defined(_MSC_VER) && !defined(_WIN64) && !defined(_M_ARM) +#define HAVE_PY_SET_53BIT_PRECISION 1 #include // __control87_2() @@ -150,7 +166,10 @@ extern void _Py_set_387controlword(unsigned short); } while (0) #endif + +// MC68881 #ifdef HAVE_GCC_ASM_FOR_MC68881 +#define HAVE_PY_SET_53BIT_PRECISION 1 #define _Py_SET_53BIT_PRECISION_HEADER \ unsigned int old_fpcr, new_fpcr #define _Py_SET_53BIT_PRECISION_START \ @@ -178,6 +197,36 @@ extern void _Py_set_387controlword(unsigned short); #endif +//--- _PY_SHORT_FLOAT_REPR macro ------------------------------------------- + +// If we can't guarantee 53-bit precision, don't use the code +// in Python/dtoa.c, but fall back to standard code. This +// means that repr of a float will be long (17 significant digits). +// +// Realistically, there are two things that could go wrong: +// +// (1) doubles aren't IEEE 754 doubles, or +// (2) we're on x86 with the rounding precision set to 64-bits +// (extended precision), and we don't know how to change +// the rounding precision. +#if !defined(DOUBLE_IS_LITTLE_ENDIAN_IEEE754) && \ + !defined(DOUBLE_IS_BIG_ENDIAN_IEEE754) && \ + !defined(DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754) +# define _PY_SHORT_FLOAT_REPR 0 +#endif + +// Double rounding is symptomatic of use of extended precision on x86. +// If we're seeing double rounding, and we don't have any mechanism available +// for changing the FPU rounding precision, then don't use Python/dtoa.c. +#if defined(X87_DOUBLE_ROUNDING) && !defined(HAVE_PY_SET_53BIT_PRECISION) +# define _PY_SHORT_FLOAT_REPR 0 +#endif + +#ifndef _PY_SHORT_FLOAT_REPR +# define _PY_SHORT_FLOAT_REPR 1 +#endif + + #ifdef __cplusplus } #endif diff --git a/Include/pyport.h b/Include/pyport.h index d27b3dde11659..62ac0989d3f15 100644 --- a/Include/pyport.h +++ b/Include/pyport.h @@ -312,61 +312,6 @@ extern "C" { #define Py_SAFE_DOWNCAST(VALUE, WIDE, NARROW) (NARROW)(VALUE) #endif -/* The functions _Py_dg_strtod and _Py_dg_dtoa in Python/dtoa.c (which are - * required to support the short float repr introduced in Python 3.1) require - * that the floating-point unit that's being used for arithmetic operations - * on C doubles is set to use 53-bit precision. It also requires that the - * FPU rounding mode is round-half-to-even, but that's less often an issue. - * - * If your FPU isn't already set to 53-bit precision/round-half-to-even, and - * you want to make use of _Py_dg_strtod and _Py_dg_dtoa, then you should - * - * #define HAVE_PY_SET_53BIT_PRECISION 1 - * - * The macros are designed to be used within a single C function: see - * Python/pystrtod.c for an example of their use. - */ - -// HAVE_PY_SET_53BIT_PRECISION macro must be kept in sync with pycore_pymath.h -#ifdef HAVE_GCC_ASM_FOR_X87 - // Get and set x87 control word for gcc/x86 -# define HAVE_PY_SET_53BIT_PRECISION 1 -#endif -#if defined(_MSC_VER) && !defined(_WIN64) && !defined(_M_ARM) - // Get and set x87 control word for VisualStudio/x86. - // x87 not supported in 64-bit or ARM. -# define HAVE_PY_SET_53BIT_PRECISION 1 -#endif -#ifdef HAVE_GCC_ASM_FOR_MC68881 -# define HAVE_PY_SET_53BIT_PRECISION 1 -#endif - - -/* If we can't guarantee 53-bit precision, don't use the code - in Python/dtoa.c, but fall back to standard code. This - means that repr of a float will be long (17 sig digits). - - Realistically, there are two things that could go wrong: - - (1) doubles aren't IEEE 754 doubles, or - (2) we're on x86 with the rounding precision set to 64-bits - (extended precision), and we don't know how to change - the rounding precision. - */ - -#if !defined(DOUBLE_IS_LITTLE_ENDIAN_IEEE754) && \ - !defined(DOUBLE_IS_BIG_ENDIAN_IEEE754) && \ - !defined(DOUBLE_IS_ARM_MIXED_ENDIAN_IEEE754) -# define PY_NO_SHORT_FLOAT_REPR -#endif - -/* double rounding is symptomatic of use of extended precision on x86. If - we're seeing double rounding, and we don't have any mechanism available for - changing the FPU rounding precision, then don't use Python/dtoa.c. */ -#if defined(X87_DOUBLE_ROUNDING) && !defined(HAVE_PY_SET_53BIT_PRECISION) -# define PY_NO_SHORT_FLOAT_REPR -#endif - /* Py_DEPRECATED(version) * Declare a variable, type, or function deprecated. diff --git a/Misc/NEWS.d/next/C API/2022-02-06-20-14-21.bpo-45412.XJVaGW.rst b/Misc/NEWS.d/next/C API/2022-02-06-20-14-21.bpo-45412.XJVaGW.rst new file mode 100644 index 0000000000000..5c0cde1f2b087 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-02-06-20-14-21.bpo-45412.XJVaGW.rst @@ -0,0 +1,2 @@ +Remove the ``HAVE_PY_SET_53BIT_PRECISION`` macro (moved to the internal C API). +Patch by Victor Stinner. diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c index 281d3937e2656..c0c0c353d196c 100644 --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -7,7 +7,8 @@ #endif #include "Python.h" -#include "pycore_dtoa.h" +#include "pycore_pymath.h" // _PY_SHORT_FLOAT_REPR +#include "pycore_dtoa.h" // _Py_dg_stdnan() /* we need DBL_MAX, DBL_MIN, DBL_EPSILON, DBL_MANT_DIG and FLT_RADIX from float.h. We assume that FLT_RADIX is either 2 or 16. */ #include @@ -89,14 +90,14 @@ else { /* Constants cmath.inf, cmath.infj, cmath.nan, cmath.nanj. cmath.nan and cmath.nanj are defined only when either - PY_NO_SHORT_FLOAT_REPR is *not* defined (which should be + _PY_SHORT_FLOAT_REPR is 1 (which should be the most common situation on machines using an IEEE 754 representation), or Py_NAN is defined. */ static double m_inf(void) { -#ifndef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 1 return _Py_dg_infinity(0); #else return Py_HUGE_VAL; @@ -112,12 +113,12 @@ c_infj(void) return r; } -#if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN) +#if _PY_SHORT_FLOAT_REPR == 1 || defined(Py_NAN) static double m_nan(void) { -#ifndef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 1 return _Py_dg_stdnan(0); #else return Py_NAN; @@ -1281,7 +1282,7 @@ cmath_exec(PyObject *mod) PyComplex_FromCComplex(c_infj())) < 0) { return -1; } -#if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN) +#if _PY_SHORT_FLOAT_REPR == 1 || defined(Py_NAN) if (PyModule_AddObject(mod, "nan", PyFloat_FromDouble(m_nan())) < 0) { return -1; } @@ -1426,4 +1427,4 @@ PyMODINIT_FUNC PyInit_cmath(void) { return PyModuleDef_Init(&cmathmodule); -} \ No newline at end of file +} diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 721c9a63d51b2..24ae1460e5a89 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -62,6 +62,7 @@ raised for division by zero and mod by zero. #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_dtoa.h" // _Py_dg_infinity() #include "pycore_long.h" // _PyLong_GetZero() +#include "pycore_pymath.h" // _PY_SHORT_FLOAT_REPR /* For DBL_EPSILON in _math.h */ #include /* For _Py_log1p with workarounds for buggy handling of zeros. */ @@ -272,7 +273,7 @@ lanczos_sum(double x) static double m_inf(void) { -#ifndef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 1 return _Py_dg_infinity(0); #else return Py_HUGE_VAL; @@ -282,12 +283,12 @@ m_inf(void) /* Constant nan value, generated in the same way as float('nan'). */ /* We don't currently assume that Py_NAN is defined everywhere. */ -#if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN) +#if _PY_SHORT_FLOAT_REPR == 1 || defined(Py_NAN) static double m_nan(void) { -#ifndef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 1 return _Py_dg_stdnan(0); #else return Py_NAN; @@ -3837,7 +3838,7 @@ math_exec(PyObject *module) if (PyModule_AddObject(module, "inf", PyFloat_FromDouble(m_inf())) < 0) { return -1; } -#if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN) +#if _PY_SHORT_FLOAT_REPR == 1 || defined(Py_NAN) if (PyModule_AddObject(module, "nan", PyFloat_FromDouble(m_nan())) < 0) { return -1; } diff --git a/Objects/floatobject.c b/Objects/floatobject.c index 79fbdabce9608..64d4c3e5a8cef 100644 --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -10,7 +10,7 @@ #include "pycore_interp.h" // _PyInterpreterState.float_state #include "pycore_long.h" // _PyLong_GetOne() #include "pycore_object.h" // _PyObject_Init() -#include "pycore_pymath.h" // _Py_ADJUST_ERANGE1() +#include "pycore_pymath.h" // _PY_SHORT_FLOAT_REPR #include "pycore_pystate.h" // _PyInterpreterState_GET() #include "pycore_structseq.h" // _PyStructSequence_FiniType() @@ -932,7 +932,7 @@ float___ceil___impl(PyObject *self) ndigits <= 323). Returns a Python float, or sets a Python error and returns NULL on failure (OverflowError and memory errors are possible). */ -#ifndef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 1 /* version of double_round that uses the correctly-rounded string<->double conversions from Python/dtoa.c */ @@ -989,7 +989,7 @@ double_round(double x, int ndigits) { return result; } -#else /* PY_NO_SHORT_FLOAT_REPR */ +#else // _PY_SHORT_FLOAT_REPR == 0 /* fallback version, to be used when correctly rounded binary<->decimal conversions aren't available */ @@ -1039,7 +1039,7 @@ double_round(double x, int ndigits) { return PyFloat_FromDouble(z); } -#endif /* PY_NO_SHORT_FLOAT_REPR */ +#endif // _PY_SHORT_FLOAT_REPR == 0 /* round a Python float v to the closest multiple of 10**-ndigits */ @@ -2479,7 +2479,7 @@ _PyFloat_Unpack2(const unsigned char *p, int le) f |= *p; if (e == 0x1f) { -#ifdef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 0 if (f == 0) { /* Infinity */ return sign ? -Py_HUGE_VAL : Py_HUGE_VAL; @@ -2494,9 +2494,9 @@ _PyFloat_Unpack2(const unsigned char *p, int le) "can't unpack IEEE 754 NaN " "on platform that does not support NaNs"); return -1; -#endif /* #ifdef Py_NAN */ +#endif // !defined(Py_NAN) } -#else +#else // _PY_SHORT_FLOAT_REPR == 1 if (f == 0) { /* Infinity */ return _Py_dg_infinity(sign); @@ -2505,7 +2505,7 @@ _PyFloat_Unpack2(const unsigned char *p, int le) /* NaN */ return _Py_dg_stdnan(sign); } -#endif /* #ifdef PY_NO_SHORT_FLOAT_REPR */ +#endif // _PY_SHORT_FLOAT_REPR == 1 } x = (double)f / 1024.0; diff --git a/Python/dtoa.c b/Python/dtoa.c index 6c44f68965176..733e70bc79169 100644 --- a/Python/dtoa.c +++ b/Python/dtoa.c @@ -118,12 +118,12 @@ /* Linking of Python's #defines to Gay's #defines starts here. */ #include "Python.h" -#include "pycore_dtoa.h" +#include "pycore_dtoa.h" // _PY_SHORT_FLOAT_REPR #include // exit() -/* if PY_NO_SHORT_FLOAT_REPR is defined, then don't even try to compile +/* if _PY_SHORT_FLOAT_REPR == 0, then don't even try to compile the following code */ -#ifndef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 1 #include "float.h" @@ -2857,4 +2857,4 @@ _Py_dg_dtoa(double dd, int mode, int ndigits, } #endif -#endif /* PY_NO_SHORT_FLOAT_REPR */ +#endif // _PY_SHORT_FLOAT_REPR == 1 diff --git a/Python/pystrtod.c b/Python/pystrtod.c index ab5814de21ba5..7469d6259d673 100644 --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -1,8 +1,8 @@ /* -*- Mode: C; c-file-style: "python" -*- */ #include -#include "pycore_dtoa.h" -#include "pycore_pymath.h" // _Py_SET_53BIT_PRECISION_START +#include "pycore_dtoa.h" // _Py_dg_strtod() +#include "pycore_pymath.h" // _PY_SHORT_FLOAT_REPR #include /* Case-insensitive string match used for nan and inf detection; t should be @@ -24,7 +24,7 @@ case_insensitive_match(const char *s, const char *t) the successfully parsed portion of the string. On failure, return -1.0 and set *endptr to point to the start of the string. */ -#ifndef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 1 double _Py_parse_inf_or_nan(const char *p, char **endptr) @@ -127,7 +127,7 @@ _Py_parse_inf_or_nan(const char *p, char **endptr) * Return value: the #gdouble value. **/ -#ifndef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 1 static double _PyOS_ascii_strtod(const char *nptr, char **endptr) @@ -441,7 +441,7 @@ _Py_string_to_number_with_underscores( return NULL; } -#ifdef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 0 /* Given a string that may have a decimal point in the current locale, change it back to a dot. Since the string cannot get @@ -942,7 +942,7 @@ char * PyOS_double_to_string(double val, return buf; } -#else +#else // _PY_SHORT_FLOAT_REPR == 1 /* _Py_dg_dtoa is available. */ @@ -1305,4 +1305,4 @@ char * PyOS_double_to_string(double val, flags & Py_DTSF_ALT, float_strings, type); } -#endif /* ifdef PY_NO_SHORT_FLOAT_REPR */ +#endif // _PY_SHORT_FLOAT_REPR == 1 diff --git a/Python/sysmodule.c b/Python/sysmodule.c index e23b879c1ab2b..342e48ed4222b 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -25,6 +25,7 @@ Data members: #include "pycore_pathconfig.h" // _PyPathConfig_ComputeSysPath0() #include "pycore_pyerrors.h" // _PyErr_Fetch() #include "pycore_pylifecycle.h" // _PyErr_WriteUnraisableDefaultHook() +#include "pycore_pymath.h" // _PY_SHORT_FLOAT_REPR #include "pycore_pymem.h" // _PyMem_SetDefaultAllocator() #include "pycore_pystate.h" // _PyThreadState_GET() #include "pycore_structseq.h" // _PyStructSequence_InitType() @@ -2837,7 +2838,7 @@ _PySys_InitCore(PyThreadState *tstate, PyObject *sysdict) #endif /* float repr style: 0.03 (short) vs 0.029999999999999999 (legacy) */ -#ifndef PY_NO_SHORT_FLOAT_REPR +#if _PY_SHORT_FLOAT_REPR == 1 SET_SYS_FROM_STRING("float_repr_style", "short"); #else SET_SYS_FROM_STRING("float_repr_style", "legacy"); From webhook-mailer at python.org Wed Feb 23 12:59:37 2022 From: webhook-mailer at python.org (benjaminp) Date: Wed, 23 Feb 2022 17:59:37 -0000 Subject: [Python-checkins] closes bpo-46736: SimpleHTTPRequestHandler now uses HTML5. (GH-31533) Message-ID: https://github.com/python/cpython/commit/0bb40a42d71873ea267aace8c92a02d66fe36dc2 commit: 0bb40a42d71873ea267aace8c92a02d66fe36dc2 branch: main author: Dong-hee Na committer: benjaminp date: 2022-02-23T09:59:32-08:00 summary: closes bpo-46736: SimpleHTTPRequestHandler now uses HTML5. (GH-31533) Co-authored-by: Oleg Iarygin files: A Misc/NEWS.d/next/Library/2022-02-24-01-49-38.bpo-46736.NJcoWO.rst M Lib/http/server.py diff --git a/Lib/http/server.py b/Lib/http/server.py index 194a503c45f75..a6100d4829751 100644 --- a/Lib/http/server.py +++ b/Lib/http/server.py @@ -109,11 +109,10 @@ # Default error message template DEFAULT_ERROR_MESSAGE = """\ - - + + - + Error response @@ -777,14 +776,13 @@ def list_directory(self, path): displaypath = urllib.parse.unquote(path) displaypath = html.escape(displaypath, quote=False) enc = sys.getfilesystemencoding() - title = 'Directory listing for %s' % displaypath - r.append('') - r.append('\n') - r.append('' % enc) - r.append('%s\n' % title) - r.append('\n

    %s

    ' % title) + title = f'Directory listing for {displaypath}' + r.append('') + r.append('') + r.append('') + r.append(f'') + r.append(f'{title}\n') + r.append(f'\n

    {title}

    ') r.append('
    \n
      ') for name in list: fullname = os.path.join(path, name) diff --git a/Misc/NEWS.d/next/Library/2022-02-24-01-49-38.bpo-46736.NJcoWO.rst b/Misc/NEWS.d/next/Library/2022-02-24-01-49-38.bpo-46736.NJcoWO.rst new file mode 100644 index 0000000000000..fca7780b82e9d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-24-01-49-38.bpo-46736.NJcoWO.rst @@ -0,0 +1,2 @@ +:class:`~http.server.SimpleHTTPRequestHandler` now uses HTML5 grammar. Patch +by Dong-hee Na. From webhook-mailer at python.org Wed Feb 23 13:49:31 2022 From: webhook-mailer at python.org (vstinner) Date: Wed, 23 Feb 2022 18:49:31 -0000 Subject: [Python-checkins] bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535) Message-ID: https://github.com/python/cpython/commit/78859e58e4e016286e648d1dc155e0f6cebfa6ff commit: 78859e58e4e016286e648d1dc155e0f6cebfa6ff branch: main author: Victor Stinner committer: vstinner date: 2022-02-23T19:49:21+01:00 summary: bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535) Fix PyFrame_GetBack() and PyFrame_GetCode() return type in the documentation. files: M Doc/c-api/reflection.rst diff --git a/Doc/c-api/reflection.rst b/Doc/c-api/reflection.rst index 64ce4d1d0c34d..fe7741a2d7a9f 100644 --- a/Doc/c-api/reflection.rst +++ b/Doc/c-api/reflection.rst @@ -31,7 +31,7 @@ Reflection See also :c:func:`PyThreadState_GetFrame`. -.. c:function:: int PyFrame_GetBack(PyFrameObject *frame) +.. c:function:: PyFrameObject* PyFrame_GetBack(PyFrameObject *frame) Get the *frame* next outer frame. @@ -42,7 +42,7 @@ Reflection .. versionadded:: 3.9 -.. c:function:: int PyFrame_GetCode(PyFrameObject *frame) +.. c:function:: PyCodeObject* PyFrame_GetCode(PyFrameObject *frame) Get the *frame* code. From webhook-mailer at python.org Wed Feb 23 13:53:34 2022 From: webhook-mailer at python.org (brandtbucher) Date: Wed, 23 Feb 2022 18:53:34 -0000 Subject: [Python-checkins] bpo-44337: Shrink the LOAD_ATTR/STORE_ATTR caches (GH-31517) Message-ID: https://github.com/python/cpython/commit/281ea9c39146a00cdf3fa2b3d0be60e2a39278ce commit: 281ea9c39146a00cdf3fa2b3d0be60e2a39278ce branch: main author: Brandt Bucher committer: brandtbucher date: 2022-02-23T10:53:24-08:00 summary: bpo-44337: Shrink the LOAD_ATTR/STORE_ATTR caches (GH-31517) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-22-17-19-45.bpo-44337.XA-egu.rst M Include/internal/pycore_code.h M Python/ceval.c M Python/specialize.c diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index d7a514d8dc988..88bbefc2474b4 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -23,7 +23,7 @@ typedef struct { typedef struct { uint32_t tp_version; - uint32_t dk_version_or_hint; + uint32_t dk_version; } _PyAttrCache; typedef struct { diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-22-17-19-45.bpo-44337.XA-egu.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-17-19-45.bpo-44337.XA-egu.rst new file mode 100644 index 0000000000000..5037aa1339864 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-17-19-45.bpo-44337.XA-egu.rst @@ -0,0 +1,2 @@ +Reduce the memory usage of specialized :opcode:`LOAD_ATTR` and +:opcode:`STORE_ATTR` instructions. diff --git a/Python/ceval.c b/Python/ceval.c index 308271b8953de..807757019020a 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1450,11 +1450,10 @@ eval_frame_handle_pending(PyThreadState *tstate) #define LOAD_MODULE_ATTR_OR_METHOD(attr_or_method) \ SpecializedCacheEntry *caches = GET_CACHE(); \ _PyAdaptiveEntry *cache0 = &caches[0].adaptive; \ - _PyAttrCache *cache1 = &caches[-1].attr; \ DEOPT_IF(!PyModule_CheckExact(owner), LOAD_##attr_or_method); \ PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict; \ assert(dict != NULL); \ - DEOPT_IF(dict->ma_keys->dk_version != cache1->dk_version_or_hint, \ + DEOPT_IF(dict->ma_keys->dk_version != cache0->version, \ LOAD_##attr_or_method); \ assert(dict->ma_keys->dk_kind == DICT_KEYS_UNICODE); \ assert(cache0->index < dict->ma_keys->dk_nentries); \ @@ -3452,9 +3451,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyTypeObject *tp = Py_TYPE(owner); SpecializedCacheEntry *caches = GET_CACHE(); _PyAdaptiveEntry *cache0 = &caches[0].adaptive; - _PyAttrCache *cache1 = &caches[-1].attr; - assert(cache1->tp_version != 0); - DEOPT_IF(tp->tp_version_tag != cache1->tp_version, LOAD_ATTR); + assert(cache0->version != 0); + DEOPT_IF(tp->tp_version_tag != cache0->version, LOAD_ATTR); assert(tp->tp_dictoffset < 0); assert(tp->tp_flags & Py_TPFLAGS_MANAGED_DICT); PyDictValues *values = *_PyObject_ValuesPointer(owner); @@ -3486,15 +3484,14 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyTypeObject *tp = Py_TYPE(owner); SpecializedCacheEntry *caches = GET_CACHE(); _PyAdaptiveEntry *cache0 = &caches[0].adaptive; - _PyAttrCache *cache1 = &caches[-1].attr; - assert(cache1->tp_version != 0); - DEOPT_IF(tp->tp_version_tag != cache1->tp_version, LOAD_ATTR); + assert(cache0->version != 0); + DEOPT_IF(tp->tp_version_tag != cache0->version, LOAD_ATTR); assert(tp->tp_flags & Py_TPFLAGS_MANAGED_DICT); PyDictObject *dict = *(PyDictObject **)_PyObject_ManagedDictPointer(owner); DEOPT_IF(dict == NULL, LOAD_ATTR); assert(PyDict_CheckExact((PyObject *)dict)); PyObject *name = GETITEM(names, cache0->original_oparg); - uint32_t hint = cache1->dk_version_or_hint; + uint16_t hint = cache0->index; DEOPT_IF(hint >= (size_t)dict->ma_keys->dk_nentries, LOAD_ATTR); PyDictKeyEntry *ep = DK_ENTRIES(dict->ma_keys) + hint; DEOPT_IF(ep->me_key != name, LOAD_ATTR); @@ -3514,9 +3511,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyTypeObject *tp = Py_TYPE(owner); SpecializedCacheEntry *caches = GET_CACHE(); _PyAdaptiveEntry *cache0 = &caches[0].adaptive; - _PyAttrCache *cache1 = &caches[-1].attr; - assert(cache1->tp_version != 0); - DEOPT_IF(tp->tp_version_tag != cache1->tp_version, LOAD_ATTR); + assert(cache0->version != 0); + DEOPT_IF(tp->tp_version_tag != cache0->version, LOAD_ATTR); char *addr = (char *)owner + cache0->index; res = *(PyObject **)addr; DEOPT_IF(res == NULL, LOAD_ATTR); @@ -3553,9 +3549,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyTypeObject *tp = Py_TYPE(owner); SpecializedCacheEntry *caches = GET_CACHE(); _PyAdaptiveEntry *cache0 = &caches[0].adaptive; - _PyAttrCache *cache1 = &caches[-1].attr; - assert(cache1->tp_version != 0); - DEOPT_IF(tp->tp_version_tag != cache1->tp_version, STORE_ATTR); + assert(cache0->version != 0); + DEOPT_IF(tp->tp_version_tag != cache0->version, STORE_ATTR); assert(tp->tp_flags & Py_TPFLAGS_MANAGED_DICT); PyDictValues *values = *_PyObject_ValuesPointer(owner); DEOPT_IF(values == NULL, STORE_ATTR); @@ -3581,15 +3576,14 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyTypeObject *tp = Py_TYPE(owner); SpecializedCacheEntry *caches = GET_CACHE(); _PyAdaptiveEntry *cache0 = &caches[0].adaptive; - _PyAttrCache *cache1 = &caches[-1].attr; - assert(cache1->tp_version != 0); - DEOPT_IF(tp->tp_version_tag != cache1->tp_version, STORE_ATTR); + assert(cache0->version != 0); + DEOPT_IF(tp->tp_version_tag != cache0->version, STORE_ATTR); assert(tp->tp_flags & Py_TPFLAGS_MANAGED_DICT); PyDictObject *dict = *(PyDictObject **)_PyObject_ManagedDictPointer(owner); DEOPT_IF(dict == NULL, STORE_ATTR); assert(PyDict_CheckExact((PyObject *)dict)); PyObject *name = GETITEM(names, cache0->original_oparg); - uint32_t hint = cache1->dk_version_or_hint; + uint16_t hint = cache0->index; DEOPT_IF(hint >= (size_t)dict->ma_keys->dk_nentries, STORE_ATTR); PyDictKeyEntry *ep = DK_ENTRIES(dict->ma_keys) + hint; DEOPT_IF(ep->me_key != name, STORE_ATTR); @@ -3616,9 +3610,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyTypeObject *tp = Py_TYPE(owner); SpecializedCacheEntry *caches = GET_CACHE(); _PyAdaptiveEntry *cache0 = &caches[0].adaptive; - _PyAttrCache *cache1 = &caches[-1].attr; - assert(cache1->tp_version != 0); - DEOPT_IF(tp->tp_version_tag != cache1->tp_version, STORE_ATTR); + assert(cache0->version != 0); + DEOPT_IF(tp->tp_version_tag != cache0->version, STORE_ATTR); char *addr = (char *)owner + cache0->index; STAT_INC(STORE_ATTR, hit); STACK_SHRINK(1); @@ -4416,7 +4409,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr assert(self_cls->tp_flags & Py_TPFLAGS_MANAGED_DICT); PyDictObject *dict = *(PyDictObject**)_PyObject_ManagedDictPointer(self); DEOPT_IF(dict != NULL, LOAD_METHOD); - DEOPT_IF(((PyHeapTypeObject *)self_cls)->ht_cached_keys->dk_version != cache1->dk_version_or_hint, LOAD_METHOD); + DEOPT_IF(((PyHeapTypeObject *)self_cls)->ht_cached_keys->dk_version != cache1->dk_version, LOAD_METHOD); STAT_INC(LOAD_METHOD, hit); PyObject *res = cache2->obj; assert(res != NULL); diff --git a/Python/specialize.c b/Python/specialize.c index 0fc992d30bdc8..816cca1074e83 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -57,14 +57,14 @@ static uint8_t adaptive_opcodes[256] = { /* The number of cache entries required for a "family" of instructions. */ static uint8_t cache_requirements[256] = { - [LOAD_ATTR] = 2, /* _PyAdaptiveEntry and _PyAttrCache */ + [LOAD_ATTR] = 1, // _PyAdaptiveEntry [LOAD_GLOBAL] = 2, /* _PyAdaptiveEntry and _PyLoadGlobalCache */ [LOAD_METHOD] = 3, /* _PyAdaptiveEntry, _PyAttrCache and _PyObjectCache */ [BINARY_SUBSCR] = 2, /* _PyAdaptiveEntry, _PyObjectCache */ [STORE_SUBSCR] = 0, [CALL] = 2, /* _PyAdaptiveEntry and _PyObjectCache/_PyCallCache */ [PRECALL] = 2, /* _PyAdaptiveEntry and _PyObjectCache/_PyCallCache */ - [STORE_ATTR] = 2, /* _PyAdaptiveEntry and _PyAttrCache */ + [STORE_ATTR] = 1, // _PyAdaptiveEntry [BINARY_OP] = 1, // _PyAdaptiveEntry [COMPARE_OP] = 1, /* _PyAdaptiveEntry */ [UNPACK_SEQUENCE] = 1, // _PyAdaptiveEntry @@ -638,7 +638,7 @@ initial_counter_value(void) { static int specialize_module_load_attr( PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, - _PyAdaptiveEntry *cache0, _PyAttrCache *cache1, int opcode, + _PyAdaptiveEntry *cache0, int opcode, int opcode_module) { PyModuleObject *m = (PyModuleObject *)owner; @@ -671,7 +671,7 @@ specialize_module_load_attr( SPECIALIZATION_FAIL(opcode, SPEC_FAIL_OUT_OF_VERSIONS); return -1; } - cache1->dk_version_or_hint = keys_version; + cache0->version = keys_version; cache0->index = (uint16_t)index; *instr = _Py_MAKECODEUNIT(opcode_module, _Py_OPARG(*instr)); return 0; @@ -760,7 +760,7 @@ static int specialize_dict_access( PyObject *owner, _Py_CODEUNIT *instr, PyTypeObject *type, DescriptorClassification kind, PyObject *name, - _PyAdaptiveEntry *cache0, _PyAttrCache *cache1, + _PyAdaptiveEntry *cache0, int base_op, int values_op, int hint_op) { assert(kind == NON_OVERRIDING || kind == NON_DESCRIPTOR || kind == ABSENT || @@ -782,7 +782,7 @@ specialize_dict_access( SPECIALIZATION_FAIL(base_op, SPEC_FAIL_OUT_OF_RANGE); return 0; } - cache1->tp_version = type->tp_version_tag; + cache0->version = type->tp_version_tag; cache0->index = (uint16_t)index; *instr = _Py_MAKECODEUNIT(values_op, _Py_OPARG(*instr)); } @@ -795,12 +795,12 @@ specialize_dict_access( PyObject *value = NULL; Py_ssize_t hint = _PyDict_GetItemHint(dict, name, -1, &value); - if (hint != (uint32_t)hint) { + if (hint != (uint16_t)hint) { SPECIALIZATION_FAIL(base_op, SPEC_FAIL_OUT_OF_RANGE); return 0; } - cache1->dk_version_or_hint = (uint32_t)hint; - cache1->tp_version = type->tp_version_tag; + cache0->index = (uint16_t)hint; + cache0->version = type->tp_version_tag; *instr = _Py_MAKECODEUNIT(hint_op, _Py_OPARG(*instr)); } return 1; @@ -810,9 +810,8 @@ int _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache) { _PyAdaptiveEntry *cache0 = &cache->adaptive; - _PyAttrCache *cache1 = &cache[-1].attr; if (PyModule_CheckExact(owner)) { - int err = specialize_module_load_attr(owner, instr, name, cache0, cache1, + int err = specialize_module_load_attr(owner, instr, name, cache0, LOAD_ATTR, LOAD_ATTR_MODULE); if (err) { goto fail; @@ -853,7 +852,7 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, Sp assert(dmem->type == T_OBJECT_EX); assert(offset > 0); cache0->index = (uint16_t)offset; - cache1->tp_version = type->tp_version_tag; + cache0->version = type->tp_version_tag; *instr = _Py_MAKECODEUNIT(LOAD_ATTR_SLOT, _Py_OPARG(*instr)); goto success; } @@ -862,7 +861,7 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, Sp Py_ssize_t offset = offsetof(PyObject, ob_type); assert(offset == (uint16_t)offset); cache0->index = (uint16_t)offset; - cache1->tp_version = type->tp_version_tag; + cache0->version = type->tp_version_tag; *instr = _Py_MAKECODEUNIT(LOAD_ATTR_SLOT, _Py_OPARG(*instr)); goto success; } @@ -883,7 +882,7 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, Sp break; } int err = specialize_dict_access( - owner, instr, type, kind, name, cache0, cache1, + owner, instr, type, kind, name, cache0, LOAD_ATTR, LOAD_ATTR_INSTANCE_VALUE, LOAD_ATTR_WITH_HINT ); if (err < 0) { @@ -908,7 +907,6 @@ int _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache) { _PyAdaptiveEntry *cache0 = &cache->adaptive; - _PyAttrCache *cache1 = &cache[-1].attr; PyTypeObject *type = Py_TYPE(owner); if (PyModule_CheckExact(owner)) { SPECIALIZATION_FAIL(STORE_ATTR, SPEC_FAIL_OVERRIDDEN); @@ -942,7 +940,7 @@ _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, S assert(dmem->type == T_OBJECT_EX); assert(offset > 0); cache0->index = (uint16_t)offset; - cache1->tp_version = type->tp_version_tag; + cache0->version = type->tp_version_tag; *instr = _Py_MAKECODEUNIT(STORE_ATTR_SLOT, _Py_OPARG(*instr)); goto success; } @@ -965,7 +963,7 @@ _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, S } int err = specialize_dict_access( - owner, instr, type, kind, name, cache0, cache1, + owner, instr, type, kind, name, cache0, STORE_ATTR, STORE_ATTR_INSTANCE_VALUE, STORE_ATTR_WITH_HINT ); if (err < 0) { @@ -1066,7 +1064,7 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, PyTypeObject *owner_cls = Py_TYPE(owner); if (PyModule_CheckExact(owner)) { - int err = specialize_module_load_attr(owner, instr, name, cache0, cache1, + int err = specialize_module_load_attr(owner, instr, name, cache0, LOAD_METHOD, LOAD_METHOD_MODULE); if (err) { goto fail; @@ -1111,7 +1109,7 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_OUT_OF_VERSIONS); goto fail; } - cache1->dk_version_or_hint = keys_version; + cache1->dk_version = keys_version; *instr = _Py_MAKECODEUNIT(LOAD_METHOD_CACHED, _Py_OPARG(*instr)); } else { From webhook-mailer at python.org Wed Feb 23 14:13:03 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 23 Feb 2022 19:13:03 -0000 Subject: [Python-checkins] bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535) Message-ID: https://github.com/python/cpython/commit/b0de6299a840a397d4fe3e6c98159d9f258d3295 commit: b0de6299a840a397d4fe3e6c98159d9f258d3295 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-23T11:12:51-08:00 summary: bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535) Fix PyFrame_GetBack() and PyFrame_GetCode() return type in the documentation. (cherry picked from commit 78859e58e4e016286e648d1dc155e0f6cebfa6ff) Co-authored-by: Victor Stinner files: M Doc/c-api/reflection.rst diff --git a/Doc/c-api/reflection.rst b/Doc/c-api/reflection.rst index 64ce4d1d0c34d..fe7741a2d7a9f 100644 --- a/Doc/c-api/reflection.rst +++ b/Doc/c-api/reflection.rst @@ -31,7 +31,7 @@ Reflection See also :c:func:`PyThreadState_GetFrame`. -.. c:function:: int PyFrame_GetBack(PyFrameObject *frame) +.. c:function:: PyFrameObject* PyFrame_GetBack(PyFrameObject *frame) Get the *frame* next outer frame. @@ -42,7 +42,7 @@ Reflection .. versionadded:: 3.9 -.. c:function:: int PyFrame_GetCode(PyFrameObject *frame) +.. c:function:: PyCodeObject* PyFrame_GetCode(PyFrameObject *frame) Get the *frame* code. From webhook-mailer at python.org Wed Feb 23 14:16:15 2022 From: webhook-mailer at python.org (miss-islington) Date: Wed, 23 Feb 2022 19:16:15 -0000 Subject: [Python-checkins] bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535) Message-ID: https://github.com/python/cpython/commit/763e23e11e785162f9d68f83539404af8e863748 commit: 763e23e11e785162f9d68f83539404af8e863748 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-23T11:15:57-08:00 summary: bpo-40421: Fix PyFrame_GetCode() documentation (GH-31535) Fix PyFrame_GetBack() and PyFrame_GetCode() return type in the documentation. (cherry picked from commit 78859e58e4e016286e648d1dc155e0f6cebfa6ff) Co-authored-by: Victor Stinner files: M Doc/c-api/reflection.rst diff --git a/Doc/c-api/reflection.rst b/Doc/c-api/reflection.rst index 9207d86012c8b..44ee009f1f94d 100644 --- a/Doc/c-api/reflection.rst +++ b/Doc/c-api/reflection.rst @@ -31,7 +31,7 @@ Reflection See also :c:func:`PyThreadState_GetFrame`. -.. c:function:: int PyFrame_GetBack(PyFrameObject *frame) +.. c:function:: PyFrameObject* PyFrame_GetBack(PyFrameObject *frame) Get the *frame* next outer frame. @@ -42,7 +42,7 @@ Reflection .. versionadded:: 3.9 -.. c:function:: int PyFrame_GetCode(PyFrameObject *frame) +.. c:function:: PyCodeObject* PyFrame_GetCode(PyFrameObject *frame) Get the *frame* code. From webhook-mailer at python.org Wed Feb 23 14:20:08 2022 From: webhook-mailer at python.org (vstinner) Date: Wed, 23 Feb 2022 19:20:08 -0000 Subject: [Python-checkins] bpo-40421: What's New in Python 3.11: PyFrameObject.f_lasti (GH-31536) Message-ID: https://github.com/python/cpython/commit/8a716bc62c8205bb9deeda17422b7e69204b6897 commit: 8a716bc62c8205bb9deeda17422b7e69204b6897 branch: main author: Victor Stinner committer: vstinner date: 2022-02-23T20:20:03+01:00 summary: bpo-40421: What's New in Python 3.11: PyFrameObject.f_lasti (GH-31536) Suggest replacing PyCode_Addr2Line() with PyFrame_GetLineNumber(). files: M Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 34642e320dd18..9744051ede6bb 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -853,6 +853,8 @@ Porting to Python 3.11 use ``PyObject_GetAttrString((PyObject*)frame, "f_locals")``. * ``f_lasti``: removed, use ``PyObject_GetAttrString((PyObject*)frame, "f_lasti")``. + Code using ``f_lasti`` with ``PyCode_Addr2Line()`` should use + :c:func:`PyFrame_GetLineNumber` instead. The following fields were removed entirely, as they were details of the old implementation: From webhook-mailer at python.org Wed Feb 23 16:16:55 2022 From: webhook-mailer at python.org (asvetlov) Date: Wed, 23 Feb 2022 21:16:55 -0000 Subject: [Python-checkins] build(deps): bump actions/upload-artifact from 2.2.4 to 2.3.1 (GH-30325) Message-ID: https://github.com/python/cpython/commit/6e117e75c35507f170f6a789538136477f9f226d commit: 6e117e75c35507f170f6a789538136477f9f226d branch: main author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> committer: asvetlov date: 2022-02-23T23:16:47+02:00 summary: build(deps): bump actions/upload-artifact from 2.2.4 to 2.3.1 (GH-30325) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2.2.4 to 2.3.1. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v2.2.4...v2.3.1) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> files: M .github/workflows/doc.yml diff --git a/.github/workflows/doc.yml b/.github/workflows/doc.yml index ed44409d118a2..dcc5181a99356 100644 --- a/.github/workflows/doc.yml +++ b/.github/workflows/doc.yml @@ -48,7 +48,7 @@ jobs: - name: 'Build HTML documentation' run: make -C Doc/ PYTHON=../python SPHINXOPTS="-q -W --keep-going -j4" html - name: 'Upload' - uses: actions/upload-artifact at v2.2.4 + uses: actions/upload-artifact at v2.3.1 with: name: doc-html path: Doc/build/html From webhook-mailer at python.org Wed Feb 23 16:51:17 2022 From: webhook-mailer at python.org (ned-deily) Date: Wed, 23 Feb 2022 21:51:17 -0000 Subject: [Python-checkins] bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) (GH-31521) Message-ID: https://github.com/python/cpython/commit/15d7594d9974cfef10e65cbb01161168c42abe9d commit: 15d7594d9974cfef10e65cbb01161168c42abe9d branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ned-deily date: 2022-02-23T16:51:08-05:00 summary: bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) (GH-31521) (cherry picked from commit 1935e1cc284942bec8006287c939e295e1a7bf13) Co-authored-by: Dong-hee Na files: A Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst M Modules/expat/expat.h M Modules/expat/xmlparse.c M Modules/expat/xmlrole.c M Modules/expat/xmltok.c M Modules/expat/xmltok_impl.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst new file mode 100644 index 0000000000000..127387d32cb7a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-12-07-53.bpo-46794.6WvJ9o.rst @@ -0,0 +1 @@ +Bump up the libexpat version into 2.4.6 diff --git a/Modules/expat/expat.h b/Modules/expat/expat.h index 4c5704fd9336b..46a0e1bcd22de 100644 --- a/Modules/expat/expat.h +++ b/Modules/expat/expat.h @@ -1041,7 +1041,7 @@ XML_SetBillionLaughsAttackProtectionActivationThreshold( */ #define XML_MAJOR_VERSION 2 #define XML_MINOR_VERSION 4 -#define XML_MICRO_VERSION 4 +#define XML_MICRO_VERSION 6 #ifdef __cplusplus } diff --git a/Modules/expat/xmlparse.c b/Modules/expat/xmlparse.c index 4b43e61321691..7db28d07acbcd 100644 --- a/Modules/expat/xmlparse.c +++ b/Modules/expat/xmlparse.c @@ -1,4 +1,4 @@ -/* 2e2c8ce5f11a473d65ec313ab20ceee6afefb355f5405afc06e7204e2e41c8c0 (2.4.4+) +/* a30d2613dcfdef81475a9d1a349134d2d42722172fdaa7d5bb12ed2aa74b9596 (2.4.6+) __ __ _ ___\ \/ /_ __ __ _| |_ / _ \\ /| '_ \ / _` | __| @@ -11,7 +11,7 @@ Copyright (c) 2000-2006 Fred L. Drake, Jr. Copyright (c) 2001-2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016 Eric Rahm Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Gaurav @@ -718,8 +718,7 @@ XML_ParserCreate(const XML_Char *encodingName) { XML_Parser XMLCALL XML_ParserCreateNS(const XML_Char *encodingName, XML_Char nsSep) { - XML_Char tmp[2]; - *tmp = nsSep; + XML_Char tmp[2] = {nsSep, 0}; return XML_ParserCreate_MM(encodingName, NULL, tmp); } @@ -1344,8 +1343,7 @@ XML_ExternalEntityParserCreate(XML_Parser oldParser, const XML_Char *context, would be otherwise. */ if (parser->m_ns) { - XML_Char tmp[2]; - *tmp = parser->m_namespaceSeparator; + XML_Char tmp[2] = {parser->m_namespaceSeparator, 0}; parser = parserCreate(encodingName, &parser->m_mem, tmp, newDtd); } else { parser = parserCreate(encodingName, &parser->m_mem, NULL, newDtd); @@ -2563,6 +2561,7 @@ storeRawNames(XML_Parser parser) { while (tag) { int bufSize; int nameLen = sizeof(XML_Char) * (tag->name.strLen + 1); + size_t rawNameLen; char *rawNameBuf = tag->buf + nameLen; /* Stop if already stored. Since m_tagStack is a stack, we can stop at the first entry that has already been copied; everything @@ -2574,7 +2573,11 @@ storeRawNames(XML_Parser parser) { /* For re-use purposes we need to ensure that the size of tag->buf is a multiple of sizeof(XML_Char). */ - bufSize = nameLen + ROUND_UP(tag->rawNameLength, sizeof(XML_Char)); + rawNameLen = ROUND_UP(tag->rawNameLength, sizeof(XML_Char)); + /* Detect and prevent integer overflow. */ + if (rawNameLen > (size_t)INT_MAX - nameLen) + return XML_FALSE; + bufSize = nameLen + (int)rawNameLen; if (bufSize > tag->bufEnd - tag->buf) { char *temp = (char *)REALLOC(parser, tag->buf, bufSize); if (temp == NULL) @@ -3756,6 +3759,17 @@ addBinding(XML_Parser parser, PREFIX *prefix, const ATTRIBUTE_ID *attId, if (! mustBeXML && isXMLNS && (len > xmlnsLen || uri[len] != xmlnsNamespace[len])) isXMLNS = XML_FALSE; + + // NOTE: While Expat does not validate namespace URIs against RFC 3986, + // we have to at least make sure that the XML processor on top of + // Expat (that is splitting tag names by namespace separator into + // 2- or 3-tuples (uri-local or uri-local-prefix)) cannot be confused + // by an attacker putting additional namespace separator characters + // into namespace declarations. That would be ambiguous and not to + // be expected. + if (parser->m_ns && (uri[len] == parser->m_namespaceSeparator)) { + return XML_ERROR_SYNTAX; + } } isXML = isXML && len == xmlLen; isXMLNS = isXMLNS && len == xmlnsLen; @@ -7317,44 +7331,15 @@ nextScaffoldPart(XML_Parser parser) { return next; } -static void -build_node(XML_Parser parser, int src_node, XML_Content *dest, - XML_Content **contpos, XML_Char **strpos) { - DTD *const dtd = parser->m_dtd; /* save one level of indirection */ - dest->type = dtd->scaffold[src_node].type; - dest->quant = dtd->scaffold[src_node].quant; - if (dest->type == XML_CTYPE_NAME) { - const XML_Char *src; - dest->name = *strpos; - src = dtd->scaffold[src_node].name; - for (;;) { - *(*strpos)++ = *src; - if (! *src) - break; - src++; - } - dest->numchildren = 0; - dest->children = NULL; - } else { - unsigned int i; - int cn; - dest->numchildren = dtd->scaffold[src_node].childcnt; - dest->children = *contpos; - *contpos += dest->numchildren; - for (i = 0, cn = dtd->scaffold[src_node].firstchild; i < dest->numchildren; - i++, cn = dtd->scaffold[cn].nextsib) { - build_node(parser, cn, &(dest->children[i]), contpos, strpos); - } - dest->name = NULL; - } -} - static XML_Content * build_model(XML_Parser parser) { + /* Function build_model transforms the existing parser->m_dtd->scaffold + * array of CONTENT_SCAFFOLD tree nodes into a new array of + * XML_Content tree nodes followed by a gapless list of zero-terminated + * strings. */ DTD *const dtd = parser->m_dtd; /* save one level of indirection */ XML_Content *ret; - XML_Content *cpos; - XML_Char *str; + XML_Char *str; /* the current string writing location */ /* Detect and prevent integer overflow. * The preprocessor guard addresses the "always false" warning @@ -7380,10 +7365,96 @@ build_model(XML_Parser parser) { if (! ret) return NULL; - str = (XML_Char *)(&ret[dtd->scaffCount]); - cpos = &ret[1]; + /* What follows is an iterative implementation (of what was previously done + * recursively in a dedicated function called "build_node". The old recursive + * build_node could be forced into stack exhaustion from input as small as a + * few megabyte, and so that was a security issue. Hence, a function call + * stack is avoided now by resolving recursion.) + * + * The iterative approach works as follows: + * + * - We have two writing pointers, both walking up the result array; one does + * the work, the other creates "jobs" for its colleague to do, and leads + * the way: + * + * - The faster one, pointer jobDest, always leads and writes "what job + * to do" by the other, once they reach that place in the + * array: leader "jobDest" stores the source node array index (relative + * to array dtd->scaffold) in field "numchildren". + * + * - The slower one, pointer dest, looks at the value stored in the + * "numchildren" field (which actually holds a source node array index + * at that time) and puts the real data from dtd->scaffold in. + * + * - Before the loop starts, jobDest writes source array index 0 + * (where the root node is located) so that dest will have something to do + * when it starts operation. + * + * - Whenever nodes with children are encountered, jobDest appends + * them as new jobs, in order. As a result, tree node siblings are + * adjacent in the resulting array, for example: + * + * [0] root, has two children + * [1] first child of 0, has three children + * [3] first child of 1, does not have children + * [4] second child of 1, does not have children + * [5] third child of 1, does not have children + * [2] second child of 0, does not have children + * + * Or (the same data) presented in flat array view: + * + * [0] root, has two children + * + * [1] first child of 0, has three children + * [2] second child of 0, does not have children + * + * [3] first child of 1, does not have children + * [4] second child of 1, does not have children + * [5] third child of 1, does not have children + * + * - The algorithm repeats until all target array indices have been processed. + */ + XML_Content *dest = ret; /* tree node writing location, moves upwards */ + XML_Content *const destLimit = &ret[dtd->scaffCount]; + XML_Content *jobDest = ret; /* next free writing location in target array */ + str = (XML_Char *)&ret[dtd->scaffCount]; + + /* Add the starting job, the root node (index 0) of the source tree */ + (jobDest++)->numchildren = 0; + + for (; dest < destLimit; dest++) { + /* Retrieve source tree array index from job storage */ + const int src_node = (int)dest->numchildren; + + /* Convert item */ + dest->type = dtd->scaffold[src_node].type; + dest->quant = dtd->scaffold[src_node].quant; + if (dest->type == XML_CTYPE_NAME) { + const XML_Char *src; + dest->name = str; + src = dtd->scaffold[src_node].name; + for (;;) { + *str++ = *src; + if (! *src) + break; + src++; + } + dest->numchildren = 0; + dest->children = NULL; + } else { + unsigned int i; + int cn; + dest->name = NULL; + dest->numchildren = dtd->scaffold[src_node].childcnt; + dest->children = jobDest; + + /* Append scaffold indices of children to array */ + for (i = 0, cn = dtd->scaffold[src_node].firstchild; + i < dest->numchildren; i++, cn = dtd->scaffold[cn].nextsib) + (jobDest++)->numchildren = (unsigned int)cn; + } + } - build_node(parser, 0, ret, &cpos, &str); return ret; } @@ -7412,7 +7483,7 @@ getElementType(XML_Parser parser, const ENCODING *enc, const char *ptr, static XML_Char * copyString(const XML_Char *s, const XML_Memory_Handling_Suite *memsuite) { - int charsRequired = 0; + size_t charsRequired = 0; XML_Char *result; /* First determine how long the string is */ diff --git a/Modules/expat/xmlrole.c b/Modules/expat/xmlrole.c index 77746ee42d10a..3f0f5c150c627 100644 --- a/Modules/expat/xmlrole.c +++ b/Modules/expat/xmlrole.c @@ -11,7 +11,7 @@ Copyright (c) 2002 Greg Stein Copyright (c) 2002-2006 Karl Waclawek Copyright (c) 2002-2003 Fred L. Drake, Jr. - Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2005-2009 Steven Solie Copyright (c) 2016-2021 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2019 David Loffredo diff --git a/Modules/expat/xmltok.c b/Modules/expat/xmltok.c index 502ca1adc33b9..c659983b4008b 100644 --- a/Modules/expat/xmltok.c +++ b/Modules/expat/xmltok.c @@ -11,8 +11,8 @@ Copyright (c) 2001-2003 Fred L. Drake, Jr. Copyright (c) 2002 Greg Stein Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2005-2009 Steven Solie - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2005-2009 Steven Solie + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2016 Pascal Cuoq Copyright (c) 2016 Don Lewis Copyright (c) 2017 Rhodri James @@ -98,11 +98,6 @@ + ((((byte)[1]) & 3) << 1) + ((((byte)[2]) >> 5) & 1)] \ & (1u << (((byte)[2]) & 0x1F))) -#define UTF8_GET_NAMING(pages, p, n) \ - ((n) == 2 \ - ? UTF8_GET_NAMING2(pages, (const unsigned char *)(p)) \ - : ((n) == 3 ? UTF8_GET_NAMING3(pages, (const unsigned char *)(p)) : 0)) - /* Detection of invalid UTF-8 sequences is based on Table 3.1B of Unicode 3.2: http://www.unicode.org/unicode/reports/tr28/ with the additional restriction of not allowing the Unicode diff --git a/Modules/expat/xmltok_impl.c b/Modules/expat/xmltok_impl.c index 0430591b42636..4072b06497d1c 100644 --- a/Modules/expat/xmltok_impl.c +++ b/Modules/expat/xmltok_impl.c @@ -10,7 +10,7 @@ Copyright (c) 2000 Clark Cooper Copyright (c) 2002 Fred L. Drake, Jr. Copyright (c) 2002-2016 Karl Waclawek - Copyright (c) 2016-2021 Sebastian Pipping + Copyright (c) 2016-2022 Sebastian Pipping Copyright (c) 2017 Rhodri James Copyright (c) 2018 Benjamin Peterson Copyright (c) 2018 Anton Maklakov @@ -69,7 +69,7 @@ case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ - if (! IS_NAME_CHAR(enc, ptr, n)) { \ + if (IS_INVALID_CHAR(enc, ptr, n) || ! IS_NAME_CHAR(enc, ptr, n)) { \ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ @@ -98,7 +98,7 @@ case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ - if (! IS_NMSTRT_CHAR(enc, ptr, n)) { \ + if (IS_INVALID_CHAR(enc, ptr, n) || ! IS_NMSTRT_CHAR(enc, ptr, n)) { \ *nextTokPtr = ptr; \ return XML_TOK_INVALID; \ } \ @@ -1142,6 +1142,10 @@ PREFIX(prologTok)(const ENCODING *enc, const char *ptr, const char *end, case BT_LEAD##n: \ if (end - ptr < n) \ return XML_TOK_PARTIAL_CHAR; \ + if (IS_INVALID_CHAR(enc, ptr, n)) { \ + *nextTokPtr = ptr; \ + return XML_TOK_INVALID; \ + } \ if (IS_NMSTRT_CHAR(enc, ptr, n)) { \ ptr += n; \ tok = XML_TOK_NAME; \ @@ -1270,7 +1274,7 @@ PREFIX(attributeValueTok)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1339,7 +1343,7 @@ PREFIX(entityValueTok)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1518,7 +1522,7 @@ PREFIX(getAtts)(const ENCODING *enc, const char *ptr, int attsMax, state = inName; \ } # define LEAD_CASE(n) \ - case BT_LEAD##n: \ + case BT_LEAD##n: /* NOTE: The encoding has already been validated. */ \ START_NAME ptr += (n - MINBPC(enc)); \ break; LEAD_CASE(2) @@ -1730,7 +1734,7 @@ PREFIX(nameLength)(const ENCODING *enc, const char *ptr) { switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ break; LEAD_CASE(2) LEAD_CASE(3) @@ -1775,7 +1779,7 @@ PREFIX(updatePosition)(const ENCODING *enc, const char *ptr, const char *end, switch (BYTE_TYPE(enc, ptr)) { # define LEAD_CASE(n) \ case BT_LEAD##n: \ - ptr += n; \ + ptr += n; /* NOTE: The encoding has already been validated. */ \ pos->columnNumber++; \ break; LEAD_CASE(2) From webhook-mailer at python.org Wed Feb 23 21:17:14 2022 From: webhook-mailer at python.org (gvanrossum) Date: Thu, 24 Feb 2022 02:17:14 -0000 Subject: [Python-checkins] bpo-46771: Implement task cancel requests counter (GH-31513) Message-ID: https://github.com/python/cpython/commit/7fce1063b6e5a366f8504e039a8ccdd6944625cd commit: 7fce1063b6e5a366f8504e039a8ccdd6944625cd branch: main author: Tin Tvrtkovi? committer: gvanrossum date: 2022-02-23T18:17:00-08:00 summary: bpo-46771: Implement task cancel requests counter (GH-31513) This changes cancelling() and uncancel() to return the count of pending cancellations. This can be used to avoid bugs in certain edge cases (e.g. two timeouts going off at the same time). files: M Lib/asyncio/tasks.py M Modules/_asynciomodule.c M Modules/clinic/_asynciomodule.c.h diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py index 25a650ffbb744..38c23851102a8 100644 --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -105,7 +105,7 @@ def __init__(self, coro, *, loop=None, name=None): else: self._name = str(name) - self._cancel_requested = False + self._num_cancels_requested = 0 self._must_cancel = False self._fut_waiter = None self._coro = coro @@ -198,13 +198,15 @@ def cancel(self, msg=None): task will be marked as cancelled when the wrapped coroutine terminates with a CancelledError exception (even if cancel() was not called). + + This also increases the task's count of cancellation requests. """ self._log_traceback = False if self.done(): return False - if self._cancel_requested: + self._num_cancels_requested += 1 + if self._num_cancels_requested > 1: return False - self._cancel_requested = True if self._fut_waiter is not None: if self._fut_waiter.cancel(msg=msg): # Leave self._fut_waiter; it may be a Task that @@ -217,14 +219,24 @@ def cancel(self, msg=None): return True def cancelling(self): - return self._cancel_requested + """Return the count of the task's cancellation requests. + + This count is incremented when .cancel() is called + and may be decremented using .uncancel(). + """ + return self._num_cancels_requested def uncancel(self): - if self._cancel_requested: - self._cancel_requested = False - return True - else: - return False + """Decrement the task's count of cancellation requests. + + This should be used by tasks that catch CancelledError + and wish to continue indefinitely until they are cancelled again. + + Returns the remaining number of cancellation requests. + """ + if self._num_cancels_requested > 0: + self._num_cancels_requested -= 1 + return self._num_cancels_requested def __step(self, exc=None): if self.done(): diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 0d6b21d3b0c39..88471239529a5 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -91,7 +91,7 @@ typedef struct { PyObject *task_context; int task_must_cancel; int task_log_destroy_pending; - int task_cancel_requested; + int task_num_cancels_requested; } TaskObj; typedef struct { @@ -2036,7 +2036,7 @@ _asyncio_Task___init___impl(TaskObj *self, PyObject *coro, PyObject *loop, Py_CLEAR(self->task_fut_waiter); self->task_must_cancel = 0; self->task_log_destroy_pending = 1; - self->task_cancel_requested = 0; + self->task_num_cancels_requested = 0; Py_INCREF(coro); Py_XSETREF(self->task_coro, coro); @@ -2191,11 +2191,13 @@ not return True (unless the task was already cancelled). A task will be marked as cancelled when the wrapped coroutine terminates with a CancelledError exception (even if cancel() was not called). + +This also increases the task's count of cancellation requests. [clinic start generated code]*/ static PyObject * _asyncio_Task_cancel_impl(TaskObj *self, PyObject *msg) -/*[clinic end generated code: output=c66b60d41c74f9f1 input=f4ff8e8ffc5f1c00]*/ +/*[clinic end generated code: output=c66b60d41c74f9f1 input=7bb51bf25974c783]*/ { self->task_log_tb = 0; @@ -2203,10 +2205,10 @@ _asyncio_Task_cancel_impl(TaskObj *self, PyObject *msg) Py_RETURN_FALSE; } - if (self->task_cancel_requested) { + self->task_num_cancels_requested += 1; + if (self->task_num_cancels_requested > 1) { Py_RETURN_FALSE; } - self->task_cancel_requested = 1; if (self->task_fut_waiter) { PyObject *res; @@ -2238,51 +2240,40 @@ _asyncio_Task_cancel_impl(TaskObj *self, PyObject *msg) /*[clinic input] _asyncio.Task.cancelling -Return True if the task is in the process of being cancelled. - -This is set once .cancel() is called -and remains set until .uncancel() is called. +Return the count of the task's cancellation requests. -As long as this flag is set, further .cancel() calls will be ignored, -until .uncancel() is called to reset it. +This count is incremented when .cancel() is called +and may be decremented using .uncancel(). [clinic start generated code]*/ static PyObject * _asyncio_Task_cancelling_impl(TaskObj *self) -/*[clinic end generated code: output=803b3af96f917d7e input=c50e50f9c3ca4676]*/ +/*[clinic end generated code: output=803b3af96f917d7e input=b625224d310cbb17]*/ /*[clinic end generated code]*/ { - if (self->task_cancel_requested) { - Py_RETURN_TRUE; - } - else { - Py_RETURN_FALSE; - } + return PyLong_FromLong(self->task_num_cancels_requested); } /*[clinic input] _asyncio.Task.uncancel -Reset the flag returned by cancelling(). +Decrement the task's count of cancellation requests. This should be used by tasks that catch CancelledError and wish to continue indefinitely until they are cancelled again. -Returns the previous value of the flag. +Returns the remaining number of cancellation requests. [clinic start generated code]*/ static PyObject * _asyncio_Task_uncancel_impl(TaskObj *self) -/*[clinic end generated code: output=58184d236a817d3c input=5db95e28fcb6f7cd]*/ +/*[clinic end generated code: output=58184d236a817d3c input=68f81a4b90b46be2]*/ /*[clinic end generated code]*/ { - if (self->task_cancel_requested) { - self->task_cancel_requested = 0; - Py_RETURN_TRUE; - } - else { - Py_RETURN_FALSE; + if (self->task_num_cancels_requested > 0) { + self->task_num_cancels_requested -= 1; } + return PyLong_FromLong(self->task_num_cancels_requested); } /*[clinic input] diff --git a/Modules/clinic/_asynciomodule.c.h b/Modules/clinic/_asynciomodule.c.h index 5648e14f337f7..2b84ef0a477c7 100644 --- a/Modules/clinic/_asynciomodule.c.h +++ b/Modules/clinic/_asynciomodule.c.h @@ -414,7 +414,9 @@ PyDoc_STRVAR(_asyncio_Task_cancel__doc__, "not return True (unless the task was already cancelled). A\n" "task will be marked as cancelled when the wrapped coroutine\n" "terminates with a CancelledError exception (even if cancel()\n" -"was not called)."); +"was not called).\n" +"\n" +"This also increases the task\'s count of cancellation requests."); #define _ASYNCIO_TASK_CANCEL_METHODDEF \ {"cancel", (PyCFunction)(void(*)(void))_asyncio_Task_cancel, METH_FASTCALL|METH_KEYWORDS, _asyncio_Task_cancel__doc__}, @@ -451,13 +453,10 @@ PyDoc_STRVAR(_asyncio_Task_cancelling__doc__, "cancelling($self, /)\n" "--\n" "\n" -"Return True if the task is in the process of being cancelled.\n" -"\n" -"This is set once .cancel() is called\n" -"and remains set until .uncancel() is called.\n" +"Return the count of the task\'s cancellation requests.\n" "\n" -"As long as this flag is set, further .cancel() calls will be ignored,\n" -"until .uncancel() is called to reset it."); +"This count is incremented when .cancel() is called\n" +"and may be decremented using .uncancel()."); #define _ASYNCIO_TASK_CANCELLING_METHODDEF \ {"cancelling", (PyCFunction)_asyncio_Task_cancelling, METH_NOARGS, _asyncio_Task_cancelling__doc__}, @@ -475,12 +474,12 @@ PyDoc_STRVAR(_asyncio_Task_uncancel__doc__, "uncancel($self, /)\n" "--\n" "\n" -"Reset the flag returned by cancelling().\n" +"Decrement the task\'s count of cancellation requests.\n" "\n" "This should be used by tasks that catch CancelledError\n" "and wish to continue indefinitely until they are cancelled again.\n" "\n" -"Returns the previous value of the flag."); +"Returns the remaining number of cancellation requests."); #define _ASYNCIO_TASK_UNCANCEL_METHODDEF \ {"uncancel", (PyCFunction)_asyncio_Task_uncancel, METH_NOARGS, _asyncio_Task_uncancel__doc__}, @@ -918,4 +917,4 @@ _asyncio__leave_task(PyObject *module, PyObject *const *args, Py_ssize_t nargs, exit: return return_value; } -/*[clinic end generated code: output=c02708a9d6a774cc input=a9049054013a1b77]*/ +/*[clinic end generated code: output=344927e9b6016ad7 input=a9049054013a1b77]*/ From webhook-mailer at python.org Thu Feb 24 07:11:12 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 24 Feb 2022 12:11:12 -0000 Subject: [Python-checkins] Add (undocumented) _co_quickened attribute for code object. (GH-31552) Message-ID: https://github.com/python/cpython/commit/ae3adbeaedb0536292f4542a0d42063410ae150f commit: ae3adbeaedb0536292f4542a0d42063410ae150f branch: main author: Mark Shannon committer: markshannon date: 2022-02-24T12:10:53Z summary: Add (undocumented) _co_quickened attribute for code object. (GH-31552) files: M Objects/codeobject.c diff --git a/Objects/codeobject.c b/Objects/codeobject.c index f8ef1e6b40cc1..f166204de2d8f 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -1537,6 +1537,16 @@ code_getfreevars(PyCodeObject *code, void *closure) return _PyCode_GetFreevars(code); } +static PyObject * +code_getquickened(PyCodeObject *code, void *closure) +{ + if (code->co_quickened == NULL) { + Py_RETURN_NONE; + } + return PyBytes_FromStringAndSize((char *)code->co_firstinstr, + PyBytes_Size(code->co_code)); +} + static PyGetSetDef code_getsetlist[] = { {"co_lnotab", (getter)code_getlnotab, NULL, NULL}, // The following old names are kept for backward compatibility. @@ -1544,6 +1554,7 @@ static PyGetSetDef code_getsetlist[] = { {"co_varnames", (getter)code_getvarnames, NULL, NULL}, {"co_cellvars", (getter)code_getcellvars, NULL, NULL}, {"co_freevars", (getter)code_getfreevars, NULL, NULL}, + {"_co_quickened", (getter)code_getquickened, NULL, NULL}, {0} }; @@ -1902,7 +1913,7 @@ _PyCode_ConstantKey(PyObject *op) return key; } -void +void _PyStaticCode_Dealloc(PyCodeObject *co) { if (co->co_quickened) { @@ -1921,7 +1932,7 @@ _PyStaticCode_Dealloc(PyCodeObject *co) } void -_PyStaticCode_InternStrings(PyCodeObject *co) +_PyStaticCode_InternStrings(PyCodeObject *co) { int res = intern_strings(co->co_names); assert(res == 0); From webhook-mailer at python.org Thu Feb 24 08:29:36 2022 From: webhook-mailer at python.org (vstinner) Date: Thu, 24 Feb 2022 13:29:36 -0000 Subject: [Python-checkins] bpo-46659: Enhance LocaleTextCalendar for C locale (GH-31214) Message-ID: https://github.com/python/cpython/commit/4fccf910738d1442852cb900747e6dccb8fe03ef commit: 4fccf910738d1442852cb900747e6dccb8fe03ef branch: main author: Victor Stinner committer: vstinner date: 2022-02-24T14:29:08+01:00 summary: bpo-46659: Enhance LocaleTextCalendar for C locale (GH-31214) If the LC_TIME locale is "C", use the user preferred locale. files: M Doc/library/calendar.rst M Lib/calendar.py diff --git a/Doc/library/calendar.rst b/Doc/library/calendar.rst index 86f5b142a6c34..66f59f0e2ced2 100644 --- a/Doc/library/calendar.rst +++ b/Doc/library/calendar.rst @@ -289,9 +289,10 @@ interpreted as prescribed by the ISO 8601 standard. Year 0 is 1 BC, year -1 is .. note:: - The :meth:`formatweekday` and :meth:`formatmonthname` methods of these two - classes temporarily change the ``LC_TIME`` locale to the given *locale*. Because - the current locale is a process-wide setting, they are not thread-safe. + The constructor, :meth:`formatweekday` and :meth:`formatmonthname` methods + of these two classes temporarily change the ``LC_TIME`` locale to the given + *locale*. Because the current locale is a process-wide setting, they are + not thread-safe. For simple text calendars this module provides the following functions. diff --git a/Lib/calendar.py b/Lib/calendar.py index 361898dc8161e..657396439c91f 100644 --- a/Lib/calendar.py +++ b/Lib/calendar.py @@ -548,15 +548,28 @@ def formatyearpage(self, theyear, width=3, css='calendar.css', encoding=None): class different_locale: def __init__(self, locale): self.locale = locale + self.oldlocale = None def __enter__(self): - self.oldlocale = _locale.getlocale(_locale.LC_TIME) + self.oldlocale = _locale.setlocale(_locale.LC_TIME, None) _locale.setlocale(_locale.LC_TIME, self.locale) def __exit__(self, *args): + if self.oldlocale is None: + return _locale.setlocale(_locale.LC_TIME, self.oldlocale) +def _get_default_locale(): + locale = _locale.setlocale(_locale.LC_TIME, None) + if locale == "C": + with different_locale(""): + # The LC_TIME locale does not seem to be configured: + # get the user preferred locale. + locale = _locale.setlocale(_locale.LC_TIME, None) + return locale + + class LocaleTextCalendar(TextCalendar): """ This class can be passed a locale name in the constructor and will return @@ -566,7 +579,7 @@ class LocaleTextCalendar(TextCalendar): def __init__(self, firstweekday=0, locale=None): TextCalendar.__init__(self, firstweekday) if locale is None: - locale = _locale.getlocale(_locale.LC_TIME) + locale = _get_default_locale() self.locale = locale def formatweekday(self, day, width): @@ -586,7 +599,7 @@ class LocaleHTMLCalendar(HTMLCalendar): def __init__(self, firstweekday=0, locale=None): HTMLCalendar.__init__(self, firstweekday) if locale is None: - locale = _locale.getlocale(_locale.LC_TIME) + locale = _get_default_locale() self.locale = locale def formatweekday(self, day): From webhook-mailer at python.org Thu Feb 24 09:56:05 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 24 Feb 2022 14:56:05 -0000 Subject: [Python-checkins] bpo-46823: Implement LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE superinstruction (GH-31484) Message-ID: https://github.com/python/cpython/commit/a52d2528a405c1e2bfeb6470cb3313a5338dc45f commit: a52d2528a405c1e2bfeb6470cb3313a5338dc45f branch: main author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: markshannon date: 2022-02-24T14:55:59Z summary: bpo-46823: Implement LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE superinstruction (GH-31484) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-22-05-14-25.bpo-46823.z9NZC9.rst M Include/opcode.h M Lib/opcode.py M Python/ceval.c M Python/opcode_targets.h M Python/specialize.c diff --git a/Include/opcode.h b/Include/opcode.h index 7820af64829b6..50153c19e9b23 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -181,6 +181,7 @@ extern "C" { #define LOAD_FAST__LOAD_CONST 169 #define LOAD_CONST__LOAD_FAST 170 #define STORE_FAST__STORE_FAST 173 +#define LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE 174 #define DO_TRACING 255 #ifdef NEED_OPCODE_JUMP_TABLES static uint32_t _PyOpcode_RelativeJump[8] = { diff --git a/Lib/opcode.py b/Lib/opcode.py index 0c859c16b718c..71a9347debad7 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -295,6 +295,7 @@ def jabs_op(name, op): "LOAD_FAST__LOAD_CONST", "LOAD_CONST__LOAD_FAST", "STORE_FAST__STORE_FAST", + "LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE", ] _specialization_stats = [ "success", diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-22-05-14-25.bpo-46823.z9NZC9.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-05-14-25.bpo-46823.z9NZC9.rst new file mode 100644 index 0000000000000..908f48d33f285 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-22-05-14-25.bpo-46823.z9NZC9.rst @@ -0,0 +1 @@ +Implement a specialized combined opcode ``LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE``. Patch by Dennis Sweeney. diff --git a/Python/ceval.c b/Python/ceval.c index 807757019020a..f3bdaf1377077 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -3444,6 +3444,34 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } } + TARGET(LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE) { + assert(cframe.use_tracing == 0); + PyObject *owner = GETLOCAL(oparg); // borrowed + if (owner == NULL) { + goto unbound_local_error; + } + // GET_CACHE(), but for the following opcode + assert(_Py_OPCODE(*next_instr) == LOAD_ATTR_INSTANCE_VALUE); + SpecializedCacheEntry *caches = _GetSpecializedCacheEntryForInstruction( + first_instr, INSTR_OFFSET() + 1, _Py_OPARG(*next_instr)); + _PyAdaptiveEntry *cache0 = &caches[0].adaptive; + assert(cache0->version != 0); + PyTypeObject *tp = Py_TYPE(owner); + // These DEOPT_IF miss branches do PUSH(Py_NewRef(owner)). + DEOPT_IF(tp->tp_version_tag != cache0->version, + LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE); + assert(tp->tp_dictoffset < 0); + assert(tp->tp_flags & Py_TPFLAGS_MANAGED_DICT); + PyDictValues *values = *_PyObject_ValuesPointer(owner); + DEOPT_IF(values == NULL, LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE); + PyObject *res = values->values[cache0->index]; + DEOPT_IF(res == NULL, LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE); + STAT_INC(LOAD_ATTR, hit); + PUSH(Py_NewRef(res)); + next_instr++; + NOTRACE_DISPATCH(); + } + TARGET(LOAD_ATTR_INSTANCE_VALUE) { assert(cframe.use_tracing == 0); PyObject *owner = TOP(); @@ -3452,13 +3480,13 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr SpecializedCacheEntry *caches = GET_CACHE(); _PyAdaptiveEntry *cache0 = &caches[0].adaptive; assert(cache0->version != 0); - DEOPT_IF(tp->tp_version_tag != cache0->version, LOAD_ATTR); + DEOPT_IF(tp->tp_version_tag != cache0->version, LOAD_ATTR_INSTANCE_VALUE); assert(tp->tp_dictoffset < 0); assert(tp->tp_flags & Py_TPFLAGS_MANAGED_DICT); PyDictValues *values = *_PyObject_ValuesPointer(owner); - DEOPT_IF(values == NULL, LOAD_ATTR); + DEOPT_IF(values == NULL, LOAD_ATTR_INSTANCE_VALUE); res = values->values[cache0->index]; - DEOPT_IF(res == NULL, LOAD_ATTR); + DEOPT_IF(res == NULL, LOAD_ATTR_INSTANCE_VALUE); STAT_INC(LOAD_ATTR, hit); Py_INCREF(res); SET_TOP(res); @@ -5515,6 +5543,52 @@ MISS_WITH_CACHE(BINARY_SUBSCR) MISS_WITH_CACHE(UNPACK_SEQUENCE) MISS_WITH_OPARG_COUNTER(STORE_SUBSCR) +LOAD_ATTR_INSTANCE_VALUE_miss: + { + // Special-cased so that if LOAD_ATTR_INSTANCE_VALUE + // gets replaced, then any preceeding + // LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE gets replaced as well + STAT_INC(LOAD_ATTR_INSTANCE_VALUE, miss); + STAT_INC(LOAD_ATTR, miss); + _PyAdaptiveEntry *cache = &GET_CACHE()->adaptive; + cache->counter--; + if (cache->counter == 0) { + next_instr[-1] = _Py_MAKECODEUNIT(LOAD_ATTR_ADAPTIVE, _Py_OPARG(next_instr[-1])); + if (_Py_OPCODE(next_instr[-2]) == LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE) { + next_instr[-2] = _Py_MAKECODEUNIT(LOAD_FAST, _Py_OPARG(next_instr[-2])); + if (_Py_OPCODE(next_instr[-3]) == LOAD_FAST) { + next_instr[-3] = _Py_MAKECODEUNIT(LOAD_FAST__LOAD_FAST, _Py_OPARG(next_instr[-3])); + } + } + STAT_INC(LOAD_ATTR, deopt); + cache_backoff(cache); + } + oparg = cache->original_oparg; + JUMP_TO_INSTRUCTION(LOAD_ATTR); + } + +LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE_miss: + { + // This is special-cased because we have a superinstruction + // that includes a specialized instruction. + // If the specialized portion misses, carry out + // the first instruction, then perform a miss + // for the second instruction as usual. + + // Do LOAD_FAST + { + PyObject *value = GETLOCAL(oparg); + assert(value != NULL); // Already checked if unbound + Py_INCREF(value); + PUSH(value); + NEXTOPARG(); + next_instr++; + } + + // Now we are in the correct state for LOAD_ATTR + goto LOAD_ATTR_INSTANCE_VALUE_miss; + } + binary_subscr_dict_error: { PyObject *sub = POP(); diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index cf5bdc772e421..fac671eb024f5 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -173,7 +173,7 @@ static void *opcode_targets[256] = { &&TARGET_CALL, &&TARGET_KW_NAMES, &&TARGET_STORE_FAST__STORE_FAST, - &&_unknown_opcode, + &&TARGET_LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE, &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, diff --git a/Python/specialize.c b/Python/specialize.c index 816cca1074e83..1641766712a60 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -889,6 +889,16 @@ _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, Sp return -1; } if (err) { + if (_Py_OPCODE(instr[0]) == LOAD_ATTR_INSTANCE_VALUE) { + // Note: instr[-1] exists because there's something on the stack, + // and instr[-2] exists because there's at least a RESUME as well. + if (_Py_OPCODE(instr[-1]) == LOAD_FAST) { + instr[-1] = _Py_MAKECODEUNIT(LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE, _Py_OPARG(instr[-1])); + if (_Py_OPCODE(instr[-2]) == LOAD_FAST__LOAD_FAST) { + instr[-2] = _Py_MAKECODEUNIT(LOAD_FAST, _Py_OPARG(instr[-2])); + } + } + } goto success; } fail: From webhook-mailer at python.org Thu Feb 24 11:07:29 2022 From: webhook-mailer at python.org (vstinner) Date: Thu, 24 Feb 2022 16:07:29 -0000 Subject: [Python-checkins] bpo-45459: Add pytypedefs.h header file (GH-31527) Message-ID: https://github.com/python/cpython/commit/ec091bd47e2f968b0d1631b9a8104283a7beeb1b commit: ec091bd47e2f968b0d1631b9a8104283a7beeb1b branch: main author: Victor Stinner committer: vstinner date: 2022-02-24T17:07:12+01:00 summary: bpo-45459: Add pytypedefs.h header file (GH-31527) Move forward declarations of Python C API types to a new pytypedefs.h header file to solve interdependency issues between header files. pytypedefs.h contains forward declarations of the following types: * PyCodeObject * PyFrameObject * PyGetSetDef * PyInterpreterState * PyLongObject * PyMemberDef * PyMethodDef * PyModuleDef * PyObject * PyThreadState * PyTypeObject files: A Include/pytypedefs.h M Include/Python.h M Include/code.h M Include/cpython/object.h M Include/descrobject.h M Include/longobject.h M Include/methodobject.h M Include/moduleobject.h M Include/object.h M Include/pybuffer.h M Include/pyframe.h M Include/pystate.h M Include/structmember.h M Makefile.pre.in M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters diff --git a/Include/Python.h b/Include/Python.h index 5bc8cc656af0d..4dc2edb4dbbe2 100644 --- a/Include/Python.h +++ b/Include/Python.h @@ -39,6 +39,7 @@ #include "pymacro.h" #include "pymath.h" #include "pymem.h" +#include "pytypedefs.h" #include "pybuffer.h" #include "object.h" #include "objimpl.h" diff --git a/Include/code.h b/Include/code.h index 2dea3c2610531..0245c32bc2544 100644 --- a/Include/code.h +++ b/Include/code.h @@ -6,8 +6,6 @@ extern "C" { #endif -typedef struct PyCodeObject PyCodeObject; - #ifndef Py_LIMITED_API # define Py_CPYTHON_CODE_H # include "cpython/code.h" diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 6cc3d72ca7084..65d7d85b7075d 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -262,10 +262,8 @@ PyAPI_FUNC(PyObject *) _PyObject_LookupSpecialId(PyObject *, _Py_Identifier *); PyAPI_FUNC(PyTypeObject *) _PyType_CalculateMetaclass(PyTypeObject *, PyObject *); PyAPI_FUNC(PyObject *) _PyType_GetDocFromInternalDoc(const char *, const char *); PyAPI_FUNC(PyObject *) _PyType_GetTextSignatureFromInternalDoc(const char *, const char *); -struct PyModuleDef; PyAPI_FUNC(PyObject *) PyType_GetModuleByDef(PyTypeObject *, struct PyModuleDef *); -struct _Py_Identifier; PyAPI_FUNC(int) PyObject_Print(PyObject *, FILE *, int); PyAPI_FUNC(void) _Py_BreakPoint(void); PyAPI_FUNC(void) _PyObject_Dump(PyObject *); @@ -462,9 +460,6 @@ partially-deallocated object. To check this, the tp_dealloc function must be passed as second argument to Py_TRASHCAN_BEGIN(). */ -/* Forward declarations for PyThreadState */ -struct _ts; - /* Python 3.9 private API, invoked by the macros below. */ PyAPI_FUNC(int) _PyTrash_begin(struct _ts *tstate, PyObject *op); PyAPI_FUNC(void) _PyTrash_end(struct _ts *tstate); diff --git a/Include/descrobject.h b/Include/descrobject.h index 36802f4c55afa..aefa7600346b5 100644 --- a/Include/descrobject.h +++ b/Include/descrobject.h @@ -8,13 +8,13 @@ extern "C" { typedef PyObject *(*getter)(PyObject *, void *); typedef int (*setter)(PyObject *, PyObject *, void *); -typedef struct PyGetSetDef { +struct PyGetSetDef { const char *name; getter get; setter set; const char *doc; void *closure; -} PyGetSetDef; +}; PyAPI_DATA(PyTypeObject) PyClassMethodDescr_Type; PyAPI_DATA(PyTypeObject) PyGetSetDescr_Type; diff --git a/Include/longobject.h b/Include/longobject.h index 7fe8f58cb3e95..81ba1239a69ec 100644 --- a/Include/longobject.h +++ b/Include/longobject.h @@ -7,8 +7,6 @@ extern "C" { /* Long (arbitrary precision) integer object interface */ -typedef struct _longobject PyLongObject; /* Revealed in longintrepr.h */ - PyAPI_DATA(PyTypeObject) PyLong_Type; #define PyLong_Check(op) \ diff --git a/Include/methodobject.h b/Include/methodobject.h index 1be5873a30569..5d2e06c3e7cea 100644 --- a/Include/methodobject.h +++ b/Include/methodobject.h @@ -39,7 +39,6 @@ struct PyMethodDef { describe the args expected by the C func */ const char *ml_doc; /* The __doc__ attribute, or NULL */ }; -typedef struct PyMethodDef PyMethodDef; /* PyCFunction_New is declared as a function for stable ABI (declaration is * needed for e.g. GCC with -fvisibility=hidden), but redefined as a macro diff --git a/Include/moduleobject.h b/Include/moduleobject.h index 49b116ca1c358..0f406832b58ea 100644 --- a/Include/moduleobject.h +++ b/Include/moduleobject.h @@ -72,7 +72,7 @@ typedef struct PyModuleDef_Slot{ #endif /* New in 3.5 */ -typedef struct PyModuleDef{ +struct PyModuleDef { PyModuleDef_Base m_base; const char* m_name; const char* m_doc; @@ -82,7 +82,7 @@ typedef struct PyModuleDef{ traverseproc m_traverse; inquiry m_clear; freefunc m_free; -} PyModuleDef; +}; // Internal C API diff --git a/Include/object.h b/Include/object.h index 3566c736a535c..5313ea6f867b9 100644 --- a/Include/object.h +++ b/Include/object.h @@ -1,6 +1,5 @@ #ifndef Py_OBJECT_H #define Py_OBJECT_H - #ifdef __cplusplus extern "C" { #endif @@ -61,10 +60,6 @@ whose size is determined when the object is allocated. # error Py_LIMITED_API is incompatible with Py_TRACE_REFS #endif -/* PyTypeObject structure is defined in cpython/object.h. - In Py_LIMITED_API, PyTypeObject is an opaque structure. */ -typedef struct _typeobject PyTypeObject; - #ifdef Py_TRACE_REFS /* Define pointers to support a doubly-linked list of all live heap objects. */ #define _PyObject_HEAD_EXTRA \ @@ -102,11 +97,11 @@ typedef struct _typeobject PyTypeObject; * by hand. Similarly every pointer to a variable-size Python object can, * in addition, be cast to PyVarObject*. */ -typedef struct _object { +struct _object { _PyObject_HEAD_EXTRA Py_ssize_t ob_refcnt; PyTypeObject *ob_type; -} PyObject; +}; /* Cast argument to PyObject* type. */ #define _PyObject_CAST(op) ((PyObject*)(op)) @@ -765,4 +760,4 @@ static inline int PyType_CheckExact(PyObject *op) { #ifdef __cplusplus } #endif -#endif /* !Py_OBJECT_H */ +#endif // !Py_OBJECT_H diff --git a/Include/pybuffer.h b/Include/pybuffer.h index 31795b780710f..6893505e66e3e 100644 --- a/Include/pybuffer.h +++ b/Include/pybuffer.h @@ -17,10 +17,6 @@ extern "C" { * */ -// Forward declaration to be able to include pybuffer.h before object.h: -// pybuffer.h uses PyObject and object.h uses Py_buffer. -typedef struct _object PyObject; - typedef struct { void *buf; PyObject *obj; /* owned reference */ diff --git a/Include/pyframe.h b/Include/pyframe.h index 3816224201c7e..feac16f69de69 100644 --- a/Include/pyframe.h +++ b/Include/pyframe.h @@ -9,8 +9,6 @@ extern "C" { #endif -typedef struct _frame PyFrameObject; - /* Return the line of code the frame is currently executing. */ PyAPI_FUNC(int) PyFrame_GetLineNumber(PyFrameObject *); diff --git a/Include/pystate.h b/Include/pystate.h index b6ee0ede81dea..5b4245ed7f396 100644 --- a/Include/pystate.h +++ b/Include/pystate.h @@ -11,16 +11,6 @@ extern "C" { removed (with effort). */ #define MAX_CO_EXTRA_USERS 255 -/* Forward declarations for PyFrameObject, PyThreadState - and PyInterpreterState */ -struct _ts; -struct _is; - -/* struct _ts is defined in cpython/pystate.h */ -typedef struct _ts PyThreadState; -/* struct _is is defined in internal/pycore_interp.h */ -typedef struct _is PyInterpreterState; - PyAPI_FUNC(PyInterpreterState *) PyInterpreterState_New(void); PyAPI_FUNC(void) PyInterpreterState_Clear(PyInterpreterState *); PyAPI_FUNC(void) PyInterpreterState_Delete(PyInterpreterState *); diff --git a/Include/pytypedefs.h b/Include/pytypedefs.h new file mode 100644 index 0000000000000..5dd841e62173c --- /dev/null +++ b/Include/pytypedefs.h @@ -0,0 +1,29 @@ +// Forward declarations of types of the Python C API. +// Declare them at the same place since redefining typedef is a C11 feature. +// Only use a forward declaration if there is an interdependency between two +// header files. + +#ifndef Py_PYTYPEDEFS_H +#define Py_PYTYPEDEFS_H +#ifdef __cplusplus +extern "C" { +#endif + +typedef struct PyModuleDef PyModuleDef; +typedef struct PyMethodDef PyMethodDef; +typedef struct PyGetSetDef PyGetSetDef; +typedef struct PyMemberDef PyMemberDef; + +typedef struct _object PyObject; +typedef struct _longobject PyLongObject; +typedef struct _typeobject PyTypeObject; +typedef struct PyCodeObject PyCodeObject; +typedef struct _frame PyFrameObject; + +typedef struct _ts PyThreadState; +typedef struct _is PyInterpreterState; + +#ifdef __cplusplus +} +#endif +#endif // !Py_PYTYPEDEFS_H diff --git a/Include/structmember.h b/Include/structmember.h index 93b7aff809184..acce4381d4e00 100644 --- a/Include/structmember.h +++ b/Include/structmember.h @@ -15,13 +15,13 @@ extern "C" { flag is set). The array must be terminated with an entry whose name pointer is NULL. */ -typedef struct PyMemberDef { +struct PyMemberDef { const char *name; int type; Py_ssize_t offset; int flags; const char *doc; -} PyMemberDef; +}; /* Types */ #define T_SHORT 0 diff --git a/Makefile.pre.in b/Makefile.pre.in index 2a3e0fbceb4c0..0383853901df1 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1488,6 +1488,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/pystrtod.h \ $(srcdir)/Include/pythonrun.h \ $(srcdir)/Include/pythread.h \ + $(srcdir)/Include/pytypedefs.h \ $(srcdir)/Include/rangeobject.h \ $(srcdir)/Include/setobject.h \ $(srcdir)/Include/sliceobject.h \ diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index c2f1a016bc111..96996988ec36b 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -279,6 +279,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index b300103bbb420..92c6bf8535b98 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -192,6 +192,9 @@ Include + + Include + Include From webhook-mailer at python.org Thu Feb 24 11:52:29 2022 From: webhook-mailer at python.org (vstinner) Date: Thu, 24 Feb 2022 16:52:29 -0000 Subject: [Python-checkins] bpo-45459: C API uses type names rather than structure names (GH-31528) Message-ID: https://github.com/python/cpython/commit/042f31da552c19054acd3ef7bb6cfd857bce172b commit: 042f31da552c19054acd3ef7bb6cfd857bce172b branch: main author: Victor Stinner committer: vstinner date: 2022-02-24T17:51:59+01:00 summary: bpo-45459: C API uses type names rather than structure names (GH-31528) Thanks to the new pytypedefs.h, it becomes to use type names like PyObject rather like structure names like "struct _object". files: M Include/boolobject.h M Include/cpython/abstract.h M Include/cpython/descrobject.h M Include/cpython/dictobject.h M Include/cpython/frameobject.h M Include/cpython/import.h M Include/cpython/object.h M Include/cpython/pystate.h M Include/cpython/traceback.h M Include/descrobject.h M Include/modsupport.h M Include/moduleobject.h M Include/object.h M Include/pystate.h M Include/structmember.h M Include/structseq.h M Objects/call.c M Objects/dictobject.c M Objects/moduleobject.c M Objects/typeobject.c M Python/import.c M Python/pystate.c M Tools/clinic/clinic.py diff --git a/Include/boolobject.h b/Include/boolobject.h index cda6f89a99e9a..28068d1cbe593 100644 --- a/Include/boolobject.h +++ b/Include/boolobject.h @@ -15,8 +15,8 @@ PyAPI_DATA(PyTypeObject) PyBool_Type; Don't forget to apply Py_INCREF() when returning either!!! */ /* Don't use these directly */ -PyAPI_DATA(struct _longobject) _Py_FalseStruct; -PyAPI_DATA(struct _longobject) _Py_TrueStruct; +PyAPI_DATA(PyLongObject) _Py_FalseStruct; +PyAPI_DATA(PyLongObject) _Py_TrueStruct; /* Use these macros */ #define Py_False ((PyObject *) &_Py_FalseStruct) diff --git a/Include/cpython/abstract.h b/Include/cpython/abstract.h index cf142b8a0bfb2..bdc0c49cd4c18 100644 --- a/Include/cpython/abstract.h +++ b/Include/cpython/abstract.h @@ -133,7 +133,7 @@ PyAPI_FUNC(PyObject *) _PyObject_CallMethodId_SizeT(PyObject *obj, PyAPI_FUNC(PyObject *) _PyObject_CallMethodIdObjArgs( PyObject *obj, - struct _Py_Identifier *name, + _Py_Identifier *name, ...); static inline PyObject * diff --git a/Include/cpython/descrobject.h b/Include/cpython/descrobject.h index 5d6c3a24d951e..e2ea1b9a2d305 100644 --- a/Include/cpython/descrobject.h +++ b/Include/cpython/descrobject.h @@ -43,7 +43,7 @@ typedef struct { typedef struct { PyDescr_COMMON; - struct PyMemberDef *d_member; + PyMemberDef *d_member; } PyMemberDescrObject; typedef struct { diff --git a/Include/cpython/dictobject.h b/Include/cpython/dictobject.h index d5119644c5212..68b4593acf6c4 100644 --- a/Include/cpython/dictobject.h +++ b/Include/cpython/dictobject.h @@ -32,7 +32,7 @@ PyAPI_FUNC(PyObject *) _PyDict_GetItem_KnownHash(PyObject *mp, PyObject *key, Py_hash_t hash); PyAPI_FUNC(PyObject *) _PyDict_GetItemWithError(PyObject *dp, PyObject *key); PyAPI_FUNC(PyObject *) _PyDict_GetItemIdWithError(PyObject *dp, - struct _Py_Identifier *key); + _Py_Identifier *key); PyAPI_FUNC(PyObject *) _PyDict_GetItemStringWithError(PyObject *, const char *); PyAPI_FUNC(PyObject *) PyDict_SetDefault( PyObject *mp, PyObject *key, PyObject *defaultobj); @@ -49,7 +49,7 @@ PyAPI_FUNC(int) _PyDict_Next( /* Get the number of items of a dictionary. */ #define PyDict_GET_SIZE(mp) (assert(PyDict_Check(mp)),((PyDictObject *)mp)->ma_used) PyAPI_FUNC(int) _PyDict_Contains_KnownHash(PyObject *, PyObject *, Py_hash_t); -PyAPI_FUNC(int) _PyDict_ContainsId(PyObject *, struct _Py_Identifier *); +PyAPI_FUNC(int) _PyDict_ContainsId(PyObject *, _Py_Identifier *); PyAPI_FUNC(PyObject *) _PyDict_NewPresized(Py_ssize_t minused); PyAPI_FUNC(void) _PyDict_MaybeUntrack(PyObject *mp); PyAPI_FUNC(int) _PyDict_HasOnlyStringKeys(PyObject *mp); @@ -66,9 +66,9 @@ PyObject *_PyDict_FromKeys(PyObject *, PyObject *, PyObject *); argument is raised. */ PyAPI_FUNC(int) _PyDict_MergeEx(PyObject *mp, PyObject *other, int override); -PyAPI_FUNC(int) _PyDict_SetItemId(PyObject *dp, struct _Py_Identifier *key, PyObject *item); +PyAPI_FUNC(int) _PyDict_SetItemId(PyObject *dp, _Py_Identifier *key, PyObject *item); -PyAPI_FUNC(int) _PyDict_DelItemId(PyObject *mp, struct _Py_Identifier *key); +PyAPI_FUNC(int) _PyDict_DelItemId(PyObject *mp, _Py_Identifier *key); PyAPI_FUNC(void) _PyDict_DebugMallocStats(FILE *out); int _PyObjectDict_SetItem(PyTypeObject *tp, PyObject **dictptr, PyObject *name, PyObject *value); diff --git a/Include/cpython/frameobject.h b/Include/cpython/frameobject.h index 67f98a7642beb..3d862d9deb000 100644 --- a/Include/cpython/frameobject.h +++ b/Include/cpython/frameobject.h @@ -6,7 +6,7 @@ struct _frame { PyObject_HEAD - struct _frame *f_back; /* previous frame, or NULL */ + PyFrameObject *f_back; /* previous frame, or NULL */ struct _interpreter_frame *f_frame; /* points to the frame data */ PyObject *f_trace; /* Trace function */ int f_lineno; /* Current line number. Only valid if non-zero */ diff --git a/Include/cpython/import.h b/Include/cpython/import.h index da9fb770a9245..c734802ff254b 100644 --- a/Include/cpython/import.h +++ b/Include/cpython/import.h @@ -6,7 +6,7 @@ PyMODINIT_FUNC PyInit__imp(void); PyAPI_FUNC(int) _PyImport_IsInitialized(PyInterpreterState *); -PyAPI_FUNC(PyObject *) _PyImport_GetModuleId(struct _Py_Identifier *name); +PyAPI_FUNC(PyObject *) _PyImport_GetModuleId(_Py_Identifier *name); PyAPI_FUNC(int) _PyImport_SetModule(PyObject *name, PyObject *module); PyAPI_FUNC(int) _PyImport_SetModuleString(const char *name, PyObject* module); diff --git a/Include/cpython/object.h b/Include/cpython/object.h index 65d7d85b7075d..5f978eec46580 100644 --- a/Include/cpython/object.h +++ b/Include/cpython/object.h @@ -201,11 +201,11 @@ struct _typeobject { iternextfunc tp_iternext; /* Attribute descriptor and subclassing stuff */ - struct PyMethodDef *tp_methods; - struct PyMemberDef *tp_members; - struct PyGetSetDef *tp_getset; + PyMethodDef *tp_methods; + PyMemberDef *tp_members; + PyGetSetDef *tp_getset; // Strong reference on a heap type, borrowed reference on a static type - struct _typeobject *tp_base; + PyTypeObject *tp_base; PyObject *tp_dict; descrgetfunc tp_descr_get; descrsetfunc tp_descr_set; @@ -262,7 +262,7 @@ PyAPI_FUNC(PyObject *) _PyObject_LookupSpecialId(PyObject *, _Py_Identifier *); PyAPI_FUNC(PyTypeObject *) _PyType_CalculateMetaclass(PyTypeObject *, PyObject *); PyAPI_FUNC(PyObject *) _PyType_GetDocFromInternalDoc(const char *, const char *); PyAPI_FUNC(PyObject *) _PyType_GetTextSignatureFromInternalDoc(const char *, const char *); -PyAPI_FUNC(PyObject *) PyType_GetModuleByDef(PyTypeObject *, struct PyModuleDef *); +PyAPI_FUNC(PyObject *) PyType_GetModuleByDef(PyTypeObject *, PyModuleDef *); PyAPI_FUNC(int) PyObject_Print(PyObject *, FILE *, int); PyAPI_FUNC(void) _Py_BreakPoint(void); @@ -270,8 +270,8 @@ PyAPI_FUNC(void) _PyObject_Dump(PyObject *); PyAPI_FUNC(int) _PyObject_IsFreed(PyObject *); PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *); -PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, struct _Py_Identifier *); -PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, struct _Py_Identifier *, PyObject *); +PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, _Py_Identifier *); +PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, _Py_Identifier *, PyObject *); /* Replacements of PyObject_GetAttr() and _PyObject_GetAttrId() which don't raise AttributeError. @@ -282,7 +282,7 @@ PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, struct _Py_Identifier *, PyObjec is raised. */ PyAPI_FUNC(int) _PyObject_LookupAttr(PyObject *, PyObject *, PyObject **); -PyAPI_FUNC(int) _PyObject_LookupAttrId(PyObject *, struct _Py_Identifier *, PyObject **); +PyAPI_FUNC(int) _PyObject_LookupAttrId(PyObject *, _Py_Identifier *, PyObject **); PyAPI_FUNC(int) _PyObject_GetMethod(PyObject *obj, PyObject *name, PyObject **method); @@ -461,8 +461,8 @@ passed as second argument to Py_TRASHCAN_BEGIN(). */ /* Python 3.9 private API, invoked by the macros below. */ -PyAPI_FUNC(int) _PyTrash_begin(struct _ts *tstate, PyObject *op); -PyAPI_FUNC(void) _PyTrash_end(struct _ts *tstate); +PyAPI_FUNC(int) _PyTrash_begin(PyThreadState *tstate, PyObject *op); +PyAPI_FUNC(void) _PyTrash_end(PyThreadState *tstate); /* Python 3.10 private API, invoked by the Py_TRASHCAN_BEGIN(). */ PyAPI_FUNC(int) _PyTrash_cond(PyObject *op, destructor dealloc); diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index 74dd44d6edccb..8150d501d418e 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -79,12 +79,11 @@ typedef struct _stack_chunk { PyObject * data[1]; /* Variable sized */ } _PyStackChunk; -// The PyThreadState typedef is in Include/pystate.h. struct _ts { /* See Python/ceval.c for comments explaining most fields */ - struct _ts *prev; - struct _ts *next; + PyThreadState *prev; + PyThreadState *next; PyInterpreterState *interp; /* Has been initialized to a safe state. @@ -308,12 +307,12 @@ PyAPI_FUNC(const PyConfig*) _Py_GetConfig(void); /* cross-interpreter data */ -struct _xid; - // _PyCrossInterpreterData is similar to Py_buffer as an effectively // opaque struct that holds data outside the object machinery. This // is necessary to pass safely between interpreters in the same process. -typedef struct _xid { +typedef struct _xid _PyCrossInterpreterData; + +struct _xid { // data is the cross-interpreter-safe derivation of a Python object // (see _PyObject_GetCrossInterpreterData). It will be NULL if the // new_object func (below) encodes the data. @@ -339,7 +338,7 @@ typedef struct _xid { // interpreter given the data. The resulting object (a new // reference) will be equivalent to the original object. This field // is required. - PyObject *(*new_object)(struct _xid *); + PyObject *(*new_object)(_PyCrossInterpreterData *); // free is called when the data is released. If it is NULL then // nothing will be done to free the data. For some types this is // okay (e.g. bytes) and for those types this field should be set @@ -350,7 +349,7 @@ typedef struct _xid { // to PyMem_RawFree (the default if not explicitly set to NULL). // The call will happen with the original interpreter activated. void (*free)(void *); -} _PyCrossInterpreterData; +}; PyAPI_FUNC(int) _PyObject_GetCrossInterpreterData(PyObject *, _PyCrossInterpreterData *); PyAPI_FUNC(PyObject *) _PyCrossInterpreterData_NewObject(_PyCrossInterpreterData *); @@ -360,7 +359,7 @@ PyAPI_FUNC(int) _PyObject_CheckCrossInterpreterData(PyObject *); /* cross-interpreter data registry */ -typedef int (*crossinterpdatafunc)(PyObject *, struct _xid *); +typedef int (*crossinterpdatafunc)(PyObject *, _PyCrossInterpreterData *); PyAPI_FUNC(int) _PyCrossInterpreterData_RegisterClass(PyTypeObject *, crossinterpdatafunc); PyAPI_FUNC(crossinterpdatafunc) _PyCrossInterpreterData_Lookup(PyObject *); diff --git a/Include/cpython/traceback.h b/Include/cpython/traceback.h index d0dde335cfee5..a4e087b2b4ece 100644 --- a/Include/cpython/traceback.h +++ b/Include/cpython/traceback.h @@ -2,13 +2,15 @@ # error "this header file must not be included directly" #endif -typedef struct _traceback { +typedef struct _traceback PyTracebackObject; + +struct _traceback { PyObject_HEAD - struct _traceback *tb_next; + PyTracebackObject *tb_next; PyFrameObject *tb_frame; int tb_lasti; int tb_lineno; -} PyTracebackObject; +}; PyAPI_FUNC(int) _Py_DisplaySourceLine(PyObject *, PyObject *, int, int, int *, PyObject **); PyAPI_FUNC(void) _PyTraceback_Add(const char *, const char *, int); diff --git a/Include/descrobject.h b/Include/descrobject.h index aefa7600346b5..77f221df07714 100644 --- a/Include/descrobject.h +++ b/Include/descrobject.h @@ -23,15 +23,11 @@ PyAPI_DATA(PyTypeObject) PyMethodDescr_Type; PyAPI_DATA(PyTypeObject) PyWrapperDescr_Type; PyAPI_DATA(PyTypeObject) PyDictProxy_Type; PyAPI_DATA(PyTypeObject) PyProperty_Type; -// Forward declaration for following prototype -struct PyMemberDef; PyAPI_FUNC(PyObject *) PyDescr_NewMethod(PyTypeObject *, PyMethodDef *); PyAPI_FUNC(PyObject *) PyDescr_NewClassMethod(PyTypeObject *, PyMethodDef *); -PyAPI_FUNC(PyObject *) PyDescr_NewMember(PyTypeObject *, - struct PyMemberDef *); -PyAPI_FUNC(PyObject *) PyDescr_NewGetSet(PyTypeObject *, - struct PyGetSetDef *); +PyAPI_FUNC(PyObject *) PyDescr_NewMember(PyTypeObject *, PyMemberDef *); +PyAPI_FUNC(PyObject *) PyDescr_NewGetSet(PyTypeObject *, PyGetSetDef *); PyAPI_FUNC(PyObject *) PyDictProxy_New(PyObject *); PyAPI_FUNC(PyObject *) PyWrapper_New(PyObject *, PyObject *); diff --git a/Include/modsupport.h b/Include/modsupport.h index baf47f0038d08..6cc2ad01ebe96 100644 --- a/Include/modsupport.h +++ b/Include/modsupport.h @@ -232,11 +232,9 @@ PyAPI_FUNC(int) PyModule_ExecDef(PyObject *module, PyModuleDef *def); #define PyModule_FromDefAndSpec2 PyModule_FromDefAndSpec2TraceRefs #endif -PyAPI_FUNC(PyObject *) PyModule_Create2(struct PyModuleDef*, - int apiver); +PyAPI_FUNC(PyObject *) PyModule_Create2(PyModuleDef*, int apiver); #ifndef Py_LIMITED_API -PyAPI_FUNC(PyObject *) _PyModule_CreateInitialized(struct PyModuleDef*, - int apiver); +PyAPI_FUNC(PyObject *) _PyModule_CreateInitialized(PyModuleDef*, int apiver); #endif #ifdef Py_LIMITED_API diff --git a/Include/moduleobject.h b/Include/moduleobject.h index 0f406832b58ea..2d41f76df4b22 100644 --- a/Include/moduleobject.h +++ b/Include/moduleobject.h @@ -32,12 +32,12 @@ PyAPI_FUNC(void) _PyModule_Clear(PyObject *); PyAPI_FUNC(void) _PyModule_ClearDict(PyObject *); PyAPI_FUNC(int) _PyModuleSpec_IsInitializing(PyObject *); #endif -PyAPI_FUNC(struct PyModuleDef*) PyModule_GetDef(PyObject*); +PyAPI_FUNC(PyModuleDef*) PyModule_GetDef(PyObject*); PyAPI_FUNC(void*) PyModule_GetState(PyObject*); #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03050000 /* New in 3.5 */ -PyAPI_FUNC(PyObject *) PyModuleDef_Init(struct PyModuleDef*); +PyAPI_FUNC(PyObject *) PyModuleDef_Init(PyModuleDef*); PyAPI_DATA(PyTypeObject) PyModuleDef_Type; #endif @@ -56,6 +56,7 @@ typedef struct PyModuleDef_Base { } struct PyModuleDef_Slot; + #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03050000 /* New in 3.5 */ typedef struct PyModuleDef_Slot{ @@ -78,7 +79,7 @@ struct PyModuleDef { const char* m_doc; Py_ssize_t m_size; PyMethodDef *m_methods; - struct PyModuleDef_Slot* m_slots; + PyModuleDef_Slot *m_slots; traverseproc m_traverse; inquiry m_clear; freefunc m_free; diff --git a/Include/object.h b/Include/object.h index 5313ea6f867b9..317515d2c8151 100644 --- a/Include/object.h +++ b/Include/object.h @@ -63,8 +63,8 @@ whose size is determined when the object is allocated. #ifdef Py_TRACE_REFS /* Define pointers to support a doubly-linked list of all live heap objects. */ #define _PyObject_HEAD_EXTRA \ - struct _object *_ob_next; \ - struct _object *_ob_prev; + PyObject *_ob_next; \ + PyObject *_ob_prev; #define _PyObject_EXTRA_INIT 0, 0, @@ -237,8 +237,8 @@ PyAPI_FUNC(void*) PyType_GetSlot(PyTypeObject*, int); #endif #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03090000 PyAPI_FUNC(PyObject*) PyType_FromModuleAndSpec(PyObject *, PyType_Spec *, PyObject *); -PyAPI_FUNC(PyObject *) PyType_GetModule(struct _typeobject *); -PyAPI_FUNC(void *) PyType_GetModuleState(struct _typeobject *); +PyAPI_FUNC(PyObject *) PyType_GetModule(PyTypeObject *); +PyAPI_FUNC(void *) PyType_GetModuleState(PyTypeObject *); #endif #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030B0000 PyAPI_FUNC(PyObject *) PyType_GetName(PyTypeObject *); diff --git a/Include/pystate.h b/Include/pystate.h index 5b4245ed7f396..e6b4de979c87b 100644 --- a/Include/pystate.h +++ b/Include/pystate.h @@ -40,10 +40,10 @@ PyAPI_FUNC(int64_t) PyInterpreterState_GetID(PyInterpreterState *); /* State unique per thread */ /* New in 3.3 */ -PyAPI_FUNC(int) PyState_AddModule(PyObject*, struct PyModuleDef*); -PyAPI_FUNC(int) PyState_RemoveModule(struct PyModuleDef*); +PyAPI_FUNC(int) PyState_AddModule(PyObject*, PyModuleDef*); +PyAPI_FUNC(int) PyState_RemoveModule(PyModuleDef*); #endif -PyAPI_FUNC(PyObject*) PyState_FindModule(struct PyModuleDef*); +PyAPI_FUNC(PyObject*) PyState_FindModule(PyModuleDef*); PyAPI_FUNC(PyThreadState *) PyThreadState_New(PyInterpreterState *); PyAPI_FUNC(void) PyThreadState_Clear(PyThreadState *); diff --git a/Include/structmember.h b/Include/structmember.h index acce4381d4e00..65a777d5f5211 100644 --- a/Include/structmember.h +++ b/Include/structmember.h @@ -65,8 +65,8 @@ struct PyMemberDef { #define PY_AUDIT_READ READ_RESTRICTED /* Current API, use this */ -PyAPI_FUNC(PyObject *) PyMember_GetOne(const char *, struct PyMemberDef *); -PyAPI_FUNC(int) PyMember_SetOne(char *, struct PyMemberDef *, PyObject *); +PyAPI_FUNC(PyObject *) PyMember_GetOne(const char *, PyMemberDef *); +PyAPI_FUNC(int) PyMember_SetOne(char *, PyMemberDef *, PyObject *); #ifdef __cplusplus diff --git a/Include/structseq.h b/Include/structseq.h index e89265a67c322..4f5c09f7ba05e 100644 --- a/Include/structseq.h +++ b/Include/structseq.h @@ -15,7 +15,7 @@ typedef struct PyStructSequence_Field { typedef struct PyStructSequence_Desc { const char *name; const char *doc; - struct PyStructSequence_Field *fields; + PyStructSequence_Field *fields; int n_in_sequence; } PyStructSequence_Desc; diff --git a/Objects/call.c b/Objects/call.c index c3939ea0d07da..9646ad2d77507 100644 --- a/Objects/call.c +++ b/Objects/call.c @@ -884,8 +884,7 @@ PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ...) PyObject * -_PyObject_CallMethodIdObjArgs(PyObject *obj, - struct _Py_Identifier *name, ...) +_PyObject_CallMethodIdObjArgs(PyObject *obj, _Py_Identifier *name, ...) { PyThreadState *tstate = _PyThreadState_GET(); if (obj == NULL || name == NULL) { diff --git a/Objects/dictobject.c b/Objects/dictobject.c index fca879f4e3cdd..68b79f2515682 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -1500,7 +1500,7 @@ _PyDict_GetItemWithError(PyObject *dp, PyObject *kv) } PyObject * -_PyDict_GetItemIdWithError(PyObject *dp, struct _Py_Identifier *key) +_PyDict_GetItemIdWithError(PyObject *dp, _Py_Identifier *key) { PyObject *kv; kv = _PyUnicode_FromId(key); /* borrowed */ @@ -3411,7 +3411,7 @@ _PyDict_Contains_KnownHash(PyObject *op, PyObject *key, Py_hash_t hash) } int -_PyDict_ContainsId(PyObject *op, struct _Py_Identifier *key) +_PyDict_ContainsId(PyObject *op, _Py_Identifier *key) { PyObject *kv = _PyUnicode_FromId(key); /* borrowed */ if (kv == NULL) { @@ -3589,7 +3589,7 @@ PyDict_GetItemString(PyObject *v, const char *key) } int -_PyDict_SetItemId(PyObject *v, struct _Py_Identifier *key, PyObject *item) +_PyDict_SetItemId(PyObject *v, _Py_Identifier *key, PyObject *item) { PyObject *kv; kv = _PyUnicode_FromId(key); /* borrowed */ diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c index bd5e5611ec27e..72ed9bb82f970 100644 --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -19,7 +19,7 @@ static PyMemberDef module_members[] = { PyTypeObject PyModuleDef_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "moduledef", /* tp_name */ - sizeof(struct PyModuleDef), /* tp_basicsize */ + sizeof(PyModuleDef), /* tp_basicsize */ 0, /* tp_itemsize */ }; @@ -32,13 +32,13 @@ _PyModule_IsExtension(PyObject *obj) } PyModuleObject *module = (PyModuleObject*)obj; - struct PyModuleDef *def = module->md_def; + PyModuleDef *def = module->md_def; return (def != NULL && def->m_methods != NULL); } PyObject* -PyModuleDef_Init(struct PyModuleDef* def) +PyModuleDef_Init(PyModuleDef* def) { assert(PyModuleDef_Type.tp_flags & Py_TPFLAGS_READY); if (def->m_base.m_index == 0) { @@ -182,7 +182,7 @@ _add_methods_to_object(PyObject *module, PyObject *name, PyMethodDef *functions) } PyObject * -PyModule_Create2(struct PyModuleDef* module, int module_api_version) +PyModule_Create2(PyModuleDef* module, int module_api_version) { if (!_PyImport_IsInitialized(_PyInterpreterState_GET())) { PyErr_SetString(PyExc_SystemError, @@ -193,7 +193,7 @@ PyModule_Create2(struct PyModuleDef* module, int module_api_version) } PyObject * -_PyModule_CreateInitialized(struct PyModuleDef* module, int module_api_version) +_PyModule_CreateInitialized(PyModuleDef* module, int module_api_version) { const char* name; PyModuleObject *m; @@ -256,7 +256,7 @@ _PyModule_CreateInitialized(struct PyModuleDef* module, int module_api_version) } PyObject * -PyModule_FromDefAndSpec2(struct PyModuleDef* def, PyObject *spec, int module_api_version) +PyModule_FromDefAndSpec2(PyModuleDef* def, PyObject *spec, int module_api_version) { PyModuleDef_Slot* cur_slot; PyObject *(*create)(PyObject *, PyModuleDef*) = NULL; diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 4b7035c81e533..286261226fa8e 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3716,7 +3716,7 @@ PyType_GetModuleState(PyTypeObject *type) * given PyModuleDef. */ PyObject * -PyType_GetModuleByDef(PyTypeObject *type, struct PyModuleDef *def) +PyType_GetModuleByDef(PyTypeObject *type, PyModuleDef *def) { assert(PyType_Check(type)); @@ -3869,7 +3869,7 @@ _PyType_Lookup(PyTypeObject *type, PyObject *name) } PyObject * -_PyType_LookupId(PyTypeObject *type, struct _Py_Identifier *name) +_PyType_LookupId(PyTypeObject *type, _Py_Identifier *name) { PyObject *oname; oname = _PyUnicode_FromId(name); /* borrowed */ diff --git a/Python/import.c b/Python/import.c index 74f8e1dd4c30d..982ec8cfe631a 100644 --- a/Python/import.c +++ b/Python/import.c @@ -283,7 +283,7 @@ _PyImport_IsInitialized(PyInterpreterState *interp) } PyObject * -_PyImport_GetModuleId(struct _Py_Identifier *nameid) +_PyImport_GetModuleId(_Py_Identifier *nameid) { PyObject *name = _PyUnicode_FromId(nameid); /* borrowed */ if (name == NULL) { diff --git a/Python/pystate.c b/Python/pystate.c index a85460c15103d..85000016cb4df 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -862,7 +862,7 @@ _PyThreadState_SetCurrent(PyThreadState *tstate) } PyObject* -PyState_FindModule(struct PyModuleDef* module) +PyState_FindModule(PyModuleDef* module) { Py_ssize_t index = module->m_base.m_index; PyInterpreterState *state = _PyInterpreterState_GET(); @@ -881,7 +881,7 @@ PyState_FindModule(struct PyModuleDef* module) } int -_PyState_AddModule(PyThreadState *tstate, PyObject* module, struct PyModuleDef* def) +_PyState_AddModule(PyThreadState *tstate, PyObject* module, PyModuleDef* def) { if (!def) { assert(_PyErr_Occurred(tstate)); @@ -914,7 +914,7 @@ _PyState_AddModule(PyThreadState *tstate, PyObject* module, struct PyModuleDef* } int -PyState_AddModule(PyObject* module, struct PyModuleDef* def) +PyState_AddModule(PyObject* module, PyModuleDef* def) { if (!def) { Py_FatalError("module definition is NULL"); @@ -935,7 +935,7 @@ PyState_AddModule(PyObject* module, struct PyModuleDef* def) } int -PyState_RemoveModule(struct PyModuleDef* def) +PyState_RemoveModule(PyModuleDef* def) { PyThreadState *tstate = _PyThreadState_GET(); PyInterpreterState *interp = tstate->interp; diff --git a/Tools/clinic/clinic.py b/Tools/clinic/clinic.py index 4b4ebb8313adc..14252b2514ea8 100755 --- a/Tools/clinic/clinic.py +++ b/Tools/clinic/clinic.py @@ -3665,8 +3665,7 @@ def pre_render(self): self.show_in_signature = False # tp_new (METHOD_NEW) functions are of type newfunc: - # typedef PyObject *(*newfunc)(struct _typeobject *, PyObject *, PyObject *); - # PyTypeObject is a typedef for struct _typeobject. + # typedef PyObject *(*newfunc)(PyTypeObject *, PyObject *, PyObject *); # # tp_init (METHOD_INIT) functions are of type initproc: # typedef int (*initproc)(PyObject *, PyObject *, PyObject *); From webhook-mailer at python.org Thu Feb 24 11:54:15 2022 From: webhook-mailer at python.org (vstinner) Date: Thu, 24 Feb 2022 16:54:15 -0000 Subject: [Python-checkins] bpo-46430: Fix memory leak in interned strings of deep-frozen modules (GH-31549) Message-ID: https://github.com/python/cpython/commit/4dc746310bd37ad6b381f9176acd167d445f4385 commit: 4dc746310bd37ad6b381f9176acd167d445f4385 branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: vstinner date: 2022-02-24T17:54:06+01:00 summary: bpo-46430: Fix memory leak in interned strings of deep-frozen modules (GH-31549) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-24-07-33-29.bpo-46430.c91TAg.rst M Include/internal/pycore_pylifecycle.h M Programs/_bootstrap_python.c M Programs/_freeze_module.c M Python/pylifecycle.c M Tools/scripts/deepfreeze.py diff --git a/Include/internal/pycore_pylifecycle.h b/Include/internal/pycore_pylifecycle.h index 439bc5b470bb2..deac6ee3d3f41 100644 --- a/Include/internal/pycore_pylifecycle.h +++ b/Include/internal/pycore_pylifecycle.h @@ -65,7 +65,7 @@ extern PyStatus _Py_HashRandomization_Init(const PyConfig *); extern PyStatus _PyImportZip_Init(PyThreadState *tstate); extern PyStatus _PyGC_Init(PyInterpreterState *interp); extern PyStatus _PyAtExit_Init(PyInterpreterState *interp); - +extern void _Py_Deepfreeze_Init(void); /* Various internal finalizers */ diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-24-07-33-29.bpo-46430.c91TAg.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-24-07-33-29.bpo-46430.c91TAg.rst new file mode 100644 index 0000000000000..0ae128ba7b73b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-24-07-33-29.bpo-46430.c91TAg.rst @@ -0,0 +1 @@ +Fix memory leak in interned strings of deep-frozen modules. diff --git a/Programs/_bootstrap_python.c b/Programs/_bootstrap_python.c index f4d0167b62e71..75d455ca17983 100644 --- a/Programs/_bootstrap_python.c +++ b/Programs/_bootstrap_python.c @@ -14,9 +14,13 @@ #include "Python/frozen_modules/importlib._bootstrap_external.h" /* End includes */ -/* Empty finalizer for deepfrozen modules*/ +/* Empty initializer for deepfrozen modules */ +void _Py_Deepfreeze_Init(void) +{ +} +/* Empty finalizer for deepfrozen modules */ void -_Py_Deepfreeze_Fini(void) +_Py_Deepfreeze_Fini(void) { } diff --git a/Programs/_freeze_module.c b/Programs/_freeze_module.c index 99a1d4dfc261f..d5a236a0c635c 100644 --- a/Programs/_freeze_module.c +++ b/Programs/_freeze_module.c @@ -22,6 +22,10 @@ #include #endif +/* Empty initializer for deepfrozen modules */ +void _Py_Deepfreeze_Init(void) +{ +} /* Empty finalizer for deepfrozen modules */ void _Py_Deepfreeze_Fini(void) diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index b6310c9aeb808..9dd769a0e87b1 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -754,7 +754,6 @@ pycore_init_types(PyInterpreterState *interp) if (_PyStatus_EXCEPTION(status)) { return status; } - return _PyStatus_OK(); } @@ -827,7 +826,10 @@ pycore_interp_init(PyThreadState *tstate) if (_PyStatus_EXCEPTION(status)) { return status; } - + // Intern strings in deep-frozen modules first so that others + // can use it instead of creating a heap allocated string. + _Py_Deepfreeze_Init(); + status = pycore_init_types(interp); if (_PyStatus_EXCEPTION(status)) { goto done; diff --git a/Tools/scripts/deepfreeze.py b/Tools/scripts/deepfreeze.py index 0edf3af71d993..b62be3713feb8 100644 --- a/Tools/scripts/deepfreeze.py +++ b/Tools/scripts/deepfreeze.py @@ -110,6 +110,7 @@ def __init__(self, file: TextIO) -> None: self.hits, self.misses = 0, 0 self.patchups: list[str] = [] self.deallocs: list[str] = [] + self.interns: list[str] = [] self.write('#include "Python.h"') self.write('#include "internal/pycore_gc.h"') self.write('#include "internal/pycore_code.h"') @@ -279,7 +280,7 @@ def generate_code(self, name: str, code: types.CodeType) -> str: self.write(f".co_cellvars = {co_cellvars},") self.write(f".co_freevars = {co_freevars},") self.deallocs.append(f"_PyStaticCode_Dealloc(&{name});") - self.patchups.append(f"_PyStaticCode_InternStrings(&{name});") + self.interns.append(f"_PyStaticCode_InternStrings(&{name});") return f"& {name}.ob_base" def generate_tuple(self, name: str, t: Tuple[object, ...]) -> str: @@ -446,6 +447,9 @@ def generate(args: list[str], output: TextIO) -> None: with printer.block(f"void\n_Py_Deepfreeze_Fini(void)"): for p in printer.deallocs: printer.write(p) + with printer.block(f"void\n_Py_Deepfreeze_Init(void)"): + for p in printer.interns: + printer.write(p) if verbose: print(f"Cache hits: {printer.hits}, misses: {printer.misses}") From webhook-mailer at python.org Thu Feb 24 14:35:02 2022 From: webhook-mailer at python.org (markshannon) Date: Thu, 24 Feb 2022 19:35:02 -0000 Subject: [Python-checkins] bpo-45107: Specialize `LOAD_METHOD` for instances with dict. (GH-31531) Message-ID: https://github.com/python/cpython/commit/2a6ece572ca38f989fca66f4c053cb16550bccd4 commit: 2a6ece572ca38f989fca66f4c053cb16550bccd4 branch: main author: Mark Shannon committer: markshannon date: 2022-02-24T19:34:57Z summary: bpo-45107: Specialize `LOAD_METHOD` for instances with dict. (GH-31531) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-23-15-26-02.bpo-45107.axcgHn.rst M Include/internal/pycore_object.h M Include/opcode.h M Lib/opcode.py M Python/ceval.c M Python/opcode_targets.h M Python/specialize.c diff --git a/Include/internal/pycore_object.h b/Include/internal/pycore_object.h index 65abc1884c3bb..06671b5057afa 100644 --- a/Include/internal/pycore_object.h +++ b/Include/internal/pycore_object.h @@ -227,6 +227,8 @@ static inline PyObject **_PyObject_ManagedDictPointer(PyObject *obj) return ((PyObject **)obj)-3; } +#define MANAGED_DICT_OFFSET (((int)sizeof(PyObject *))*-3) + extern PyObject ** _PyObject_DictPointer(PyObject *); extern int _PyObject_VisitInstanceAttributes(PyObject *self, visitproc visit, void *arg); extern void _PyObject_ClearInstanceAttributes(PyObject *self); diff --git a/Include/opcode.h b/Include/opcode.h index 50153c19e9b23..d5fa5f42b0920 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -147,41 +147,42 @@ extern "C" { #define LOAD_GLOBAL_MODULE 45 #define LOAD_GLOBAL_BUILTIN 46 #define LOAD_METHOD_ADAPTIVE 47 -#define LOAD_METHOD_CACHED 48 -#define LOAD_METHOD_CLASS 55 -#define LOAD_METHOD_MODULE 56 -#define LOAD_METHOD_NO_DICT 57 -#define PRECALL_ADAPTIVE 58 -#define PRECALL_BUILTIN_CLASS 59 -#define PRECALL_NO_KW_BUILTIN_O 62 -#define PRECALL_NO_KW_BUILTIN_FAST 63 -#define PRECALL_BUILTIN_FAST_WITH_KEYWORDS 64 -#define PRECALL_NO_KW_LEN 65 -#define PRECALL_NO_KW_ISINSTANCE 66 -#define PRECALL_NO_KW_LIST_APPEND 67 -#define PRECALL_NO_KW_METHOD_DESCRIPTOR_O 72 -#define PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 76 -#define PRECALL_NO_KW_STR_1 77 -#define PRECALL_NO_KW_TUPLE_1 78 -#define PRECALL_NO_KW_TYPE_1 79 -#define PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST 80 -#define PRECALL_BOUND_METHOD 81 -#define PRECALL_PYFUNC 131 -#define RESUME_QUICK 140 -#define STORE_ATTR_ADAPTIVE 141 -#define STORE_ATTR_INSTANCE_VALUE 143 -#define STORE_ATTR_SLOT 150 -#define STORE_ATTR_WITH_HINT 153 -#define UNPACK_SEQUENCE_ADAPTIVE 154 -#define UNPACK_SEQUENCE_LIST 158 -#define UNPACK_SEQUENCE_TUPLE 159 -#define UNPACK_SEQUENCE_TWO_TUPLE 161 -#define LOAD_FAST__LOAD_FAST 167 -#define STORE_FAST__LOAD_FAST 168 -#define LOAD_FAST__LOAD_CONST 169 -#define LOAD_CONST__LOAD_FAST 170 -#define STORE_FAST__STORE_FAST 173 -#define LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE 174 +#define LOAD_METHOD_CLASS 48 +#define LOAD_METHOD_MODULE 55 +#define LOAD_METHOD_NO_DICT 56 +#define LOAD_METHOD_WITH_DICT 57 +#define LOAD_METHOD_WITH_VALUES 58 +#define PRECALL_ADAPTIVE 59 +#define PRECALL_BUILTIN_CLASS 62 +#define PRECALL_NO_KW_BUILTIN_O 63 +#define PRECALL_NO_KW_BUILTIN_FAST 64 +#define PRECALL_BUILTIN_FAST_WITH_KEYWORDS 65 +#define PRECALL_NO_KW_LEN 66 +#define PRECALL_NO_KW_ISINSTANCE 67 +#define PRECALL_NO_KW_LIST_APPEND 72 +#define PRECALL_NO_KW_METHOD_DESCRIPTOR_O 76 +#define PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 77 +#define PRECALL_NO_KW_STR_1 78 +#define PRECALL_NO_KW_TUPLE_1 79 +#define PRECALL_NO_KW_TYPE_1 80 +#define PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST 81 +#define PRECALL_BOUND_METHOD 131 +#define PRECALL_PYFUNC 140 +#define RESUME_QUICK 141 +#define STORE_ATTR_ADAPTIVE 143 +#define STORE_ATTR_INSTANCE_VALUE 150 +#define STORE_ATTR_SLOT 153 +#define STORE_ATTR_WITH_HINT 154 +#define UNPACK_SEQUENCE_ADAPTIVE 158 +#define UNPACK_SEQUENCE_LIST 159 +#define UNPACK_SEQUENCE_TUPLE 161 +#define UNPACK_SEQUENCE_TWO_TUPLE 167 +#define LOAD_FAST__LOAD_FAST 168 +#define STORE_FAST__LOAD_FAST 169 +#define LOAD_FAST__LOAD_CONST 170 +#define LOAD_CONST__LOAD_FAST 173 +#define STORE_FAST__STORE_FAST 174 +#define LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE 175 #define DO_TRACING 255 #ifdef NEED_OPCODE_JUMP_TABLES static uint32_t _PyOpcode_RelativeJump[8] = { diff --git a/Lib/opcode.py b/Lib/opcode.py index 71a9347debad7..bb451b9f1a6cd 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -260,10 +260,11 @@ def jabs_op(name, op): "LOAD_GLOBAL_MODULE", "LOAD_GLOBAL_BUILTIN", "LOAD_METHOD_ADAPTIVE", - "LOAD_METHOD_CACHED", "LOAD_METHOD_CLASS", "LOAD_METHOD_MODULE", "LOAD_METHOD_NO_DICT", + "LOAD_METHOD_WITH_DICT", + "LOAD_METHOD_WITH_VALUES", "PRECALL_ADAPTIVE", "PRECALL_BUILTIN_CLASS", "PRECALL_NO_KW_BUILTIN_O", diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-23-15-26-02.bpo-45107.axcgHn.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-23-15-26-02.bpo-45107.axcgHn.rst new file mode 100644 index 0000000000000..2f85e8c701797 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-23-15-26-02.bpo-45107.axcgHn.rst @@ -0,0 +1 @@ +Specialize ``LOAD_METHOD`` for instances with a dict. diff --git a/Python/ceval.c b/Python/ceval.c index f3bdaf1377077..a64a24f8a0a83 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4424,7 +4424,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } } - TARGET(LOAD_METHOD_CACHED) { + TARGET(LOAD_METHOD_WITH_VALUES) { /* LOAD_METHOD, with cached method object */ assert(cframe.use_tracing == 0); PyObject *self = TOP(); @@ -4432,7 +4432,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr SpecializedCacheEntry *caches = GET_CACHE(); _PyAttrCache *cache1 = &caches[-1].attr; _PyObjectCache *cache2 = &caches[-2].obj; - + assert(cache1->tp_version != 0); DEOPT_IF(self_cls->tp_version_tag != cache1->tp_version, LOAD_METHOD); assert(self_cls->tp_flags & Py_TPFLAGS_MANAGED_DICT); PyDictObject *dict = *(PyDictObject**)_PyObject_ManagedDictPointer(self); @@ -4448,6 +4448,38 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr NOTRACE_DISPATCH(); } + TARGET(LOAD_METHOD_WITH_DICT) { + /* LOAD_METHOD, with a dict + Can be either a managed dict, or a tp_dictoffset offset.*/ + assert(cframe.use_tracing == 0); + PyObject *self = TOP(); + PyTypeObject *self_cls = Py_TYPE(self); + SpecializedCacheEntry *caches = GET_CACHE(); + _PyAdaptiveEntry *cache0 = &caches[0].adaptive; + _PyAttrCache *cache1 = &caches[-1].attr; + _PyObjectCache *cache2 = &caches[-2].obj; + + DEOPT_IF(self_cls->tp_version_tag != cache1->tp_version, LOAD_METHOD); + /* Treat index as a signed 16 bit value */ + int dictoffset = *(int16_t *)&cache0->index; + PyDictObject **dictptr = (PyDictObject**)(((char *)self)+dictoffset); + assert( + dictoffset == MANAGED_DICT_OFFSET || + (dictoffset == self_cls->tp_dictoffset && dictoffset > 0) + ); + PyDictObject *dict = *dictptr; + DEOPT_IF(dict == NULL, LOAD_METHOD); + DEOPT_IF(dict->ma_keys->dk_version != cache1->dk_version, LOAD_METHOD); + STAT_INC(LOAD_METHOD, hit); + PyObject *res = cache2->obj; + assert(res != NULL); + assert(_PyType_HasFeature(Py_TYPE(res), Py_TPFLAGS_METHOD_DESCRIPTOR)); + Py_INCREF(res); + SET_TOP(res); + PUSH(self); + NOTRACE_DISPATCH(); + } + TARGET(LOAD_METHOD_NO_DICT) { assert(cframe.use_tracing == 0); PyObject *self = TOP(); diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index fac671eb024f5..3339ab1a04327 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -47,40 +47,40 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_GLOBAL_MODULE, &&TARGET_LOAD_GLOBAL_BUILTIN, &&TARGET_LOAD_METHOD_ADAPTIVE, - &&TARGET_LOAD_METHOD_CACHED, + &&TARGET_LOAD_METHOD_CLASS, &&TARGET_WITH_EXCEPT_START, &&TARGET_GET_AITER, &&TARGET_GET_ANEXT, &&TARGET_BEFORE_ASYNC_WITH, &&TARGET_BEFORE_WITH, &&TARGET_END_ASYNC_FOR, - &&TARGET_LOAD_METHOD_CLASS, &&TARGET_LOAD_METHOD_MODULE, &&TARGET_LOAD_METHOD_NO_DICT, + &&TARGET_LOAD_METHOD_WITH_DICT, + &&TARGET_LOAD_METHOD_WITH_VALUES, &&TARGET_PRECALL_ADAPTIVE, - &&TARGET_PRECALL_BUILTIN_CLASS, &&TARGET_STORE_SUBSCR, &&TARGET_DELETE_SUBSCR, + &&TARGET_PRECALL_BUILTIN_CLASS, &&TARGET_PRECALL_NO_KW_BUILTIN_O, &&TARGET_PRECALL_NO_KW_BUILTIN_FAST, &&TARGET_PRECALL_BUILTIN_FAST_WITH_KEYWORDS, &&TARGET_PRECALL_NO_KW_LEN, &&TARGET_PRECALL_NO_KW_ISINSTANCE, - &&TARGET_PRECALL_NO_KW_LIST_APPEND, &&TARGET_GET_ITER, &&TARGET_GET_YIELD_FROM_ITER, &&TARGET_PRINT_EXPR, &&TARGET_LOAD_BUILD_CLASS, - &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_O, + &&TARGET_PRECALL_NO_KW_LIST_APPEND, &&TARGET_GET_AWAITABLE, &&TARGET_LOAD_ASSERTION_ERROR, &&TARGET_RETURN_GENERATOR, + &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_O, &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS, &&TARGET_PRECALL_NO_KW_STR_1, &&TARGET_PRECALL_NO_KW_TUPLE_1, &&TARGET_PRECALL_NO_KW_TYPE_1, &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST, - &&TARGET_PRECALL_BOUND_METHOD, &&TARGET_LIST_TO_TUPLE, &&TARGET_RETURN_VALUE, &&TARGET_IMPORT_STAR, @@ -130,7 +130,7 @@ static void *opcode_targets[256] = { &&TARGET_POP_JUMP_IF_NOT_NONE, &&TARGET_POP_JUMP_IF_NONE, &&TARGET_RAISE_VARARGS, - &&TARGET_PRECALL_PYFUNC, + &&TARGET_PRECALL_BOUND_METHOD, &&TARGET_MAKE_FUNCTION, &&TARGET_BUILD_SLICE, &&TARGET_JUMP_NO_INTERRUPT, @@ -139,39 +139,40 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_DEREF, &&TARGET_STORE_DEREF, &&TARGET_DELETE_DEREF, + &&TARGET_PRECALL_PYFUNC, &&TARGET_RESUME_QUICK, - &&TARGET_STORE_ATTR_ADAPTIVE, &&TARGET_CALL_FUNCTION_EX, - &&TARGET_STORE_ATTR_INSTANCE_VALUE, + &&TARGET_STORE_ATTR_ADAPTIVE, &&TARGET_EXTENDED_ARG, &&TARGET_LIST_APPEND, &&TARGET_SET_ADD, &&TARGET_MAP_ADD, &&TARGET_LOAD_CLASSDEREF, &&TARGET_COPY_FREE_VARS, - &&TARGET_STORE_ATTR_SLOT, + &&TARGET_STORE_ATTR_INSTANCE_VALUE, &&TARGET_RESUME, &&TARGET_MATCH_CLASS, + &&TARGET_STORE_ATTR_SLOT, &&TARGET_STORE_ATTR_WITH_HINT, - &&TARGET_UNPACK_SEQUENCE_ADAPTIVE, &&TARGET_FORMAT_VALUE, &&TARGET_BUILD_CONST_KEY_MAP, &&TARGET_BUILD_STRING, + &&TARGET_UNPACK_SEQUENCE_ADAPTIVE, &&TARGET_UNPACK_SEQUENCE_LIST, - &&TARGET_UNPACK_SEQUENCE_TUPLE, &&TARGET_LOAD_METHOD, - &&TARGET_UNPACK_SEQUENCE_TWO_TUPLE, + &&TARGET_UNPACK_SEQUENCE_TUPLE, &&TARGET_LIST_EXTEND, &&TARGET_SET_UPDATE, &&TARGET_DICT_MERGE, &&TARGET_DICT_UPDATE, &&TARGET_PRECALL, + &&TARGET_UNPACK_SEQUENCE_TWO_TUPLE, &&TARGET_LOAD_FAST__LOAD_FAST, &&TARGET_STORE_FAST__LOAD_FAST, &&TARGET_LOAD_FAST__LOAD_CONST, - &&TARGET_LOAD_CONST__LOAD_FAST, &&TARGET_CALL, &&TARGET_KW_NAMES, + &&TARGET_LOAD_CONST__LOAD_FAST, &&TARGET_STORE_FAST__STORE_FAST, &&TARGET_LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE, &&_unknown_opcode, @@ -253,6 +254,5 @@ static void *opcode_targets[256] = { &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, - &&_unknown_opcode, &&TARGET_DO_TRACING }; diff --git a/Python/specialize.c b/Python/specialize.c index 1641766712a60..6e4eb46443dd0 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -1062,6 +1062,13 @@ specialize_class_load_method(PyObject *owner, _Py_CODEUNIT *instr, PyObject *nam } } +typedef enum { + MANAGED_VALUES = 1, + MANAGED_DICT = 2, + OFFSET_DICT = 3, + NO_DICT = 4 +} ObjectDictKind; + // Please collect stats carefully before and after modifying. A subtle change // can cause a significant drop in cache hits. A possible test is // python.exe -m test_typing test_re test_dis test_zlib. @@ -1071,8 +1078,8 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, _PyAdaptiveEntry *cache0 = &cache->adaptive; _PyAttrCache *cache1 = &cache[-1].attr; _PyObjectCache *cache2 = &cache[-2].obj; - PyTypeObject *owner_cls = Py_TYPE(owner); + if (PyModule_CheckExact(owner)) { int err = specialize_module_load_attr(owner, instr, name, cache0, LOAD_METHOD, LOAD_METHOD_MODULE); @@ -1102,13 +1109,39 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SPECIALIZATION_FAIL(LOAD_METHOD, load_method_fail_kind(kind)); goto fail; } + ObjectDictKind dictkind; + PyDictKeysObject *keys; if (owner_cls->tp_flags & Py_TPFLAGS_MANAGED_DICT) { - PyObject **owner_dictptr = _PyObject_ManagedDictPointer(owner); - if (*owner_dictptr) { - SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_LOAD_METHOD_HAS_MANAGED_DICT); + PyObject *dict = *_PyObject_ManagedDictPointer(owner); + keys = ((PyHeapTypeObject *)owner_cls)->ht_cached_keys; + if (dict == NULL) { + dictkind = MANAGED_VALUES; + } + else { + dictkind = MANAGED_DICT; + } + } + else { + Py_ssize_t dictoffset = owner_cls->tp_dictoffset; + if (dictoffset < 0 || dictoffset > INT16_MAX) { + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_OUT_OF_RANGE); goto fail; } - PyDictKeysObject *keys = ((PyHeapTypeObject *)owner_cls)->ht_cached_keys; + if (dictoffset == 0) { + dictkind = NO_DICT; + keys = NULL; + } + else { + PyObject *dict = *(PyObject **) ((char *)owner + dictoffset); + if (dict == NULL) { + SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_NO_DICT); + goto fail; + } + keys = ((PyDictObject *)dict)->ma_keys; + dictkind = OFFSET_DICT; + } + } + if (dictkind != NO_DICT) { Py_ssize_t index = _PyDictKeys_StringLookup(keys, name); if (index != DKIX_EMPTY) { SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_LOAD_METHOD_IS_ATTR); @@ -1120,16 +1153,23 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, goto fail; } cache1->dk_version = keys_version; - *instr = _Py_MAKECODEUNIT(LOAD_METHOD_CACHED, _Py_OPARG(*instr)); } - else { - if (owner_cls->tp_dictoffset == 0) { + switch(dictkind) { + case NO_DICT: *instr = _Py_MAKECODEUNIT(LOAD_METHOD_NO_DICT, _Py_OPARG(*instr)); - } - else { - SPECIALIZATION_FAIL(LOAD_METHOD, SPEC_FAIL_LOAD_METHOD_HAS_DICT); - goto fail; - } + break; + case MANAGED_VALUES: + *instr = _Py_MAKECODEUNIT(LOAD_METHOD_WITH_VALUES, _Py_OPARG(*instr)); + break; + case MANAGED_DICT: + *(int16_t *)&cache0->index = (int16_t)MANAGED_DICT_OFFSET; + *instr = _Py_MAKECODEUNIT(LOAD_METHOD_WITH_DICT, _Py_OPARG(*instr)); + break; + case OFFSET_DICT: + assert(owner_cls->tp_dictoffset > 0 && owner_cls->tp_dictoffset <= INT16_MAX); + cache0->index = (uint16_t)owner_cls->tp_dictoffset; + *instr = _Py_MAKECODEUNIT(LOAD_METHOD_WITH_DICT, _Py_OPARG(*instr)); + break; } /* `descr` is borrowed. This is safe for methods (even inherited ones from * super classes!) as long as tp_version_tag is validated for two main reasons: From webhook-mailer at python.org Thu Feb 24 14:52:09 2022 From: webhook-mailer at python.org (tiran) Date: Thu, 24 Feb 2022 19:52:09 -0000 Subject: [Python-checkins] bpo-45898: Remove duplicate symbols from _ctypes/cfield.c (GH-29791) Message-ID: https://github.com/python/cpython/commit/38f331d4656394ae0f425568e26790ace778e076 commit: 38f331d4656394ae0f425568e26790ace778e076 branch: main author: Christian Heimes committer: tiran date: 2022-02-24T20:51:57+01:00 summary: bpo-45898: Remove duplicate symbols from _ctypes/cfield.c (GH-29791) files: A Misc/NEWS.d/next/Library/2021-11-26-10-46-09.bpo-45898.UIfhsb.rst M Modules/_ctypes/cfield.c M PCbuild/_ctypes.vcxproj diff --git a/Misc/NEWS.d/next/Library/2021-11-26-10-46-09.bpo-45898.UIfhsb.rst b/Misc/NEWS.d/next/Library/2021-11-26-10-46-09.bpo-45898.UIfhsb.rst new file mode 100644 index 0000000000000..9da5c258ab7fd --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-11-26-10-46-09.bpo-45898.UIfhsb.rst @@ -0,0 +1,2 @@ +:mod:`ctypes` no longer defines ``ffi_type_*`` symbols in ``cfield.c``. The +symbols have been provided by libffi for over a decade. diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c index 515b280182098..2992d34934194 100644 --- a/Modules/_ctypes/cfield.c +++ b/Modules/_ctypes/cfield.c @@ -1481,55 +1481,40 @@ P_get(void *ptr, Py_ssize_t size) } static struct fielddesc formattable[] = { - { 's', s_set, s_get, &ffi_type_pointer}, - { 'b', b_set, b_get, &ffi_type_schar}, - { 'B', B_set, B_get, &ffi_type_uchar}, - { 'c', c_set, c_get, &ffi_type_schar}, - { 'd', d_set, d_get, &ffi_type_double, d_set_sw, d_get_sw}, - { 'g', g_set, g_get, &ffi_type_longdouble}, - { 'f', f_set, f_get, &ffi_type_float, f_set_sw, f_get_sw}, - { 'h', h_set, h_get, &ffi_type_sshort, h_set_sw, h_get_sw}, - { 'H', H_set, H_get, &ffi_type_ushort, H_set_sw, H_get_sw}, - { 'i', i_set, i_get, &ffi_type_sint, i_set_sw, i_get_sw}, - { 'I', I_set, I_get, &ffi_type_uint, I_set_sw, I_get_sw}, -/* XXX Hm, sizeof(int) == sizeof(long) doesn't hold on every platform */ -/* As soon as we can get rid of the type codes, this is no longer a problem */ -#if SIZEOF_LONG == 4 - { 'l', l_set, l_get, &ffi_type_sint32, l_set_sw, l_get_sw}, - { 'L', L_set, L_get, &ffi_type_uint32, L_set_sw, L_get_sw}, -#elif SIZEOF_LONG == 8 - { 'l', l_set, l_get, &ffi_type_sint64, l_set_sw, l_get_sw}, - { 'L', L_set, L_get, &ffi_type_uint64, L_set_sw, L_get_sw}, -#else -# error -#endif -#if SIZEOF_LONG_LONG == 8 - { 'q', q_set, q_get, &ffi_type_sint64, q_set_sw, q_get_sw}, - { 'Q', Q_set, Q_get, &ffi_type_uint64, Q_set_sw, Q_get_sw}, -#else -# error -#endif - { 'P', P_set, P_get, &ffi_type_pointer}, - { 'z', z_set, z_get, &ffi_type_pointer}, - { 'u', u_set, u_get, NULL}, /* ffi_type set later */ - { 'U', U_set, U_get, &ffi_type_pointer}, - { 'Z', Z_set, Z_get, &ffi_type_pointer}, + { 's', s_set, s_get, NULL}, + { 'b', b_set, b_get, NULL}, + { 'B', B_set, B_get, NULL}, + { 'c', c_set, c_get, NULL}, + { 'd', d_set, d_get, NULL, d_set_sw, d_get_sw}, + { 'g', g_set, g_get, NULL}, + { 'f', f_set, f_get, NULL, f_set_sw, f_get_sw}, + { 'h', h_set, h_get, NULL, h_set_sw, h_get_sw}, + { 'H', H_set, H_get, NULL, H_set_sw, H_get_sw}, + { 'i', i_set, i_get, NULL, i_set_sw, i_get_sw}, + { 'I', I_set, I_get, NULL, I_set_sw, I_get_sw}, + { 'l', l_set, l_get, NULL, l_set_sw, l_get_sw}, + { 'L', L_set, L_get, NULL, L_set_sw, L_get_sw}, + { 'q', q_set, q_get, NULL, q_set_sw, q_get_sw}, + { 'Q', Q_set, Q_get, NULL, Q_set_sw, Q_get_sw}, + { 'P', P_set, P_get, NULL}, + { 'z', z_set, z_get, NULL}, + { 'u', u_set, u_get, NULL}, + { 'U', U_set, U_get, NULL}, + { 'Z', Z_set, Z_get, NULL}, #ifdef MS_WIN32 - { 'X', BSTR_set, BSTR_get, &ffi_type_pointer}, + { 'X', BSTR_set, BSTR_get, NULL}, #endif - { 'v', vBOOL_set, vBOOL_get, &ffi_type_sshort}, -#if SIZEOF__BOOL == 1 - { '?', bool_set, bool_get, &ffi_type_uchar}, /* Also fallback for no native _Bool support */ -#elif SIZEOF__BOOL == SIZEOF_SHORT - { '?', bool_set, bool_get, &ffi_type_ushort}, -#elif SIZEOF__BOOL == SIZEOF_INT - { '?', bool_set, bool_get, &ffi_type_uint, I_set_sw, I_get_sw}, + { 'v', vBOOL_set, vBOOL_get, NULL}, +#if SIZEOF__BOOL == SIZEOF_INT + { '?', bool_set, bool_get, NULL, I_set_sw, I_get_sw}, #elif SIZEOF__BOOL == SIZEOF_LONG - { '?', bool_set, bool_get, &ffi_type_ulong, L_set_sw, L_get_sw}, + { '?', bool_set, bool_get, NULL, L_set_sw, L_get_sw}, #elif SIZEOF__BOOL == SIZEOF_LONG_LONG - { '?', bool_set, bool_get, &ffi_type_ulong, Q_set_sw, Q_get_sw}, + { '?', bool_set, bool_get, NULL, Q_set_sw, Q_get_sw}, +#else + { '?', bool_set, bool_get, NULL}, #endif /* SIZEOF__BOOL */ - { 'O', O_set, O_get, &ffi_type_pointer}, + { 'O', O_set, O_get, NULL}, { 0, NULL, NULL, NULL}, }; @@ -1538,6 +1523,79 @@ static struct fielddesc formattable[] = { Use '?' as code for BOOL. */ +/* Delayed initialization. Windows cannot statically reference dynamically + loaded addresses from DLLs. */ +void +_ctypes_init_fielddesc(void) +{ + struct fielddesc *fd = formattable; + for (; fd->code; ++fd) { + switch (fd->code) { + case 's': fd->pffi_type = &ffi_type_pointer; break; + case 'b': fd->pffi_type = &ffi_type_schar; break; + case 'B': fd->pffi_type = &ffi_type_uchar; break; + case 'c': fd->pffi_type = &ffi_type_schar; break; + case 'd': fd->pffi_type = &ffi_type_double; break; + case 'g': fd->pffi_type = &ffi_type_longdouble; break; + case 'f': fd->pffi_type = &ffi_type_float; break; + case 'h': fd->pffi_type = &ffi_type_sshort; break; + case 'H': fd->pffi_type = &ffi_type_ushort; break; + case 'i': fd->pffi_type = &ffi_type_sint; break; + case 'I': fd->pffi_type = &ffi_type_uint; break; + /* XXX Hm, sizeof(int) == sizeof(long) doesn't hold on every platform */ + /* As soon as we can get rid of the type codes, this is no longer a problem */ + #if SIZEOF_LONG == 4 + case 'l': fd->pffi_type = &ffi_type_sint32; break; + case 'L': fd->pffi_type = &ffi_type_uint32; break; + #elif SIZEOF_LONG == 8 + case 'l': fd->pffi_type = &ffi_type_sint64; break; + case 'L': fd->pffi_type = &ffi_type_uint64; break; + #else + #error + #endif + #if SIZEOF_LONG_LONG == 8 + case 'q': fd->pffi_type = &ffi_type_sint64; break; + case 'Q': fd->pffi_type = &ffi_type_uint64; break; + #else + #error + #endif + case 'P': fd->pffi_type = &ffi_type_pointer; break; + case 'z': fd->pffi_type = &ffi_type_pointer; break; + case 'u': + if (sizeof(wchar_t) == sizeof(short)) + fd->pffi_type = &ffi_type_sshort; + else if (sizeof(wchar_t) == sizeof(int)) + fd->pffi_type = &ffi_type_sint; + else if (sizeof(wchar_t) == sizeof(long)) + fd->pffi_type = &ffi_type_slong; + else + Py_UNREACHABLE(); + break; + case 'U': fd->pffi_type = &ffi_type_pointer; break; + case 'Z': fd->pffi_type = &ffi_type_pointer; break; + #ifdef MS_WIN32 + case 'X': fd->pffi_type = &ffi_type_pointer; break; + #endif + case 'v': fd->pffi_type = &ffi_type_sshort; break; + #if SIZEOF__BOOL == 1 + case '?': fd->pffi_type = &ffi_type_uchar; break; /* Also fallback for no native _Bool support */ + #elif SIZEOF__BOOL == SIZEOF_SHORT + case '?': fd->pffi_type = &ffi_type_ushort; break; + #elif SIZEOF__BOOL == SIZEOF_INT + case '?': fd->pffi_type = &ffi_type_uint; break; + #elif SIZEOF__BOOL == SIZEOF_LONG + case '?': fd->pffi_type = &ffi_type_ulong; break; + #elif SIZEOF__BOOL == SIZEOF_LONG_LONG + case '?': fd->pffi_type = &ffi_type_ulong; break; + #endif /* SIZEOF__BOOL */ + case 'O': fd->pffi_type = &ffi_type_pointer; break; + default: + Py_UNREACHABLE(); + } + } + +} + struct fielddesc * _ctypes_get_fielddesc(const char *fmt) { @@ -1545,13 +1603,7 @@ _ctypes_get_fielddesc(const char *fmt) struct fielddesc *table = formattable; if (!initialized) { - initialized = 1; - if (sizeof(wchar_t) == sizeof(short)) - _ctypes_get_fielddesc("u")->pffi_type = &ffi_type_sshort; - else if (sizeof(wchar_t) == sizeof(int)) - _ctypes_get_fielddesc("u")->pffi_type = &ffi_type_sint; - else if (sizeof(wchar_t) == sizeof(long)) - _ctypes_get_fielddesc("u")->pffi_type = &ffi_type_slong; + _ctypes_init_fielddesc(); } for (; table->code; ++table) { @@ -1561,77 +1613,4 @@ _ctypes_get_fielddesc(const char *fmt) return NULL; } -typedef struct { char c; char x; } s_char; -typedef struct { char c; short x; } s_short; -typedef struct { char c; int x; } s_int; -typedef struct { char c; long x; } s_long; -typedef struct { char c; float x; } s_float; -typedef struct { char c; double x; } s_double; -typedef struct { char c; long double x; } s_long_double; -typedef struct { char c; char *x; } s_char_p; -typedef struct { char c; void *x; } s_void_p; - -/* -#define CHAR_ALIGN (sizeof(s_char) - sizeof(char)) -#define SHORT_ALIGN (sizeof(s_short) - sizeof(short)) -#define LONG_ALIGN (sizeof(s_long) - sizeof(long)) -*/ -#define INT_ALIGN (sizeof(s_int) - sizeof(int)) -#define FLOAT_ALIGN (sizeof(s_float) - sizeof(float)) -#define DOUBLE_ALIGN (sizeof(s_double) - sizeof(double)) -#define LONGDOUBLE_ALIGN (sizeof(s_long_double) - sizeof(long double)) - -/* #define CHAR_P_ALIGN (sizeof(s_char_p) - sizeof(char*)) */ -#define VOID_P_ALIGN (sizeof(s_void_p) - sizeof(void*)) - -/* -#ifdef HAVE_USABLE_WCHAR_T -typedef struct { char c; wchar_t x; } s_wchar; -typedef struct { char c; wchar_t *x; } s_wchar_p; - -#define WCHAR_ALIGN (sizeof(s_wchar) - sizeof(wchar_t)) -#define WCHAR_P_ALIGN (sizeof(s_wchar_p) - sizeof(wchar_t*)) -#endif -*/ - -typedef struct { char c; long long x; } s_long_long; -#define LONG_LONG_ALIGN (sizeof(s_long_long) - sizeof(long long)) - -/* from ffi.h: -typedef struct _ffi_type -{ - size_t size; - unsigned short alignment; - unsigned short type; - struct _ffi_type **elements; -} ffi_type; -*/ - -/* align and size are bogus for void, but they must not be zero */ -ffi_type ffi_type_void = { 1, 1, FFI_TYPE_VOID }; - -ffi_type ffi_type_uint8 = { 1, 1, FFI_TYPE_UINT8 }; -ffi_type ffi_type_sint8 = { 1, 1, FFI_TYPE_SINT8 }; - -ffi_type ffi_type_uint16 = { 2, 2, FFI_TYPE_UINT16 }; -ffi_type ffi_type_sint16 = { 2, 2, FFI_TYPE_SINT16 }; - -ffi_type ffi_type_uint32 = { 4, INT_ALIGN, FFI_TYPE_UINT32 }; -ffi_type ffi_type_sint32 = { 4, INT_ALIGN, FFI_TYPE_SINT32 }; - -ffi_type ffi_type_uint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_UINT64 }; -ffi_type ffi_type_sint64 = { 8, LONG_LONG_ALIGN, FFI_TYPE_SINT64 }; - -ffi_type ffi_type_float = { sizeof(float), FLOAT_ALIGN, FFI_TYPE_FLOAT }; -ffi_type ffi_type_double = { sizeof(double), DOUBLE_ALIGN, FFI_TYPE_DOUBLE }; - -#ifdef ffi_type_longdouble -#undef ffi_type_longdouble -#endif - /* This is already defined on OSX */ -ffi_type ffi_type_longdouble = { sizeof(long double), LONGDOUBLE_ALIGN, - FFI_TYPE_LONGDOUBLE }; - -ffi_type ffi_type_pointer = { sizeof(void *), VOID_P_ALIGN, FFI_TYPE_POINTER }; - /*---------------- EOF ----------------*/ diff --git a/PCbuild/_ctypes.vcxproj b/PCbuild/_ctypes.vcxproj index 69e4271a9bd8f..6ac26f1916c9f 100644 --- a/PCbuild/_ctypes.vcxproj +++ b/PCbuild/_ctypes.vcxproj @@ -94,7 +94,7 @@ - FFI_BUILDING;%(PreprocessorDefinitions) + USING_MALLOC_CLOSURE_DOT_C=1;%(PreprocessorDefinitions) /EXPORT:DllGetClassObject,PRIVATE /EXPORT:DllCanUnloadNow,PRIVATE %(AdditionalOptions) From webhook-mailer at python.org Thu Feb 24 18:03:24 2022 From: webhook-mailer at python.org (vstinner) Date: Thu, 24 Feb 2022 23:03:24 -0000 Subject: [Python-checkins] bpo-1635741: test_embed cheks that Python does not leak (GH-31555) Message-ID: https://github.com/python/cpython/commit/c9c178fdb1af01e441a6c83f3a21a67e5dd9f17d commit: c9c178fdb1af01e441a6c83f3a21a67e5dd9f17d branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T00:03:20+01:00 summary: bpo-1635741: test_embed cheks that Python does not leak (GH-31555) files: M Lib/test/test_cmd_line.py M Lib/test/test_embed.py diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index 352109ed4b2ff..1521b5b50ca24 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -118,7 +118,7 @@ def run_python(*args): self.assertEqual(out.rstrip(), b'{}') self.assertEqual(err, b'') # "-X showrefcount" shows the refcount, but only in debug builds - rc, out, err = run_python('-X', 'showrefcount', '-c', code) + rc, out, err = run_python('-I', '-X', 'showrefcount', '-c', code) self.assertEqual(out.rstrip(), b"{'showrefcount': True}") if Py_DEBUG: # bpo-46417: Tolerate negative reference count which can occur diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 15c6b05916f34..e376331063d86 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -1641,6 +1641,27 @@ def test_frozenmain(self): """).lstrip() self.assertEqual(out, expected) + @unittest.skipUnless(hasattr(sys, 'gettotalrefcount'), + '-X showrefcount requires a Python debug build') + def test_no_memleak(self): + # bpo-1635741: Python must release all memory at exit + cmd = [sys.executable, "-I", "-X", "showrefcount", "-c", "pass"] + proc = subprocess.run(cmd, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True) + self.assertEqual(proc.returncode, 0) + out = proc.stdout.rstrip() + match = re.match(r'^\[(-?\d+) refs, (-?\d+) blocks\]', out) + if not match: + self.fail(f"unexpected output: {out!a}") + refs = int(match.group(1)) + blocks = int(match.group(2)) + # bpo-46417: Tolerate negative reference count which can occur because + # of bugs in C extensions. It is only wrong if it's greater than 0. + self.assertLessEqual(refs, 0, out) + self.assertEqual(blocks, 0, out) + class StdPrinterTests(EmbeddingTestsMixin, unittest.TestCase): # Test PyStdPrinter_Type which is used by _PySys_SetPreliminaryStderr(): From webhook-mailer at python.org Thu Feb 24 18:32:14 2022 From: webhook-mailer at python.org (vstinner) Date: Thu, 24 Feb 2022 23:32:14 -0000 Subject: [Python-checkins] bpo-46623: Skip two test_zlib tests on s390x (GH-31096) Message-ID: https://github.com/python/cpython/commit/9475dc0b8d2a0db40278bbcb88a89b1265a77ec9 commit: 9475dc0b8d2a0db40278bbcb88a89b1265a77ec9 branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T00:32:02+01:00 summary: bpo-46623: Skip two test_zlib tests on s390x (GH-31096) Skip test_pair() and test_speech128() of test_zlib on s390x since they fail if zlib uses the s390x hardware accelerator. files: A Misc/NEWS.d/next/Tests/2022-02-03-09-45-26.bpo-46623.vxzuhV.rst M Lib/test/test_zlib.py diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py index 04fb4d93de45e..f20aad051da96 100644 --- a/Lib/test/test_zlib.py +++ b/Lib/test/test_zlib.py @@ -3,6 +3,7 @@ from test.support import import_helper import binascii import copy +import os import pickle import random import sys @@ -18,6 +19,35 @@ hasattr(zlib.decompressobj(), "copy"), 'requires Decompress.copy()') +# bpo-46623: On s390x, when a hardware accelerator is used, using different +# ways to compress data with zlib can produce different compressed data. +# Simplified test_pair() code: +# +# def func1(data): +# return zlib.compress(data) +# +# def func2(data) +# co = zlib.compressobj() +# x1 = co.compress(data) +# x2 = co.flush() +# return x1 + x2 +# +# On s390x if zlib uses a hardware accelerator, func1() creates a single +# "final" compressed block whereas func2() produces 3 compressed blocks (the +# last one is a final block). On other platforms with no accelerator, func1() +# and func2() produce the same compressed data made of a single (final) +# compressed block. +# +# Only the compressed data is different, the decompression returns the original +# data: +# +# zlib.decompress(func1(data)) == zlib.decompress(func2(data)) == data +# +# Make the assumption that s390x always has an accelerator to simplify the skip +# condition. Windows doesn't have os.uname() but it doesn't support s390x. +skip_on_s390x = unittest.skipIf(hasattr(os, 'uname') and os.uname().machine == 's390x', + 'skipped on s390x') + class VersionTestCase(unittest.TestCase): @@ -182,6 +212,7 @@ def test_keywords(self): bufsize=zlib.DEF_BUF_SIZE), HAMLET_SCENE) + @skip_on_s390x def test_speech128(self): # compress more data data = HAMLET_SCENE * 128 @@ -233,6 +264,7 @@ def test_64bit_compress(self, size): class CompressObjectTestCase(BaseCompressTestCase, unittest.TestCase): # Test compression object + @skip_on_s390x def test_pair(self): # straightforward compress/decompress objects datasrc = HAMLET_SCENE * 128 diff --git a/Misc/NEWS.d/next/Tests/2022-02-03-09-45-26.bpo-46623.vxzuhV.rst b/Misc/NEWS.d/next/Tests/2022-02-03-09-45-26.bpo-46623.vxzuhV.rst new file mode 100644 index 0000000000000..be085c067a376 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-02-03-09-45-26.bpo-46623.vxzuhV.rst @@ -0,0 +1,2 @@ +Skip test_pair() and test_speech128() of test_zlib on s390x since they fail +if zlib uses the s390x hardware accelerator. Patch by Victor Stinner. From webhook-mailer at python.org Thu Feb 24 19:31:36 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 00:31:36 -0000 Subject: [Python-checkins] bpo-46656: Building Python now requires a C11 compiler (GH-31557) Message-ID: https://github.com/python/cpython/commit/5f8b5e2f2150d2223ff9e286bd146de92ff16865 commit: 5f8b5e2f2150d2223ff9e286bd146de92ff16865 branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T01:31:30+01:00 summary: bpo-46656: Building Python now requires a C11 compiler (GH-31557) See PEP 7: https://python.github.io/peps/pep-0007/#c-dialect files: A Misc/NEWS.d/next/Build/2022-02-25-00-51-16.bpo-46656.MD783M.rst M Doc/whatsnew/3.11.rst diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 9744051ede6bb..c717aad9c87f7 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -618,6 +618,9 @@ Changes in the Python API Build Changes ============= +* Building Python now requires a C11 compiler without optional C11 features. + (Contributed by Victor Stinner in :issue:`46656`.) + * CPython can now be built with the ThinLTO option via ``--with-lto=thin``. (Contributed by Dong-hee Na and Brett Holman in :issue:`44340`.) diff --git a/Misc/NEWS.d/next/Build/2022-02-25-00-51-16.bpo-46656.MD783M.rst b/Misc/NEWS.d/next/Build/2022-02-25-00-51-16.bpo-46656.MD783M.rst new file mode 100644 index 0000000000000..f5b789b23eea1 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2022-02-25-00-51-16.bpo-46656.MD783M.rst @@ -0,0 +1,2 @@ +Building Python now requires a C11 compiler without optional C11 features. +Patch by Victor Stinner. From webhook-mailer at python.org Thu Feb 24 19:33:02 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 00:33:02 -0000 Subject: [Python-checkins] bpo-46656: Remove Py_NO_NAN macro (GH-31160) Message-ID: https://github.com/python/cpython/commit/1b2611eb0283055835e5df632a7a735db8c894b8 commit: 1b2611eb0283055835e5df632a7a735db8c894b8 branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T01:32:57+01:00 summary: bpo-46656: Remove Py_NO_NAN macro (GH-31160) Building Python now requires support for floating point Not-a-Number (NaN): remove the Py_NO_NAN macro. files: A Misc/NEWS.d/next/Build/2022-02-06-14-04-20.bpo-46656.ajJjkh.rst M Doc/whatsnew/3.11.rst M Include/floatobject.h M Include/pymath.h M Misc/NEWS.d/next/Build/2022-02-04-21-26-50.bpo-46640.HXUmQp.rst M Modules/cmathmodule.c M Modules/mathmodule.c M Objects/floatobject.c M Python/pystrtod.c diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index c717aad9c87f7..0556a44d4c7d4 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -633,11 +633,13 @@ Build Changes (Contributed by Victor Stinner in :issue:`45440`.) * Building Python now requires a C99 ```` header file providing - a ``NAN`` constant, or the ``__builtin_nan()`` built-in function. If a - platform does not support Not-a-Number (NaN), the ``Py_NO_NAN`` macro can be - defined in the ``pyconfig.h`` file. + a ``NAN`` constant, or the ``__builtin_nan()`` built-in function. (Contributed by Victor Stinner in :issue:`46640`.) +* Building Python now requires support for floating point Not-a-Number (NaN): + remove the ``Py_NO_NAN`` macro. + (Contributed by Victor Stinner in :issue:`46656`.) + * Freelists for object structs can now be disabled. A new :program:`configure` option :option:`!--without-freelists` can be used to disable all freelists except empty tuple singleton. diff --git a/Include/floatobject.h b/Include/floatobject.h index 3b6ca478eaef2..9d2fff3097e8e 100644 --- a/Include/floatobject.h +++ b/Include/floatobject.h @@ -16,9 +16,7 @@ PyAPI_DATA(PyTypeObject) PyFloat_Type; #define PyFloat_Check(op) PyObject_TypeCheck(op, &PyFloat_Type) #define PyFloat_CheckExact(op) Py_IS_TYPE(op, &PyFloat_Type) -#ifdef Py_NAN -# define Py_RETURN_NAN return PyFloat_FromDouble(Py_NAN) -#endif +#define Py_RETURN_NAN return PyFloat_FromDouble(Py_NAN) #define Py_RETURN_INF(sign) \ do { \ diff --git a/Include/pymath.h b/Include/pymath.h index edd0841be9259..772b67e497756 100644 --- a/Include/pymath.h +++ b/Include/pymath.h @@ -50,11 +50,8 @@ # define Py_HUGE_VAL HUGE_VAL #endif -/* Py_NAN - * A value that evaluates to a quiet Not-a-Number (NaN). - * Define Py_NO_NAN in pyconfig.h if your platform doesn't support NaNs. - */ -#if !defined(Py_NAN) && !defined(Py_NO_NAN) +// Py_NAN: Value that evaluates to a quiet Not-a-Number (NaN). +#if !defined(Py_NAN) # if _Py__has_builtin(__builtin_nan) // Built-in implementation of the ISO C99 function nan(): quiet NaN. # define Py_NAN (__builtin_nan("")) diff --git a/Misc/NEWS.d/next/Build/2022-02-04-21-26-50.bpo-46640.HXUmQp.rst b/Misc/NEWS.d/next/Build/2022-02-04-21-26-50.bpo-46640.HXUmQp.rst index c7381110ebbcb..9f11c72f131eb 100644 --- a/Misc/NEWS.d/next/Build/2022-02-04-21-26-50.bpo-46640.HXUmQp.rst +++ b/Misc/NEWS.d/next/Build/2022-02-04-21-26-50.bpo-46640.HXUmQp.rst @@ -1,4 +1,3 @@ Building Python now requires a C99 ```` header file providing a ``NAN`` -constant, or the ``__builtin_nan()`` built-in function. If a platform does not -support Not-a-Number (NaN), the ``Py_NO_NAN`` macro can be defined in the -``pyconfig.h`` file. Patch by Victor Stinner. +constant, or the ``__builtin_nan()`` built-in function. +Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Build/2022-02-06-14-04-20.bpo-46656.ajJjkh.rst b/Misc/NEWS.d/next/Build/2022-02-06-14-04-20.bpo-46656.ajJjkh.rst new file mode 100644 index 0000000000000..98e37862daea7 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2022-02-06-14-04-20.bpo-46656.ajJjkh.rst @@ -0,0 +1,2 @@ +Building Python now requires support for floating point Not-a-Number (NaN): +remove the ``Py_NO_NAN`` macro. Patch by by Victor Stinner. diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c index c0c0c353d196c..2038ac26e6585 100644 --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -113,7 +113,7 @@ c_infj(void) return r; } -#if _PY_SHORT_FLOAT_REPR == 1 || defined(Py_NAN) +#if _PY_SHORT_FLOAT_REPR == 1 static double m_nan(void) @@ -1282,7 +1282,7 @@ cmath_exec(PyObject *mod) PyComplex_FromCComplex(c_infj())) < 0) { return -1; } -#if _PY_SHORT_FLOAT_REPR == 1 || defined(Py_NAN) +#if _PY_SHORT_FLOAT_REPR == 1 if (PyModule_AddObject(mod, "nan", PyFloat_FromDouble(m_nan())) < 0) { return -1; } diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 24ae1460e5a89..f0aaf23845f6d 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -283,7 +283,7 @@ m_inf(void) /* Constant nan value, generated in the same way as float('nan'). */ /* We don't currently assume that Py_NAN is defined everywhere. */ -#if _PY_SHORT_FLOAT_REPR == 1 || defined(Py_NAN) +#if _PY_SHORT_FLOAT_REPR == 1 static double m_nan(void) @@ -3838,7 +3838,7 @@ math_exec(PyObject *module) if (PyModule_AddObject(module, "inf", PyFloat_FromDouble(m_inf())) < 0) { return -1; } -#if _PY_SHORT_FLOAT_REPR == 1 || defined(Py_NAN) +#if _PY_SHORT_FLOAT_REPR == 1 if (PyModule_AddObject(module, "nan", PyFloat_FromDouble(m_nan())) < 0) { return -1; } diff --git a/Objects/floatobject.c b/Objects/floatobject.c index 64d4c3e5a8cef..6d796566c4eb6 100644 --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -2486,15 +2486,7 @@ _PyFloat_Unpack2(const unsigned char *p, int le) } else { /* NaN */ -#ifdef Py_NAN return sign ? -Py_NAN : Py_NAN; -#else - PyErr_SetString( - PyExc_ValueError, - "can't unpack IEEE 754 NaN " - "on platform that does not support NaNs"); - return -1; -#endif // !defined(Py_NAN) } #else // _PY_SHORT_FLOAT_REPR == 1 if (f == 0) { diff --git a/Python/pystrtod.c b/Python/pystrtod.c index 7469d6259d673..1b27f0a3ad36a 100644 --- a/Python/pystrtod.c +++ b/Python/pystrtod.c @@ -82,12 +82,10 @@ _Py_parse_inf_or_nan(const char *p, char **endptr) s += 5; retval = negate ? -Py_HUGE_VAL : Py_HUGE_VAL; } -#ifdef Py_NAN else if (case_insensitive_match(s, "nan")) { s += 3; retval = negate ? -Py_NAN : Py_NAN; } -#endif else { s = p; retval = -1.0; From webhook-mailer at python.org Thu Feb 24 21:05:45 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 02:05:45 -0000 Subject: [Python-checkins] bpo-46852: Rename float.__set_format__() to float.__setformat__() (GH-31558) Message-ID: https://github.com/python/cpython/commit/7d03c8be5af2f1559dbc35b775b3116dfd63cfb6 commit: 7d03c8be5af2f1559dbc35b775b3116dfd63cfb6 branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T03:05:25+01:00 summary: bpo-46852: Rename float.__set_format__() to float.__setformat__() (GH-31558) Rename the private undocumented float.__set_format__() method to float.__setformat__() to fix a typo introduced in Python 3.7. The method is only used by test_float. The change enables again test_float tests on the float format which were previously skipped because of the typo. The typo was introduced in Python 3.7 by bpo-20185 in commit b5c51d3dd95bbfde533655fb86ac0f96f771ba7b. files: A Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst M Objects/clinic/floatobject.c.h M Objects/floatobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst new file mode 100644 index 0000000000000..65b826473b915 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst @@ -0,0 +1,3 @@ +Rename the private undocumented ``float.__set_format__()`` method to +``float.__setformat__()`` to fix a typo introduced in Python 3.7. The method +is only used by test_float. Patch by Victor Stinner. diff --git a/Objects/clinic/floatobject.c.h b/Objects/clinic/floatobject.c.h index 88dc482e32e52..2c638a28d7e1e 100644 --- a/Objects/clinic/floatobject.c.h +++ b/Objects/clinic/floatobject.c.h @@ -290,8 +290,8 @@ float___getformat__(PyTypeObject *type, PyObject *arg) return return_value; } -PyDoc_STRVAR(float___set_format____doc__, -"__set_format__($type, typestr, fmt, /)\n" +PyDoc_STRVAR(float___setformat____doc__, +"__setformat__($type, typestr, fmt, /)\n" "--\n" "\n" "You probably don\'t want to use this function.\n" @@ -308,25 +308,25 @@ PyDoc_STRVAR(float___set_format____doc__, "Override the automatic determination of C-level floating point type.\n" "This affects how floats are converted to and from binary strings."); -#define FLOAT___SET_FORMAT___METHODDEF \ - {"__set_format__", (PyCFunction)(void(*)(void))float___set_format__, METH_FASTCALL|METH_CLASS, float___set_format____doc__}, +#define FLOAT___SETFORMAT___METHODDEF \ + {"__setformat__", (PyCFunction)(void(*)(void))float___setformat__, METH_FASTCALL|METH_CLASS, float___setformat____doc__}, static PyObject * -float___set_format___impl(PyTypeObject *type, const char *typestr, - const char *fmt); +float___setformat___impl(PyTypeObject *type, const char *typestr, + const char *fmt); static PyObject * -float___set_format__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs) +float___setformat__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; const char *typestr; const char *fmt; - if (!_PyArg_CheckPositional("__set_format__", nargs, 2, 2)) { + if (!_PyArg_CheckPositional("__setformat__", nargs, 2, 2)) { goto exit; } if (!PyUnicode_Check(args[0])) { - _PyArg_BadArgument("__set_format__", "argument 1", "str", args[0]); + _PyArg_BadArgument("__setformat__", "argument 1", "str", args[0]); goto exit; } Py_ssize_t typestr_length; @@ -339,7 +339,7 @@ float___set_format__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs goto exit; } if (!PyUnicode_Check(args[1])) { - _PyArg_BadArgument("__set_format__", "argument 2", "str", args[1]); + _PyArg_BadArgument("__setformat__", "argument 2", "str", args[1]); goto exit; } Py_ssize_t fmt_length; @@ -351,7 +351,7 @@ float___set_format__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs PyErr_SetString(PyExc_ValueError, "embedded null character"); goto exit; } - return_value = float___set_format___impl(type, typestr, fmt); + return_value = float___setformat___impl(type, typestr, fmt); exit: return return_value; @@ -388,4 +388,4 @@ float___format__(PyObject *self, PyObject *arg) exit: return return_value; } -/*[clinic end generated code: output=122a73f4c9d25806 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=3fb0c0624cc87dff input=a9049054013a1b77]*/ diff --git a/Objects/floatobject.c b/Objects/floatobject.c index 6d796566c4eb6..342e768c62f4a 100644 --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -1762,7 +1762,7 @@ float___getformat___impl(PyTypeObject *type, const char *typestr) /*[clinic input] @classmethod -float.__set_format__ +float.__setformat__ typestr: str Must be 'double' or 'float'. @@ -1781,9 +1781,9 @@ This affects how floats are converted to and from binary strings. [clinic start generated code]*/ static PyObject * -float___set_format___impl(PyTypeObject *type, const char *typestr, - const char *fmt) -/*[clinic end generated code: output=504460f5dc85acbd input=5306fa2b81a997e4]*/ +float___setformat___impl(PyTypeObject *type, const char *typestr, + const char *fmt) +/*[clinic end generated code: output=06864de1fb5f1f04 input=c0e9e04dd87f9988]*/ { float_format_type f; float_format_type detected; @@ -1885,7 +1885,7 @@ static PyMethodDef float_methods[] = { FLOAT_IS_INTEGER_METHODDEF FLOAT___GETNEWARGS___METHODDEF FLOAT___GETFORMAT___METHODDEF - FLOAT___SET_FORMAT___METHODDEF + FLOAT___SETFORMAT___METHODDEF FLOAT___FORMAT___METHODDEF {NULL, NULL} /* sentinel */ }; From webhook-mailer at python.org Fri Feb 25 00:13:48 2022 From: webhook-mailer at python.org (methane) Date: Fri, 25 Feb 2022 05:13:48 -0000 Subject: [Python-checkins] bpo-46606: Remove redundant +1. (GH-31561) Message-ID: https://github.com/python/cpython/commit/ad6c7003e38a9f8bdf8d865fb5fa0f3c03690315 commit: ad6c7003e38a9f8bdf8d865fb5fa0f3c03690315 branch: main author: Inada Naoki committer: methane date: 2022-02-25T14:13:14+09:00 summary: bpo-46606: Remove redundant +1. (GH-31561) files: M Modules/posixmodule.c diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 5e942c43dd6c7..d3cfc828184e5 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -8186,7 +8186,7 @@ os_setgroups(PyObject *module, PyObject *groups) return NULL; } - gid_t *grouplist = PyMem_New(gid_t, len+1); // Avoid malloc(0) + gid_t *grouplist = PyMem_New(gid_t, len); for (Py_ssize_t i = 0; i < len; i++) { PyObject *elem; elem = PySequence_GetItem(groups, i); From webhook-mailer at python.org Fri Feb 25 03:25:27 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 25 Feb 2022 08:25:27 -0000 Subject: [Python-checkins] bpo-46820: Refactor tests for ambiguous end of numerical literal (GH-31494) Message-ID: https://github.com/python/cpython/commit/98c3bea4d1c7335135e60946d0ec8cd5031fb6c0 commit: 98c3bea4d1c7335135e60946d0ec8cd5031fb6c0 branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2022-02-25T10:25:12+02:00 summary: bpo-46820: Refactor tests for ambiguous end of numerical literal (GH-31494) files: M Lib/test/test_grammar.py diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py index d35530e4803fa..2e29cdb39301d 100644 --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -214,99 +214,35 @@ def test_bad_numerical_literals(self): check("1e+", "invalid decimal literal") def test_end_of_numerical_literals(self): - def check(test): - with self.assertWarns(DeprecationWarning): - compile(test, "", "eval") - - def check_error(test): - with warnings.catch_warnings(record=True) as w: - with self.assertRaises(SyntaxError): - compile(test, "", "eval") - self.assertEqual(w, []) - - check_error("0xfand x") - check("0o7and x") - check("0b1and x") - check("9and x") - check("0and x") - check("1.and x") - check("1e3and x") - check("1jand x") - - check("0xfor x") - check("0o7or x") - check("0b1or x") - check("9or x") - check_error("0or x") - check("1.or x") - check("1e3or x") - check("1jor x") - - check("0xfin x") - check("0o7in x") - check("0b1in x") - check("9in x") - check("0in x") - check("1.in x") - check("1e3in x") - check("1jin x") - - check("0xfnot in x") - check("0o7not in x") - check("0b1not in x") - check("9not in x") - check("0not in x") - check("1.not in x") - check("1e3not in x") - check("1jnot in x") - - with warnings.catch_warnings(): - warnings.simplefilter('ignore', SyntaxWarning) - check("0xfis x") - check("0o7is x") - check("0b1is x") - check("9is x") - check("0is x") - check("1.is x") - check("1e3is x") - check("1jis x") - - check("0xfif x else y") - check("0o7if x else y") - check("0b1if x else y") - check("9if x else y") - check("0if x else y") - check("1.if x else y") - check("1e3if x else y") - check("1jif x else y") - - check_error("x if 0xfelse y") - check("x if 0o7else y") - check("x if 0b1else y") - check("x if 9else y") - check("x if 0else y") - check("x if 1.else y") - check("x if 1e3else y") - check("x if 1jelse y") + def check(test, error=False): + with self.subTest(expr=test): + if error: + with warnings.catch_warnings(record=True) as w: + with self.assertRaises(SyntaxError): + compile(test, "", "eval") + self.assertEqual(w, []) + else: + with self.assertWarns(DeprecationWarning): + compile(test, "", "eval") + + for num in "0xf", "0o7", "0b1", "9", "0", "1.", "1e3", "1j": + compile(num, "", "eval") + check(f"{num}and x", error=(num == "0xf")) + check(f"{num}or x", error=(num == "0")) + check(f"{num}in x") + check(f"{num}not in x") + with warnings.catch_warnings(): + warnings.filterwarnings('ignore', '"is" with a literal', + SyntaxWarning) + check(f"{num}is x") + check(f"{num}if x else y") + check(f"x if {num}else y", error=(num == "0xf")) + check(f"[{num}for x in ()]") + check(f"{num}spam", error=True) check("[0x1ffor x in ()]") check("[0x1for x in ()]") check("[0xfor x in ()]") - check("[0o7for x in ()]") - check("[0b1for x in ()]") - check("[9for x in ()]") - check("[1.for x in ()]") - check("[1e3for x in ()]") - check("[1jfor x in ()]") - - check_error("0xfspam") - check_error("0o7spam") - check_error("0b1spam") - check_error("9spam") - check_error("0spam") - check_error("1.spam") - check_error("1e3spam") - check_error("1jspam") def test_string_literals(self): x = ''; y = ""; self.assertTrue(len(x) == 0 and x == y) From webhook-mailer at python.org Fri Feb 25 03:26:08 2022 From: webhook-mailer at python.org (encukou) Date: Fri, 25 Feb 2022 08:26:08 -0000 Subject: [Python-checkins] bpo-46748: Don't import in public headers (GH-31553) Message-ID: https://github.com/python/cpython/commit/2c228a7b8f89e9ed8d390370abd771d4993b79d8 commit: 2c228a7b8f89e9ed8d390370abd771d4993b79d8 branch: main author: Petr Viktorin committer: encukou date: 2022-02-25T09:25:54+01:00 summary: bpo-46748: Don't import in public headers (GH-31553) is the standard/modern way to define embedd/extends Python free to define bool, true and false, but there are existing applications that use slightly different redefinitions, which fail if the header is included. It's OK to use stdbool outside the public headers, though. https://bugs.python.org/issue46748 files: A Misc/NEWS.d/next/C API/2022-02-24-13-13-16.bpo-46748.aG1zb3.rst M Include/cpython/import.h M Include/cpython/pystate.h M Modules/_testcapimodule.c M Python/frozen.c M Tools/freeze/makefreeze.py diff --git a/Include/cpython/import.h b/Include/cpython/import.h index c734802ff254b..ef6be689468ee 100644 --- a/Include/cpython/import.h +++ b/Include/cpython/import.h @@ -32,7 +32,7 @@ struct _frozen { const char *name; /* ASCII encoded string */ const unsigned char *code; int size; - bool is_package; + int is_package; PyObject *(*get_code)(void); }; diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index 8150d501d418e..248320cceb716 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -2,8 +2,6 @@ # error "this header file must not be included directly" #endif -#include - PyAPI_FUNC(int) _PyInterpreterState_RequiresIDRef(PyInterpreterState *); PyAPI_FUNC(void) _PyInterpreterState_RequireIDRef(PyInterpreterState *, int); @@ -93,7 +91,7 @@ struct _ts { int _initialized; /* Was this thread state statically allocated? */ - bool _static; + int _static; int recursion_remaining; int recursion_limit; diff --git a/Misc/NEWS.d/next/C API/2022-02-24-13-13-16.bpo-46748.aG1zb3.rst b/Misc/NEWS.d/next/C API/2022-02-24-13-13-16.bpo-46748.aG1zb3.rst new file mode 100644 index 0000000000000..b6b2db1e8ba33 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-02-24-13-13-16.bpo-46748.aG1zb3.rst @@ -0,0 +1,2 @@ +Python's public headers no longer import ````, leaving code that +embedd/extends Python free to define ``bool``, ``true`` and ``false``. diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 0be42f33e23c9..6fa0cced4ecfb 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -40,6 +40,9 @@ # error "_testcapi must test the public Python C API, not CPython internal C API" #endif +#ifdef bool +# error "The public headers should not include , see bpo-46748" +#endif // Forward declarations static struct PyModuleDef _testcapimodule; diff --git a/Python/frozen.c b/Python/frozen.c index c5b36f73b4a47..8a2a7243537cc 100644 --- a/Python/frozen.c +++ b/Python/frozen.c @@ -38,6 +38,8 @@ #include "Python.h" #include "pycore_import.h" +#include + /* Includes for frozen modules: */ /* End includes */ diff --git a/Tools/freeze/makefreeze.py b/Tools/freeze/makefreeze.py index bc5f8567448bf..c464f4bbb2655 100644 --- a/Tools/freeze/makefreeze.py +++ b/Tools/freeze/makefreeze.py @@ -45,9 +45,9 @@ def makefreeze(base, dict, debug=0, entry_point=None, fail_import=()): print("freezing", mod, "...") str = marshal.dumps(m.__code__) size = len(str) - is_package = 'false' + is_package = '0' if m.__path__: - is_package = 'true' + is_package = '1' done.append((mod, mangled, size, is_package)) writecode(outfp, mangled, str) if debug: From webhook-mailer at python.org Fri Feb 25 03:50:19 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 25 Feb 2022 08:50:19 -0000 Subject: [Python-checkins] bpo-46820: Refactor tests for ambiguous end of numerical literal (GH-31494) Message-ID: https://github.com/python/cpython/commit/c7a0fd2d3fa178c83c269f2b39ebf64a0c5babcd commit: c7a0fd2d3fa178c83c269f2b39ebf64a0c5babcd branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-25T00:49:55-08:00 summary: bpo-46820: Refactor tests for ambiguous end of numerical literal (GH-31494) (cherry picked from commit 98c3bea4d1c7335135e60946d0ec8cd5031fb6c0) Co-authored-by: Serhiy Storchaka files: M Lib/test/test_grammar.py diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py index 9d473770f3b20..2b9f7df0bd25d 100644 --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -214,99 +214,35 @@ def test_bad_numerical_literals(self): check("1e+", "invalid decimal literal") def test_end_of_numerical_literals(self): - def check(test): - with self.assertWarns(DeprecationWarning): - compile(test, "", "eval") - - def check_error(test): - with warnings.catch_warnings(record=True) as w: - with self.assertRaises(SyntaxError): - compile(test, "", "eval") - self.assertEqual(w, []) - - check_error("0xfand x") - check("0o7and x") - check("0b1and x") - check("9and x") - check("0and x") - check("1.and x") - check("1e3and x") - check("1jand x") - - check("0xfor x") - check("0o7or x") - check("0b1or x") - check("9or x") - check_error("0or x") - check("1.or x") - check("1e3or x") - check("1jor x") - - check("0xfin x") - check("0o7in x") - check("0b1in x") - check("9in x") - check("0in x") - check("1.in x") - check("1e3in x") - check("1jin x") - - check("0xfnot in x") - check("0o7not in x") - check("0b1not in x") - check("9not in x") - check("0not in x") - check("1.not in x") - check("1e3not in x") - check("1jnot in x") - - with warnings.catch_warnings(): - warnings.simplefilter('ignore', SyntaxWarning) - check("0xfis x") - check("0o7is x") - check("0b1is x") - check("9is x") - check("0is x") - check("1.is x") - check("1e3is x") - check("1jis x") - - check("0xfif x else y") - check("0o7if x else y") - check("0b1if x else y") - check("9if x else y") - check("0if x else y") - check("1.if x else y") - check("1e3if x else y") - check("1jif x else y") - - check_error("x if 0xfelse y") - check("x if 0o7else y") - check("x if 0b1else y") - check("x if 9else y") - check("x if 0else y") - check("x if 1.else y") - check("x if 1e3else y") - check("x if 1jelse y") + def check(test, error=False): + with self.subTest(expr=test): + if error: + with warnings.catch_warnings(record=True) as w: + with self.assertRaises(SyntaxError): + compile(test, "", "eval") + self.assertEqual(w, []) + else: + with self.assertWarns(DeprecationWarning): + compile(test, "", "eval") + + for num in "0xf", "0o7", "0b1", "9", "0", "1.", "1e3", "1j": + compile(num, "", "eval") + check(f"{num}and x", error=(num == "0xf")) + check(f"{num}or x", error=(num == "0")) + check(f"{num}in x") + check(f"{num}not in x") + with warnings.catch_warnings(): + warnings.filterwarnings('ignore', '"is" with a literal', + SyntaxWarning) + check(f"{num}is x") + check(f"{num}if x else y") + check(f"x if {num}else y", error=(num == "0xf")) + check(f"[{num}for x in ()]") + check(f"{num}spam", error=True) check("[0x1ffor x in ()]") check("[0x1for x in ()]") check("[0xfor x in ()]") - check("[0o7for x in ()]") - check("[0b1for x in ()]") - check("[9for x in ()]") - check("[1.for x in ()]") - check("[1e3for x in ()]") - check("[1jfor x in ()]") - - check_error("0xfspam") - check_error("0o7spam") - check_error("0b1spam") - check_error("9spam") - check_error("0spam") - check_error("1.spam") - check_error("1e3spam") - check_error("1jspam") def test_string_literals(self): x = ''; y = ""; self.assertTrue(len(x) == 0 and x == y) From webhook-mailer at python.org Fri Feb 25 06:08:29 2022 From: webhook-mailer at python.org (iritkatriel) Date: Fri, 25 Feb 2022 11:08:29 -0000 Subject: [Python-checkins] bpo-34429: Noted TemporaryFile behavior on non-Posix/non-Cygwin systems (GH-31547) Message-ID: https://github.com/python/cpython/commit/53ecf9e08d35801807daf74492c090a325f995b7 commit: 53ecf9e08d35801807daf74492c090a325f995b7 branch: main author: slateny <46876382+slateny at users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel at users.noreply.github.com> date: 2022-02-25T11:07:52Z summary: bpo-34429: Noted TemporaryFile behavior on non-Posix/non-Cygwin systems (GH-31547) files: M Doc/library/tempfile.rst diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index 3e904d04642d8..3184e50488427 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -62,6 +62,9 @@ The module defines the following user-callable items: The :py:data:`os.O_TMPFILE` flag is used if it is available and works (Linux-specific, requires Linux kernel 3.11 or later). + On platforms that are neither Posix nor Cygwin, TemporaryFile is an alias + for NamedTemporaryFile. + .. audit-event:: tempfile.mkstemp fullpath tempfile.TemporaryFile .. versionchanged:: 3.5 From webhook-mailer at python.org Fri Feb 25 06:30:36 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 25 Feb 2022 11:30:36 -0000 Subject: [Python-checkins] bpo-34429: Noted TemporaryFile behavior on non-Posix/non-Cygwin systems (GH-31547) Message-ID: https://github.com/python/cpython/commit/632a8121d4d577541c3fddffc986bcb8d8d545b6 commit: 632a8121d4d577541c3fddffc986bcb8d8d545b6 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-25T03:30:31-08:00 summary: bpo-34429: Noted TemporaryFile behavior on non-Posix/non-Cygwin systems (GH-31547) (cherry picked from commit 53ecf9e08d35801807daf74492c090a325f995b7) Co-authored-by: slateny <46876382+slateny at users.noreply.github.com> files: M Doc/library/tempfile.rst diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index b8a5fa0027ffb..6b23a1bcd630b 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -62,6 +62,9 @@ The module defines the following user-callable items: The :py:data:`os.O_TMPFILE` flag is used if it is available and works (Linux-specific, requires Linux kernel 3.11 or later). + On platforms that are neither Posix nor Cygwin, TemporaryFile is an alias + for NamedTemporaryFile. + .. audit-event:: tempfile.mkstemp fullpath tempfile.TemporaryFile .. versionchanged:: 3.5 From webhook-mailer at python.org Fri Feb 25 06:31:10 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Fri, 25 Feb 2022 11:31:10 -0000 Subject: [Python-checkins] bpo-46756: Fix authorization check in urllib.request (GH-31353) Message-ID: https://github.com/python/cpython/commit/e2e72567a1c94c548868f6ee5329363e6036057a commit: e2e72567a1c94c548868f6ee5329363e6036057a branch: main author: Serhiy Storchaka committer: serhiy-storchaka date: 2022-02-25T13:31:03+02:00 summary: bpo-46756: Fix authorization check in urllib.request (GH-31353) Fix a bug in urllib.request.HTTPPasswordMgr.find_user_password() and urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated() which allowed to bypass authorization. For example, access to URI "example.org/foobar" was allowed if the user was authorized for URI "example.org/foo". files: A Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst M Lib/test/test_urllib2.py M Lib/urllib/request.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 2c93fcada4387..a4772a5b1d944 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -164,7 +164,6 @@ def test_password_manager(self): self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam"), ('joe', 'password')) - self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam/spam"), ('joe', 'password')) @@ -173,12 +172,29 @@ def test_password_manager(self): add("c", "http://example.com/foo", "foo", "ni") add("c", "http://example.com/bar", "bar", "nini") + add("c", "http://example.com/foo/bar", "foobar", "nibar") self.assertEqual(find_user_pass("c", "http://example.com/foo"), ('foo', 'ni')) - self.assertEqual(find_user_pass("c", "http://example.com/bar"), ('bar', 'nini')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/bar"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/baz"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foobar"), + (None, None)) + + add("c", "http://example.com/baz/", "baz", "ninini") + + self.assertEqual(find_user_pass("c", "http://example.com/baz"), + (None, None)) + self.assertEqual(find_user_pass("c", "http://example.com/baz/"), + ('baz', 'ninini')) + self.assertEqual(find_user_pass("c", "http://example.com/baz/bar"), + ('baz', 'ninini')) # For the same path, newer password should be considered. @@ -1658,8 +1674,9 @@ def test_basic_prior_auth_auto_send(self): auth_prior_handler.add_password( None, request_url, user, password, is_authenticated=True) - is_auth = pwd_manager.is_authenticated(request_url) - self.assertTrue(is_auth) + self.assertTrue(pwd_manager.is_authenticated(request_url)) + self.assertTrue(pwd_manager.is_authenticated(request_url + '/nested')) + self.assertFalse(pwd_manager.is_authenticated(request_url + 'plain')) opener = OpenerDirector() opener.add_handler(auth_prior_handler) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py index fd6fc36aee04b..02f96265a8900 100644 --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -889,10 +889,10 @@ def is_suburi(self, base, test): return True if base[0] != test[0]: return False - common = posixpath.commonprefix((base[1], test[1])) - if len(common) == len(base[1]): - return True - return False + prefix = base[1] + if prefix[-1:] != '/': + prefix += '/' + return test[1].startswith(prefix) class HTTPPasswordMgrWithDefaultRealm(HTTPPasswordMgr): diff --git a/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst new file mode 100644 index 0000000000000..1660640c5d3fb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst @@ -0,0 +1,5 @@ +Fix a bug in :meth:`urllib.request.HTTPPasswordMgr.find_user_password` and +:meth:`urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated` which +allowed to bypass authorization. For example, access to URI +``example.org/foobar`` was allowed if the user was authorized for URI +``example.org/foo``. From webhook-mailer at python.org Fri Feb 25 06:34:18 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 11:34:18 -0000 Subject: [Python-checkins] bpo-1635741: Fix winreg reference leaks (GH-31560) Message-ID: https://github.com/python/cpython/commit/4657bf701670215ce69b89401b2307022a3b0a7d commit: 4657bf701670215ce69b89401b2307022a3b0a7d branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T12:34:00+01:00 summary: bpo-1635741: Fix winreg reference leaks (GH-31560) Clear also the PyHKEY_Type static type at exit. files: M Objects/object.c M PC/winreg.c diff --git a/Objects/object.c b/Objects/object.c index 3044c862fb9da..77a457223764a 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -1825,6 +1825,11 @@ _PyTypes_InitState(PyInterpreterState *interp) } + +#ifdef MS_WINDOWS +extern PyTypeObject PyHKEY_Type; +#endif + static PyTypeObject* static_types[] = { // The two most important base types: must be initialized first and // deallocated last. @@ -1869,6 +1874,9 @@ static PyTypeObject* static_types[] = { &PyFunction_Type, &PyGen_Type, &PyGetSetDescr_Type, +#ifdef MS_WINDOWS + &PyHKEY_Type, +#endif &PyInstanceMethod_Type, &PyListIter_Type, &PyListRevIter_Type, diff --git a/PC/winreg.c b/PC/winreg.c index 004a89a5355f6..2d44c82000c68 100644 --- a/PC/winreg.c +++ b/PC/winreg.c @@ -2046,11 +2046,9 @@ PyMODINIT_FUNC PyInit_winreg(void) PyHKEY_Type.tp_doc = PyHKEY_doc; if (PyType_Ready(&PyHKEY_Type) < 0) return NULL; - Py_INCREF(&PyHKEY_Type); if (PyDict_SetItemString(d, "HKEYType", (PyObject *)&PyHKEY_Type) != 0) return NULL; - Py_INCREF(PyExc_OSError); if (PyDict_SetItemString(d, "error", PyExc_OSError) != 0) return NULL; @@ -2116,5 +2114,3 @@ PyMODINIT_FUNC PyInit_winreg(void) ADD_INT(REG_RESOURCE_REQUIREMENTS_LIST); return m; } - - From webhook-mailer at python.org Fri Feb 25 06:34:19 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 25 Feb 2022 11:34:19 -0000 Subject: [Python-checkins] bpo-34429: Noted TemporaryFile behavior on non-Posix/non-Cygwin systems (GH-31547) Message-ID: https://github.com/python/cpython/commit/b7f6e8e13574aaa69d132ea67d7a515bf257a0cb commit: b7f6e8e13574aaa69d132ea67d7a515bf257a0cb branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-25T03:34:15-08:00 summary: bpo-34429: Noted TemporaryFile behavior on non-Posix/non-Cygwin systems (GH-31547) (cherry picked from commit 53ecf9e08d35801807daf74492c090a325f995b7) Co-authored-by: slateny <46876382+slateny at users.noreply.github.com> files: M Doc/library/tempfile.rst diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index 915489fadd401..6bd7d003f5290 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -62,6 +62,9 @@ The module defines the following user-callable items: The :py:data:`os.O_TMPFILE` flag is used if it is available and works (Linux-specific, requires Linux kernel 3.11 or later). + On platforms that are neither Posix nor Cygwin, TemporaryFile is an alias + for NamedTemporaryFile. + .. audit-event:: tempfile.mkstemp fullpath tempfile.TemporaryFile .. versionchanged:: 3.5 From webhook-mailer at python.org Fri Feb 25 06:53:30 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 11:53:30 -0000 Subject: [Python-checkins] bpo-46836: Move PyFrameObject to pycore_frame.h (GH-31530) Message-ID: https://github.com/python/cpython/commit/18b5dd68c6b616257ae243c0b6bb965ffc885a23 commit: 18b5dd68c6b616257ae243c0b6bb965ffc885a23 branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T12:53:19+01:00 summary: bpo-46836: Move PyFrameObject to pycore_frame.h (GH-31530) Move the PyFrameObject type definition (struct _frame) to the internal C API pycore_frame.h header file. files: A Misc/NEWS.d/next/C API/2022-02-23-16-13-17.bpo-46836.ZYyPF_.rst M Doc/c-api/veryhigh.rst M Doc/whatsnew/3.11.rst M Include/cpython/frameobject.h M Include/internal/pycore_frame.h diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 5b8735de75e9d..2f5720d493d79 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -288,8 +288,16 @@ the same library that the Python runtime is using. .. c:type:: PyFrameObject - The C structure of the objects used to describe frame objects. The - fields of this type are subject to change at any time. + The C structure of the objects used to describe frame objects. + + The structure is only part of the internal C API: fields should not be + access directly. Use getter functions like :c:func:`PyFrame_GetCode` and + :c:func:`PyFrame_GetBack`. + + Debuggers and profilers can use the limited C API to access this structure. + + .. versionchanged:: 3.11 + The structure moved to the internal C API headers. .. c:function:: PyObject* PyEval_EvalFrame(PyFrameObject *f) diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 0556a44d4c7d4..3f73bdfa3020f 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -837,40 +837,40 @@ Porting to Python 3.11 which are not available in the limited C API. (Contributed by Victor Stinner in :issue:`46007`.) -* Changes of the private :c:type:`PyFrameObject` structure members. +* The :c:type:`PyFrameObject` structure member has been moved to the internal C + API headers. - While the documentation notes that the fields of ``PyFrameObject`` are - subject to change at any time, they have been stable for a long time - and were used in several popular extensions. - In Python 3.11, the frame struct was reorganized to allow performance - optimizations. Rather than reading the fields directly, extensions should - use functions: + While the documentation notes that the :c:type:`PyFrameObject` fields are + subject to change at any time, they have been stable for a long time and were + used in several popular extensions. - * ``f_code``: removed, use :c:func:`PyFrame_GetCode` instead. - Warning: the function returns a :term:`strong reference`, need to call - :c:func:`Py_DECREF`. - * ``f_back``: changed (see below), use :c:func:`PyFrame_GetBack`. - * ``f_builtins``: removed, - use ``PyObject_GetAttrString((PyObject*)frame, "f_builtins")``. - * ``f_globals``: removed, - use ``PyObject_GetAttrString((PyObject*)frame, "f_globals")``. - * ``f_locals``: removed, - use ``PyObject_GetAttrString((PyObject*)frame, "f_locals")``. - * ``f_lasti``: removed, - use ``PyObject_GetAttrString((PyObject*)frame, "f_lasti")``. - Code using ``f_lasti`` with ``PyCode_Addr2Line()`` should use + In Python 3.11, the frame struct was reorganized to allow performance + optimizations. Some fields were removed entirely, as they were details of the + old implementation. + + :c:type:`PyFrameObject` fields: + + * ``f_back``: use :c:func:`PyFrame_GetBack`. + * ``f_blockstack``: removed. + * ``f_builtins``: use ``PyObject_GetAttrString((PyObject*)frame, "f_builtins")``. + * ``f_code``: use :c:func:`PyFrame_GetCode`. + * ``f_gen``: removed. + * ``f_globals``: use ``PyObject_GetAttrString((PyObject*)frame, "f_globals")``. + * ``f_iblock``: removed. + * ``f_lasti``: use ``PyObject_GetAttrString((PyObject*)frame, "f_lasti")``. + Code using ``f_lasti`` with ``PyCode_Addr2Line()`` must use :c:func:`PyFrame_GetLineNumber` instead. - - The following fields were removed entirely, as they were details - of the old implementation: - - * ``f_valuesstack`` - * ``f_stackdepth`` - * ``f_gen`` - * ``f_iblock`` - * ``f_state`` - * ``f_blockstack`` - * ``f_localsplus`` + * ``f_lineno``: use :c:func:`PyFrame_GetLineNumber` + * ``f_locals``: use ``PyObject_GetAttrString((PyObject*)frame, "f_locals")``. + * ``f_stackdepth``: removed. + * ``f_state``: no public API (renamed to ``f_frame.f_state``). + * ``f_trace``: no public API. + * ``f_trace_lines``: use ``PyObject_GetAttrString((PyObject*)frame, "f_trace_lines")`` + (it also be modified). + * ``f_trace_opcodes``: use ``PyObject_GetAttrString((PyObject*)frame, "f_trace_opcodes")`` + (it also be modified). + * ``f_localsplus``: no public API (renamed to ``f_frame.localsplus``). + * ``f_valuestack``: removed. The Python frame object is now created lazily. A side effect is that the ``f_back`` member must not be accessed directly, since its value is now also @@ -897,9 +897,9 @@ Porting to Python 3.11 } #endif - Or use `the pythoncapi_compat project - `__ to get these APIs - on older Python versions. + Or use the `pythoncapi_compat project + `__ to get these two + functions on older Python versions. * Changes of the :c:type:`PyThreadState` structure members: diff --git a/Include/cpython/frameobject.h b/Include/cpython/frameobject.h index 3d862d9deb000..e69209686bee0 100644 --- a/Include/cpython/frameobject.h +++ b/Include/cpython/frameobject.h @@ -4,19 +4,6 @@ # error "this header file must not be included directly" #endif -struct _frame { - PyObject_HEAD - PyFrameObject *f_back; /* previous frame, or NULL */ - struct _interpreter_frame *f_frame; /* points to the frame data */ - PyObject *f_trace; /* Trace function */ - int f_lineno; /* Current line number. Only valid if non-zero */ - char f_trace_lines; /* Emit per-line trace events? */ - char f_trace_opcodes; /* Emit per-opcode trace events? */ - char f_owns_frame; /* This frame owns the frame */ - /* The frame data, if this frame object owns the frame */ - PyObject *_f_frame_data[1]; -}; - /* Standard object interface */ PyAPI_DATA(PyTypeObject) PyFrame_Type; diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h index 1ad156290a55e..09d41222b61fc 100644 --- a/Include/internal/pycore_frame.h +++ b/Include/internal/pycore_frame.h @@ -6,6 +6,18 @@ extern "C" { #include +struct _frame { + PyObject_HEAD + PyFrameObject *f_back; /* previous frame, or NULL */ + struct _interpreter_frame *f_frame; /* points to the frame data */ + PyObject *f_trace; /* Trace function */ + int f_lineno; /* Current line number. Only valid if non-zero */ + char f_trace_lines; /* Emit per-line trace events? */ + char f_trace_opcodes; /* Emit per-opcode trace events? */ + char f_owns_frame; /* This frame owns the frame */ + /* The frame data, if this frame object owns the frame */ + PyObject *_f_frame_data[1]; +}; /* runtime lifecycle */ diff --git a/Misc/NEWS.d/next/C API/2022-02-23-16-13-17.bpo-46836.ZYyPF_.rst b/Misc/NEWS.d/next/C API/2022-02-23-16-13-17.bpo-46836.ZYyPF_.rst new file mode 100644 index 0000000000000..2867bfd518c33 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-02-23-16-13-17.bpo-46836.ZYyPF_.rst @@ -0,0 +1,2 @@ +Move the :c:type:`PyFrameObject` type definition (``struct _frame``) to the +internal C API ``pycore_frame.h`` header file. Patch by Victor Stinner. From webhook-mailer at python.org Fri Feb 25 06:56:23 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 25 Feb 2022 11:56:23 -0000 Subject: [Python-checkins] bpo-46756: Fix authorization check in urllib.request (GH-31353) Message-ID: https://github.com/python/cpython/commit/4560c7e605887fda3af63f8ce157abf94954d4d2 commit: 4560c7e605887fda3af63f8ce157abf94954d4d2 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-25T03:56:16-08:00 summary: bpo-46756: Fix authorization check in urllib.request (GH-31353) Fix a bug in urllib.request.HTTPPasswordMgr.find_user_password() and urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated() which allowed to bypass authorization. For example, access to URI "example.org/foobar" was allowed if the user was authorized for URI "example.org/foo". (cherry picked from commit e2e72567a1c94c548868f6ee5329363e6036057a) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst M Lib/test/test_urllib2.py M Lib/urllib/request.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 648f65d6257d7..4ddf51073b0dd 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -162,7 +162,6 @@ def test_password_manager(self): self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam"), ('joe', 'password')) - self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam/spam"), ('joe', 'password')) @@ -171,12 +170,29 @@ def test_password_manager(self): add("c", "http://example.com/foo", "foo", "ni") add("c", "http://example.com/bar", "bar", "nini") + add("c", "http://example.com/foo/bar", "foobar", "nibar") self.assertEqual(find_user_pass("c", "http://example.com/foo"), ('foo', 'ni')) - self.assertEqual(find_user_pass("c", "http://example.com/bar"), ('bar', 'nini')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/bar"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/baz"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foobar"), + (None, None)) + + add("c", "http://example.com/baz/", "baz", "ninini") + + self.assertEqual(find_user_pass("c", "http://example.com/baz"), + (None, None)) + self.assertEqual(find_user_pass("c", "http://example.com/baz/"), + ('baz', 'ninini')) + self.assertEqual(find_user_pass("c", "http://example.com/baz/bar"), + ('baz', 'ninini')) # For the same path, newer password should be considered. @@ -1656,8 +1672,9 @@ def test_basic_prior_auth_auto_send(self): auth_prior_handler.add_password( None, request_url, user, password, is_authenticated=True) - is_auth = pwd_manager.is_authenticated(request_url) - self.assertTrue(is_auth) + self.assertTrue(pwd_manager.is_authenticated(request_url)) + self.assertTrue(pwd_manager.is_authenticated(request_url + '/nested')) + self.assertFalse(pwd_manager.is_authenticated(request_url + 'plain')) opener = OpenerDirector() opener.add_handler(auth_prior_handler) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py index bbdc2254e3cb2..4e289fc67df6c 100644 --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -887,10 +887,10 @@ def is_suburi(self, base, test): return True if base[0] != test[0]: return False - common = posixpath.commonprefix((base[1], test[1])) - if len(common) == len(base[1]): - return True - return False + prefix = base[1] + if prefix[-1:] != '/': + prefix += '/' + return test[1].startswith(prefix) class HTTPPasswordMgrWithDefaultRealm(HTTPPasswordMgr): diff --git a/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst new file mode 100644 index 0000000000000..1660640c5d3fb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst @@ -0,0 +1,5 @@ +Fix a bug in :meth:`urllib.request.HTTPPasswordMgr.find_user_password` and +:meth:`urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated` which +allowed to bypass authorization. For example, access to URI +``example.org/foobar`` was allowed if the user was authorized for URI +``example.org/foo``. From webhook-mailer at python.org Fri Feb 25 06:57:34 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 25 Feb 2022 11:57:34 -0000 Subject: [Python-checkins] bpo-46756: Fix authorization check in urllib.request (GH-31353) Message-ID: https://github.com/python/cpython/commit/2b7e04d61274af03426975fe824ed83eca35b035 commit: 2b7e04d61274af03426975fe824ed83eca35b035 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-25T03:57:30-08:00 summary: bpo-46756: Fix authorization check in urllib.request (GH-31353) Fix a bug in urllib.request.HTTPPasswordMgr.find_user_password() and urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated() which allowed to bypass authorization. For example, access to URI "example.org/foobar" was allowed if the user was authorized for URI "example.org/foo". (cherry picked from commit e2e72567a1c94c548868f6ee5329363e6036057a) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst M Lib/test/test_urllib2.py M Lib/urllib/request.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 135741e64bb91..399c94213a635 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -164,7 +164,6 @@ def test_password_manager(self): self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam"), ('joe', 'password')) - self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam/spam"), ('joe', 'password')) @@ -173,12 +172,29 @@ def test_password_manager(self): add("c", "http://example.com/foo", "foo", "ni") add("c", "http://example.com/bar", "bar", "nini") + add("c", "http://example.com/foo/bar", "foobar", "nibar") self.assertEqual(find_user_pass("c", "http://example.com/foo"), ('foo', 'ni')) - self.assertEqual(find_user_pass("c", "http://example.com/bar"), ('bar', 'nini')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/bar"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/baz"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foobar"), + (None, None)) + + add("c", "http://example.com/baz/", "baz", "ninini") + + self.assertEqual(find_user_pass("c", "http://example.com/baz"), + (None, None)) + self.assertEqual(find_user_pass("c", "http://example.com/baz/"), + ('baz', 'ninini')) + self.assertEqual(find_user_pass("c", "http://example.com/baz/bar"), + ('baz', 'ninini')) # For the same path, newer password should be considered. @@ -1658,8 +1674,9 @@ def test_basic_prior_auth_auto_send(self): auth_prior_handler.add_password( None, request_url, user, password, is_authenticated=True) - is_auth = pwd_manager.is_authenticated(request_url) - self.assertTrue(is_auth) + self.assertTrue(pwd_manager.is_authenticated(request_url)) + self.assertTrue(pwd_manager.is_authenticated(request_url + '/nested')) + self.assertFalse(pwd_manager.is_authenticated(request_url + 'plain')) opener = OpenerDirector() opener.add_handler(auth_prior_handler) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py index eca6cc350161f..34b1b0b0b7643 100644 --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -889,10 +889,10 @@ def is_suburi(self, base, test): return True if base[0] != test[0]: return False - common = posixpath.commonprefix((base[1], test[1])) - if len(common) == len(base[1]): - return True - return False + prefix = base[1] + if prefix[-1:] != '/': + prefix += '/' + return test[1].startswith(prefix) class HTTPPasswordMgrWithDefaultRealm(HTTPPasswordMgr): diff --git a/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst new file mode 100644 index 0000000000000..1660640c5d3fb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst @@ -0,0 +1,5 @@ +Fix a bug in :meth:`urllib.request.HTTPPasswordMgr.find_user_password` and +:meth:`urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated` which +allowed to bypass authorization. For example, access to URI +``example.org/foobar`` was allowed if the user was authorized for URI +``example.org/foo``. From webhook-mailer at python.org Fri Feb 25 07:11:42 2022 From: webhook-mailer at python.org (markshannon) Date: Fri, 25 Feb 2022 12:11:42 -0000 Subject: [Python-checkins] bpo-46841: Use *inline* caching for `BINARY_OP` (GH-31543) Message-ID: https://github.com/python/cpython/commit/0f41aac109c45aa468c432f798947c54d4178b3d commit: 0f41aac109c45aa468c432f798947c54d4178b3d branch: main author: Brandt Bucher committer: markshannon date: 2022-02-25T12:11:34Z summary: bpo-46841: Use *inline* caching for `BINARY_OP` (GH-31543) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-23-18-17-30.bpo-46841.fns8HB.rst D Python/wordcode_helpers.h M Doc/library/dis.rst M Include/cpython/code.h M Include/internal/pycore_code.h M Include/opcode.h M Lib/dis.py M Lib/importlib/_bootstrap_external.py M Lib/opcode.py M Lib/test/test_code.py M Lib/test/test_compile.py M Lib/test/test_dis.py M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters M Python/ceval.c M Python/compile.c M Python/opcode_targets.h M Python/specialize.c M Tools/scripts/generate_opcode_h.py diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 003d79864cbf0..3dac3911da276 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -24,6 +24,12 @@ interpreter. Use 2 bytes for each instruction. Previously the number of bytes varied by instruction. + .. versionchanged:: 3.11 + Some instructions are accompanied by one or more inline cache entries, + which take the form of :opcode:`CACHE` instructions. These instructions + are hidden by default, but can be shown by passing ``show_caches=True`` to + any :mod:`dis` utility. + Example: Given the function :func:`myfunc`:: @@ -54,7 +60,7 @@ The bytecode analysis API allows pieces of Python code to be wrapped in a :class:`Bytecode` object that provides easy access to details of the compiled code. -.. class:: Bytecode(x, *, first_line=None, current_offset=None) +.. class:: Bytecode(x, *, first_line=None, current_offset=None, show_caches=False) Analyse the bytecode corresponding to a function, generator, asynchronous @@ -74,7 +80,7 @@ code. disassembled code. Setting this means :meth:`.dis` will display a "current instruction" marker against the specified opcode. - .. classmethod:: from_traceback(tb) + .. classmethod:: from_traceback(tb, *, show_caches=False) Construct a :class:`Bytecode` instance from the given traceback, setting *current_offset* to the instruction responsible for the exception. @@ -100,6 +106,9 @@ code. .. versionchanged:: 3.7 This can now handle coroutine and asynchronous generator objects. + .. versionchanged:: 3.11 + Added the ``show_caches`` parameter. + Example:: >>> bytecode = dis.Bytecode(myfunc) @@ -153,7 +162,7 @@ operation is being performed, so the intermediate analysis object isn't useful: Added *file* parameter. -.. function:: dis(x=None, *, file=None, depth=None) +.. function:: dis(x=None, *, file=None, depth=None, show_caches=False) Disassemble the *x* object. *x* can denote either a module, a class, a method, a function, a generator, an asynchronous generator, a coroutine, @@ -183,8 +192,11 @@ operation is being performed, so the intermediate analysis object isn't useful: .. versionchanged:: 3.7 This can now handle coroutine and asynchronous generator objects. + .. versionchanged:: 3.11 + Added the ``show_caches`` parameter. + -.. function:: distb(tb=None, *, file=None) +.. function:: distb(tb=None, *, file=None, show_caches=False) Disassemble the top-of-stack function of a traceback, using the last traceback if none was passed. The instruction causing the exception is @@ -196,9 +208,12 @@ operation is being performed, so the intermediate analysis object isn't useful: .. versionchanged:: 3.4 Added *file* parameter. + .. versionchanged:: 3.11 + Added the ``show_caches`` parameter. + -.. function:: disassemble(code, lasti=-1, *, file=None) - disco(code, lasti=-1, *, file=None) +.. function:: disassemble(code, lasti=-1, *, file=None, show_caches=False) + disco(code, lasti=-1, *, file=None, show_caches=False) Disassemble a code object, indicating the last instruction if *lasti* was provided. The output is divided in the following columns: @@ -220,8 +235,11 @@ operation is being performed, so the intermediate analysis object isn't useful: .. versionchanged:: 3.4 Added *file* parameter. + .. versionchanged:: 3.11 + Added the ``show_caches`` parameter. + -.. function:: get_instructions(x, *, first_line=None) +.. function:: get_instructions(x, *, first_line=None, show_caches=False) Return an iterator over the instructions in the supplied function, method, source code string or code object. @@ -236,6 +254,9 @@ operation is being performed, so the intermediate analysis object isn't useful: .. versionadded:: 3.4 + .. versionchanged:: 3.11 + Added the ``show_caches`` parameter. + .. function:: findlinestarts(code) diff --git a/Include/cpython/code.h b/Include/cpython/code.h index b232f80367e77..39a44ce3d7e61 100644 --- a/Include/cpython/code.h +++ b/Include/cpython/code.h @@ -5,7 +5,7 @@ /* Each instruction in a code object is a fixed-width value, * currently 2 bytes: 1-byte opcode + 1-byte oparg. The EXTENDED_ARG * opcode allows for larger values but the current limit is 3 uses - * of EXTENDED_ARG (see Python/wordcode_helpers.h), for a maximum + * of EXTENDED_ARG (see Python/compile.c), for a maximum * 32-bit value. This aligns with the note in Python/compile.c * (compiler_addop_i_line) indicating that the max oparg value is * 2**32 - 1, rather than INT_MAX. diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index 88bbefc2474b4..2791495e704e1 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -64,6 +64,13 @@ typedef union { #define INSTRUCTIONS_PER_ENTRY (sizeof(SpecializedCacheEntry)/sizeof(_Py_CODEUNIT)) +typedef struct { + _Py_CODEUNIT counter; +} _PyBinaryOpCache; + +#define INLINE_CACHE_ENTRIES_BINARY_OP \ + (sizeof(_PyBinaryOpCache) / sizeof(_Py_CODEUNIT)) + /* Maximum size of code to quicken, in code units. */ #define MAX_SIZE_TO_QUICKEN 5000 @@ -276,7 +283,7 @@ int _Py_Specialize_Call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, int _Py_Specialize_Precall(PyObject *callable, _Py_CODEUNIT *instr, int nargs, PyObject *kwnames, SpecializedCacheEntry *cache, PyObject *builtins); void _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, - SpecializedCacheEntry *cache); + int oparg); void _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); void _Py_Specialize_UnpackSequence(PyObject *seq, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); diff --git a/Include/opcode.h b/Include/opcode.h index d5fa5f42b0920..0862f96e7de9e 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -9,6 +9,7 @@ extern "C" { /* Instruction opcodes for compiled code */ #define POP_TOP 1 #define PUSH_NULL 2 +#define CACHE 3 #define NOP 9 #define UNARY_POSITIVE 10 #define UNARY_NEGATIVE 11 @@ -113,76 +114,76 @@ extern "C" { #define PRECALL 166 #define CALL 171 #define KW_NAMES 172 -#define BINARY_OP_ADAPTIVE 3 -#define BINARY_OP_ADD_INT 4 -#define BINARY_OP_ADD_FLOAT 5 -#define BINARY_OP_ADD_UNICODE 6 -#define BINARY_OP_INPLACE_ADD_UNICODE 7 -#define BINARY_OP_MULTIPLY_INT 8 -#define BINARY_OP_MULTIPLY_FLOAT 13 -#define BINARY_OP_SUBTRACT_INT 14 -#define BINARY_OP_SUBTRACT_FLOAT 16 -#define COMPARE_OP_ADAPTIVE 17 -#define COMPARE_OP_FLOAT_JUMP 18 -#define COMPARE_OP_INT_JUMP 19 -#define COMPARE_OP_STR_JUMP 20 -#define BINARY_SUBSCR_ADAPTIVE 21 -#define BINARY_SUBSCR_GETITEM 22 -#define BINARY_SUBSCR_LIST_INT 23 -#define BINARY_SUBSCR_TUPLE_INT 24 -#define BINARY_SUBSCR_DICT 26 -#define STORE_SUBSCR_ADAPTIVE 27 -#define STORE_SUBSCR_LIST_INT 28 -#define STORE_SUBSCR_DICT 29 -#define CALL_ADAPTIVE 34 -#define CALL_PY_EXACT_ARGS 36 -#define CALL_PY_WITH_DEFAULTS 37 -#define JUMP_ABSOLUTE_QUICK 38 -#define LOAD_ATTR_ADAPTIVE 39 -#define LOAD_ATTR_INSTANCE_VALUE 40 -#define LOAD_ATTR_WITH_HINT 41 -#define LOAD_ATTR_SLOT 42 -#define LOAD_ATTR_MODULE 43 -#define LOAD_GLOBAL_ADAPTIVE 44 -#define LOAD_GLOBAL_MODULE 45 -#define LOAD_GLOBAL_BUILTIN 46 -#define LOAD_METHOD_ADAPTIVE 47 -#define LOAD_METHOD_CLASS 48 -#define LOAD_METHOD_MODULE 55 -#define LOAD_METHOD_NO_DICT 56 -#define LOAD_METHOD_WITH_DICT 57 -#define LOAD_METHOD_WITH_VALUES 58 -#define PRECALL_ADAPTIVE 59 -#define PRECALL_BUILTIN_CLASS 62 -#define PRECALL_NO_KW_BUILTIN_O 63 -#define PRECALL_NO_KW_BUILTIN_FAST 64 -#define PRECALL_BUILTIN_FAST_WITH_KEYWORDS 65 -#define PRECALL_NO_KW_LEN 66 -#define PRECALL_NO_KW_ISINSTANCE 67 -#define PRECALL_NO_KW_LIST_APPEND 72 -#define PRECALL_NO_KW_METHOD_DESCRIPTOR_O 76 -#define PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 77 -#define PRECALL_NO_KW_STR_1 78 -#define PRECALL_NO_KW_TUPLE_1 79 -#define PRECALL_NO_KW_TYPE_1 80 -#define PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST 81 -#define PRECALL_BOUND_METHOD 131 -#define PRECALL_PYFUNC 140 -#define RESUME_QUICK 141 -#define STORE_ATTR_ADAPTIVE 143 -#define STORE_ATTR_INSTANCE_VALUE 150 -#define STORE_ATTR_SLOT 153 -#define STORE_ATTR_WITH_HINT 154 -#define UNPACK_SEQUENCE_ADAPTIVE 158 -#define UNPACK_SEQUENCE_LIST 159 -#define UNPACK_SEQUENCE_TUPLE 161 -#define UNPACK_SEQUENCE_TWO_TUPLE 167 -#define LOAD_FAST__LOAD_FAST 168 -#define STORE_FAST__LOAD_FAST 169 -#define LOAD_FAST__LOAD_CONST 170 -#define LOAD_CONST__LOAD_FAST 173 -#define STORE_FAST__STORE_FAST 174 -#define LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE 175 +#define BINARY_OP_ADAPTIVE 4 +#define BINARY_OP_ADD_INT 5 +#define BINARY_OP_ADD_FLOAT 6 +#define BINARY_OP_ADD_UNICODE 7 +#define BINARY_OP_INPLACE_ADD_UNICODE 8 +#define BINARY_OP_MULTIPLY_INT 13 +#define BINARY_OP_MULTIPLY_FLOAT 14 +#define BINARY_OP_SUBTRACT_INT 16 +#define BINARY_OP_SUBTRACT_FLOAT 17 +#define COMPARE_OP_ADAPTIVE 18 +#define COMPARE_OP_FLOAT_JUMP 19 +#define COMPARE_OP_INT_JUMP 20 +#define COMPARE_OP_STR_JUMP 21 +#define BINARY_SUBSCR_ADAPTIVE 22 +#define BINARY_SUBSCR_GETITEM 23 +#define BINARY_SUBSCR_LIST_INT 24 +#define BINARY_SUBSCR_TUPLE_INT 26 +#define BINARY_SUBSCR_DICT 27 +#define STORE_SUBSCR_ADAPTIVE 28 +#define STORE_SUBSCR_LIST_INT 29 +#define STORE_SUBSCR_DICT 34 +#define CALL_ADAPTIVE 36 +#define CALL_PY_EXACT_ARGS 37 +#define CALL_PY_WITH_DEFAULTS 38 +#define JUMP_ABSOLUTE_QUICK 39 +#define LOAD_ATTR_ADAPTIVE 40 +#define LOAD_ATTR_INSTANCE_VALUE 41 +#define LOAD_ATTR_WITH_HINT 42 +#define LOAD_ATTR_SLOT 43 +#define LOAD_ATTR_MODULE 44 +#define LOAD_GLOBAL_ADAPTIVE 45 +#define LOAD_GLOBAL_MODULE 46 +#define LOAD_GLOBAL_BUILTIN 47 +#define LOAD_METHOD_ADAPTIVE 48 +#define LOAD_METHOD_CLASS 55 +#define LOAD_METHOD_MODULE 56 +#define LOAD_METHOD_NO_DICT 57 +#define LOAD_METHOD_WITH_DICT 58 +#define LOAD_METHOD_WITH_VALUES 59 +#define PRECALL_ADAPTIVE 62 +#define PRECALL_BUILTIN_CLASS 63 +#define PRECALL_NO_KW_BUILTIN_O 64 +#define PRECALL_NO_KW_BUILTIN_FAST 65 +#define PRECALL_BUILTIN_FAST_WITH_KEYWORDS 66 +#define PRECALL_NO_KW_LEN 67 +#define PRECALL_NO_KW_ISINSTANCE 72 +#define PRECALL_NO_KW_LIST_APPEND 76 +#define PRECALL_NO_KW_METHOD_DESCRIPTOR_O 77 +#define PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS 78 +#define PRECALL_NO_KW_STR_1 79 +#define PRECALL_NO_KW_TUPLE_1 80 +#define PRECALL_NO_KW_TYPE_1 81 +#define PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST 131 +#define PRECALL_BOUND_METHOD 140 +#define PRECALL_PYFUNC 141 +#define RESUME_QUICK 143 +#define STORE_ATTR_ADAPTIVE 150 +#define STORE_ATTR_INSTANCE_VALUE 153 +#define STORE_ATTR_SLOT 154 +#define STORE_ATTR_WITH_HINT 158 +#define UNPACK_SEQUENCE_ADAPTIVE 159 +#define UNPACK_SEQUENCE_LIST 161 +#define UNPACK_SEQUENCE_TUPLE 167 +#define UNPACK_SEQUENCE_TWO_TUPLE 168 +#define LOAD_FAST__LOAD_FAST 169 +#define STORE_FAST__LOAD_FAST 170 +#define LOAD_FAST__LOAD_CONST 173 +#define LOAD_CONST__LOAD_FAST 174 +#define STORE_FAST__STORE_FAST 175 +#define LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE 176 #define DO_TRACING 255 #ifdef NEED_OPCODE_JUMP_TABLES static uint32_t _PyOpcode_RelativeJump[8] = { @@ -239,6 +240,10 @@ static uint32_t _PyOpcode_Jump[8] = { #define NB_INPLACE_TRUE_DIVIDE 24 #define NB_INPLACE_XOR 25 +static const uint8_t _PyOpcode_InlineCacheEntries[256] = { + [BINARY_OP] = 1, +}; + #define HAS_ARG(op) ((op) >= HAVE_ARGUMENT) /* Reserve some bytecodes for internal use in the compiler. diff --git a/Lib/dis.py b/Lib/dis.py index dc3ec169abecf..2598d4a51c6a2 100644 --- a/Lib/dis.py +++ b/Lib/dis.py @@ -30,6 +30,8 @@ LOAD_CONST = opmap['LOAD_CONST'] BINARY_OP = opmap['BINARY_OP'] +CACHE = opmap["CACHE"] + def _try_compile(source, name): """Attempts to compile the given source, first as an expression and then as a statement if the first approach fails. @@ -43,7 +45,7 @@ def _try_compile(source, name): c = compile(source, name, 'exec') return c -def dis(x=None, *, file=None, depth=None): +def dis(x=None, *, file=None, depth=None, show_caches=False): """Disassemble classes, methods, functions, and other compiled objects. With no argument, disassemble the last traceback. @@ -53,7 +55,7 @@ def dis(x=None, *, file=None, depth=None): in a special attribute. """ if x is None: - distb(file=file) + distb(file=file, show_caches=show_caches) return # Extract functions from methods. if hasattr(x, '__func__'): @@ -74,21 +76,21 @@ def dis(x=None, *, file=None, depth=None): if isinstance(x1, _have_code): print("Disassembly of %s:" % name, file=file) try: - dis(x1, file=file, depth=depth) + dis(x1, file=file, depth=depth, show_caches=show_caches) except TypeError as msg: print("Sorry:", msg, file=file) print(file=file) elif hasattr(x, 'co_code'): # Code object - _disassemble_recursive(x, file=file, depth=depth) + _disassemble_recursive(x, file=file, depth=depth, show_caches=show_caches) elif isinstance(x, (bytes, bytearray)): # Raw bytecode - _disassemble_bytes(x, file=file) + _disassemble_bytes(x, file=file, show_caches=show_caches) elif isinstance(x, str): # Source code - _disassemble_str(x, file=file, depth=depth) + _disassemble_str(x, file=file, depth=depth, show_caches=show_caches) else: raise TypeError("don't know how to disassemble %s objects" % type(x).__name__) -def distb(tb=None, *, file=None): +def distb(tb=None, *, file=None, show_caches=False): """Disassemble a traceback (default: last traceback).""" if tb is None: try: @@ -96,7 +98,7 @@ def distb(tb=None, *, file=None): except AttributeError: raise RuntimeError("no last traceback to disassemble") from None while tb.tb_next: tb = tb.tb_next - disassemble(tb.tb_frame.f_code, tb.tb_lasti, file=file) + disassemble(tb.tb_frame.f_code, tb.tb_lasti, file=file, show_caches=show_caches) # The inspect module interrogates this dictionary to build its # list of CO_* constants. It is also used by pretty_flags to @@ -298,7 +300,7 @@ def _disassemble(self, lineno_width=3, mark_as_current=False, offset_width=4): return ' '.join(fields).rstrip() -def get_instructions(x, *, first_line=None): +def get_instructions(x, *, first_line=None, show_caches=False): """Iterator for the opcodes in methods, functions or code Generates a series of Instruction named tuples giving the details of @@ -318,7 +320,9 @@ def get_instructions(x, *, first_line=None): return _get_instructions_bytes(co.co_code, co._varname_from_oparg, co.co_names, co.co_consts, - linestarts, line_offset, co_positions=co.co_positions()) + linestarts, line_offset, + co_positions=co.co_positions(), + show_caches=show_caches) def _get_const_value(op, arg, co_consts): """Helper to get the value of the const in a hasconst op. @@ -389,7 +393,8 @@ def parse_exception_table(code): def _get_instructions_bytes(code, varname_from_oparg=None, names=None, co_consts=None, linestarts=None, line_offset=0, - exception_entries=(), co_positions=None): + exception_entries=(), co_positions=None, + show_caches=False): """Iterate over the instructions in a bytecode string. Generates a sequence of Instruction namedtuples giving the details of each @@ -406,6 +411,8 @@ def _get_instructions_bytes(code, varname_from_oparg=None, labels.add(target) starts_line = None for offset, op, arg in _unpack_opargs(code): + if not show_caches and op == CACHE: + continue if linestarts is not None: starts_line = linestarts.get(offset, None) if starts_line is not None: @@ -451,17 +458,18 @@ def _get_instructions_bytes(code, varname_from_oparg=None, arg, argval, argrepr, offset, starts_line, is_jump_target, positions) -def disassemble(co, lasti=-1, *, file=None): +def disassemble(co, lasti=-1, *, file=None, show_caches=False): """Disassemble a code object.""" linestarts = dict(findlinestarts(co)) exception_entries = parse_exception_table(co) _disassemble_bytes(co.co_code, lasti, co._varname_from_oparg, co.co_names, co.co_consts, linestarts, file=file, - exception_entries=exception_entries, co_positions=co.co_positions()) + exception_entries=exception_entries, + co_positions=co.co_positions(), show_caches=show_caches) -def _disassemble_recursive(co, *, file=None, depth=None): - disassemble(co, file=file) +def _disassemble_recursive(co, *, file=None, depth=None, show_caches=False): + disassemble(co, file=file, show_caches=show_caches) if depth is None or depth > 0: if depth is not None: depth = depth - 1 @@ -469,12 +477,14 @@ def _disassemble_recursive(co, *, file=None, depth=None): if hasattr(x, 'co_code'): print(file=file) print("Disassembly of %r:" % (x,), file=file) - _disassemble_recursive(x, file=file, depth=depth) + _disassemble_recursive( + x, file=file, depth=depth, show_caches=show_caches + ) def _disassemble_bytes(code, lasti=-1, varname_from_oparg=None, names=None, co_consts=None, linestarts=None, *, file=None, line_offset=0, exception_entries=(), - co_positions=None): + co_positions=None, show_caches=False): # Omit the line number column entirely if we have no line number info show_lineno = bool(linestarts) if show_lineno: @@ -492,8 +502,10 @@ def _disassemble_bytes(code, lasti=-1, varname_from_oparg=None, offset_width = 4 for instr in _get_instructions_bytes(code, varname_from_oparg, names, co_consts, linestarts, - line_offset=line_offset, exception_entries=exception_entries, - co_positions=co_positions): + line_offset=line_offset, + exception_entries=exception_entries, + co_positions=co_positions, + show_caches=show_caches): new_source_line = (show_lineno and instr.starts_line is not None and instr.offset > 0) @@ -616,7 +628,7 @@ class Bytecode: Iterating over this yields the bytecode operations as Instruction instances. """ - def __init__(self, x, *, first_line=None, current_offset=None): + def __init__(self, x, *, first_line=None, current_offset=None, show_caches=False): self.codeobj = co = _get_code_object(x) if first_line is None: self.first_line = co.co_firstlineno @@ -628,6 +640,7 @@ def __init__(self, x, *, first_line=None, current_offset=None): self._original_object = x self.current_offset = current_offset self.exception_entries = parse_exception_table(co) + self.show_caches = show_caches def __iter__(self): co = self.codeobj @@ -637,18 +650,21 @@ def __iter__(self): self._linestarts, line_offset=self._line_offset, exception_entries=self.exception_entries, - co_positions=co.co_positions()) + co_positions=co.co_positions(), + show_caches=self.show_caches) def __repr__(self): return "{}({!r})".format(self.__class__.__name__, self._original_object) @classmethod - def from_traceback(cls, tb): + def from_traceback(cls, tb, *, show_caches=False): """ Construct a Bytecode from the given traceback """ while tb.tb_next: tb = tb.tb_next - return cls(tb.tb_frame.f_code, current_offset=tb.tb_lasti) + return cls( + tb.tb_frame.f_code, current_offset=tb.tb_lasti, show_caches=show_caches + ) def info(self): """Return formatted information about the code object.""" @@ -670,7 +686,8 @@ def dis(self): file=output, lasti=offset, exception_entries=self.exception_entries, - co_positions=co.co_positions()) + co_positions=co.co_positions(), + show_caches=self.show_caches) return output.getvalue() diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py index 5255d7e19ce8e..53fe1b8fc217a 100644 --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -387,9 +387,7 @@ def _write_atomic(path, data, mode=0o666): # Python 3.11a5 3478 (New CALL opcodes) # Python 3.11a5 3479 (Add PUSH_NULL opcode) # Python 3.11a5 3480 (New CALL opcodes, second iteration) - -# Python 3.12 will start with magic number 3500 - +# Python 3.11a5 3481 (Use inline CACHE instructions) # Python 3.12 will start with magic number 3500 @@ -404,7 +402,7 @@ def _write_atomic(path, data, mode=0o666): # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # in PC/launcher.c must also be updated. -MAGIC_NUMBER = (3480).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3481).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Lib/opcode.py b/Lib/opcode.py index bb451b9f1a6cd..84ad002c8dde8 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -35,20 +35,23 @@ opmap = {} opname = ['<%r>' % (op,) for op in range(256)] -def def_op(name, op): +_inline_cache_entries = [0] * 256 + +def def_op(name, op, entries=0): opname[op] = name opmap[name] = op + _inline_cache_entries[op] = entries -def name_op(name, op): - def_op(name, op) +def name_op(name, op, entries=0): + def_op(name, op, entries) hasname.append(op) -def jrel_op(name, op): - def_op(name, op) +def jrel_op(name, op, entries=0): + def_op(name, op, entries) hasjrel.append(op) -def jabs_op(name, op): - def_op(name, op) +def jabs_op(name, op, entries=0): + def_op(name, op, entries) hasjabs.append(op) # Instruction opcodes for compiled code @@ -56,6 +59,7 @@ def jabs_op(name, op): def_op('POP_TOP', 1) def_op('PUSH_NULL', 2) +def_op('CACHE', 3) def_op('NOP', 9) def_op('UNARY_POSITIVE', 10) @@ -137,7 +141,7 @@ def jabs_op(name, op): def_op('RERAISE', 119) def_op('COPY', 120) jabs_op('JUMP_IF_NOT_EXC_MATCH', 121) -def_op('BINARY_OP', 122) +def_op('BINARY_OP', 122, 1) jrel_op('SEND', 123) # Number of bytes to skip def_op('LOAD_FAST', 124) # Local variable number haslocal.append(124) diff --git a/Lib/test/test_code.py b/Lib/test/test_code.py index 9319f200e34fb..872f7283fc504 100644 --- a/Lib/test/test_code.py +++ b/Lib/test/test_code.py @@ -357,7 +357,7 @@ def test_co_positions_artificial_instructions(self): artificial_instructions = [] for instr, positions in zip( - dis.get_instructions(code), + dis.get_instructions(code, show_caches=True), code.co_positions(), strict=True ): diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py index 2e1cb5a67197c..5268192b7cd97 100644 --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -1062,7 +1062,9 @@ def generic_visit(self, node): def assertOpcodeSourcePositionIs(self, code, opcode, line, end_line, column, end_column, occurrence=1): - for instr, position in zip(dis.Bytecode(code), code.co_positions()): + for instr, position in zip( + dis.Bytecode(code, show_caches=True), code.co_positions(), strict=True + ): if instr.opname == opcode: occurrence -= 1 if not occurrence: diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index e81c834878f08..e478ca966612b 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -375,7 +375,7 @@ def bug42562(): >> PUSH_EXC_INFO %3d LOAD_GLOBAL 0 (Exception) - JUMP_IF_NOT_EXC_MATCH 25 (to 50) + JUMP_IF_NOT_EXC_MATCH 26 (to 52) STORE_FAST 0 (e) %3d LOAD_FAST 0 (e) @@ -571,7 +571,7 @@ def foo(x): %3d RESUME 0 BUILD_LIST 0 LOAD_FAST 0 (.0) - >> FOR_ITER 6 (to 22) + >> FOR_ITER 7 (to 24) STORE_FAST 1 (z) LOAD_DEREF 2 (x) LOAD_FAST 1 (z) @@ -622,8 +622,8 @@ def assert_offsets_increasing(self, text, delta): if line.startswith("Exception"): break offset = int(line[start:end]) - self.assertEqual(offset, expected_offset, line) - expected_offset += delta + self.assertGreaterEqual(offset, expected_offset, line) + expected_offset = offset + delta def strip_offsets(self, text): lines = text.splitlines(True) @@ -762,20 +762,20 @@ def expected(count, w): %*d LOAD_CONST 1 (1) %*d BINARY_OP 0 (+) %*d STORE_FAST 0 (x) -''' % (w, 8*i + 2, w, 8*i + 4, w, 8*i + 6, w, 8*i + 8) +''' % (w, 10*i + 2, w, 10*i + 4, w, 10*i + 6, w, 10*i + 10) for i in range(count)] s += ['''\ 3 %*d LOAD_FAST 0 (x) %*d RETURN_VALUE -''' % (w, 8*count + 2, w, 8*count + 4)] +''' % (w, 10*count + 2, w, 10*count + 4)] s[1] = ' 2' + s[1][3:] return ''.join(s) for i in range(1, 5): self.do_disassembly_test(func(i), expected(i, 4), True) - self.do_disassembly_test(func(1248), expected(1248, 4), True) - self.do_disassembly_test(func(1250), expected(1250, 5), True) + self.do_disassembly_test(func(999), expected(999, 4), True) + self.do_disassembly_test(func(1000), expected(1000, 5), True) def test_disassemble_str(self): self.do_disassembly_test(expr_str, dis_expr_str) @@ -1250,7 +1250,7 @@ def _prepare_test_cases(): Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=62, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=64, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=66, starts_line=11, is_jump_target=True, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=57, argval=114, argrepr='to 114', offset=68, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=58, argval=116, argrepr='to 116', offset=68, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=70, starts_line=12, is_jump_target=True, positions=None), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=72, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=74, starts_line=None, is_jump_target=False, positions=None), @@ -1260,102 +1260,102 @@ def _prepare_test_cases(): Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=82, starts_line=13, is_jump_target=False, positions=None), Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=84, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='BINARY_OP', opcode=122, arg=23, argval=23, argrepr='-=', offset=86, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=88, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=90, starts_line=14, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=92, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=94, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=50, argval=100, argrepr='to 100', offset=96, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=33, argval=66, argrepr='to 66', offset=98, starts_line=15, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=100, starts_line=16, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=102, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=104, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=55, argval=110, argrepr='to 110', offset=106, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=8, argval=126, argrepr='to 126', offset=108, starts_line=17, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=110, starts_line=11, is_jump_target=True, positions=None), - Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=35, argval=70, argrepr='to 70', offset=112, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=114, starts_line=19, is_jump_target=True, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=116, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=118, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=120, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=122, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=124, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=126, starts_line=20, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=128, starts_line=21, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=130, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='BINARY_OP', opcode=122, arg=11, argval=11, argrepr='/', offset=132, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=134, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=136, starts_line=25, is_jump_target=False, positions=None), - Instruction(opname='BEFORE_WITH', opcode=53, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=140, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=142, starts_line=26, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=144, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Never reach this', argrepr="'Never reach this'", offset=146, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=148, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=150, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=152, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=154, starts_line=25, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=156, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=158, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=2, argval=2, argrepr='', offset=160, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=2, argval=2, argrepr='', offset=162, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=190, argrepr='to 190', offset=166, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=168, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=170, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=91, argval=182, argrepr='to 182', offset=172, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=2, argval=2, argrepr='', offset=174, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=176, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=178, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=180, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=True, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=184, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=16, argval=224, argrepr='to 224', offset=190, starts_line=None, is_jump_target=True, positions=None), - Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=192, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=194, starts_line=22, is_jump_target=False, positions=None), - Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=108, argval=216, argrepr='to 216', offset=196, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=198, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=200, starts_line=23, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=202, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=204, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=206, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=208, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=210, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=212, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=12, argval=240, argrepr='to 240', offset=214, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=216, starts_line=22, is_jump_target=True, positions=None), - Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=218, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=220, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=222, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=224, starts_line=28, is_jump_target=True, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=226, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=228, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=230, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=232, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=234, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=236, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=238, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=240, starts_line=23, is_jump_target=True, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=242, starts_line=28, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=244, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=246, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=248, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=250, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=252, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=254, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=256, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=258, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=260, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=262, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=264, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=266, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=268, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=270, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=272, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=274, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=276, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=278, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=90, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=92, starts_line=14, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=94, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=96, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=51, argval=102, argrepr='to 102', offset=98, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=33, argval=66, argrepr='to 66', offset=100, starts_line=15, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=102, starts_line=16, is_jump_target=True, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=104, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=106, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=56, argval=112, argrepr='to 112', offset=108, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=8, argval=128, argrepr='to 128', offset=110, starts_line=17, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=112, starts_line=11, is_jump_target=True, positions=None), + Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=35, argval=70, argrepr='to 70', offset=114, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=116, starts_line=19, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=118, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=120, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=122, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=124, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=126, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=128, starts_line=20, is_jump_target=True, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=130, starts_line=21, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=132, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='BINARY_OP', opcode=122, arg=11, argval=11, argrepr='/', offset=134, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=140, starts_line=25, is_jump_target=False, positions=None), + Instruction(opname='BEFORE_WITH', opcode=53, arg=None, argval=None, argrepr='', offset=142, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=144, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=146, starts_line=26, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=148, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Never reach this', argrepr="'Never reach this'", offset=150, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=152, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=154, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=156, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=158, starts_line=25, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=160, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=162, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=2, argval=2, argrepr='', offset=164, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=2, argval=2, argrepr='', offset=166, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=168, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=194, argrepr='to 194', offset=170, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=174, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=93, argval=186, argrepr='to 186', offset=176, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=2, argval=2, argrepr='', offset=178, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=180, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=184, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=True, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=190, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=192, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=16, argval=228, argrepr='to 228', offset=194, starts_line=None, is_jump_target=True, positions=None), + Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=196, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=198, starts_line=22, is_jump_target=False, positions=None), + Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=110, argval=220, argrepr='to 220', offset=200, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=202, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=204, starts_line=23, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=206, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=208, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=210, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=212, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=214, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=216, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=12, argval=244, argrepr='to 244', offset=218, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=220, starts_line=22, is_jump_target=True, positions=None), + Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=222, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=224, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=226, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=228, starts_line=28, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=230, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=232, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=234, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=236, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=238, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=240, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=242, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=244, starts_line=23, is_jump_target=True, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=246, starts_line=28, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=248, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=250, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=252, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=254, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=256, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=258, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=260, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=262, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=264, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=266, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=268, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=270, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=272, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=274, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=276, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=278, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=280, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=282, starts_line=None, is_jump_target=False, positions=None), ] # One last piece of inspect fodder to check the default line number handling diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-23-18-17-30.bpo-46841.fns8HB.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-23-18-17-30.bpo-46841.fns8HB.rst new file mode 100644 index 0000000000000..5eedd34d26d57 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-23-18-17-30.bpo-46841.fns8HB.rst @@ -0,0 +1,2 @@ +Store :opcode:`BINARY_OP` caches inline using a new :opcode:`CACHE` +instruction. diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 96996988ec36b..3686233a1ad50 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -327,7 +327,6 @@ - diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 92c6bf8535b98..86049a2a5df5d 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -321,9 +321,6 @@ Python - - Python - Python diff --git a/Python/ceval.c b/Python/ceval.c index a64a24f8a0a83..c09914fadcaab 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1939,6 +1939,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (prod == NULL) { goto error; } + JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP); NOTRACE_DISPATCH(); } @@ -1959,6 +1960,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (prod == NULL) { goto error; } + JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP); NOTRACE_DISPATCH(); } @@ -1977,6 +1979,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (sub == NULL) { goto error; } + JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP); NOTRACE_DISPATCH(); } @@ -1996,6 +1999,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (sub == NULL) { goto error; } + JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP); NOTRACE_DISPATCH(); } @@ -2014,6 +2018,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (TOP() == NULL) { goto error; } + JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP); NOTRACE_DISPATCH(); } @@ -2043,6 +2048,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (TOP() == NULL) { goto error; } + JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP); NOTRACE_DISPATCH(); } @@ -2063,6 +2069,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (sum == NULL) { goto error; } + JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP); NOTRACE_DISPATCH(); } @@ -2081,6 +2088,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (sum == NULL) { goto error; } + JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP); NOTRACE_DISPATCH(); } @@ -5425,23 +5433,23 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr if (res == NULL) { goto error; } + JUMPBY(INLINE_CACHE_ENTRIES_BINARY_OP); DISPATCH(); } TARGET(BINARY_OP_ADAPTIVE) { assert(cframe.use_tracing == 0); - SpecializedCacheEntry *cache = GET_CACHE(); - if (cache->adaptive.counter == 0) { + _PyBinaryOpCache *cache = (_PyBinaryOpCache *)next_instr; + if (cache->counter == 0) { PyObject *lhs = SECOND(); PyObject *rhs = TOP(); next_instr--; - _Py_Specialize_BinaryOp(lhs, rhs, next_instr, cache); + _Py_Specialize_BinaryOp(lhs, rhs, next_instr, oparg); DISPATCH(); } else { STAT_INC(BINARY_OP, deferred); - cache->adaptive.counter--; - oparg = cache->adaptive.original_oparg; + cache->counter--; JUMP_TO_INSTRUCTION(BINARY_OP); } } @@ -5462,6 +5470,10 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr DISPATCH_GOTO(); } + TARGET(CACHE) { + Py_UNREACHABLE(); + } + #if USE_COMPUTED_GOTOS TARGET_DO_TRACING: { #else @@ -5548,6 +5560,22 @@ opname ## _miss: \ JUMP_TO_INSTRUCTION(opname); \ } +#define MISS_WITH_INLINE_CACHE(opname) \ +opname ## _miss: \ + { \ + STAT_INC(opcode, miss); \ + STAT_INC(opname, miss); \ + /* The counter is always the first cache entry: */ \ + _Py_CODEUNIT *counter = (_Py_CODEUNIT *)next_instr; \ + *counter -= 1; \ + if (*counter == 0) { \ + next_instr[-1] = _Py_MAKECODEUNIT(opname ## _ADAPTIVE, _Py_OPARG(next_instr[-1])); \ + STAT_INC(opname, deopt); \ + *counter = ADAPTIVE_CACHE_BACKOFF; \ + } \ + JUMP_TO_INSTRUCTION(opname); \ + } + #define MISS_WITH_OPARG_COUNTER(opname) \ opname ## _miss: \ { \ @@ -5569,7 +5597,7 @@ MISS_WITH_CACHE(LOAD_GLOBAL) MISS_WITH_CACHE(LOAD_METHOD) MISS_WITH_CACHE(PRECALL) MISS_WITH_CACHE(CALL) -MISS_WITH_CACHE(BINARY_OP) +MISS_WITH_INLINE_CACHE(BINARY_OP) MISS_WITH_CACHE(COMPARE_OP) MISS_WITH_CACHE(BINARY_SUBSCR) MISS_WITH_CACHE(UNPACK_SEQUENCE) diff --git a/Python/compile.c b/Python/compile.c index cb6e5e3550fdf..40e6263ff7dd8 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -33,7 +33,6 @@ #define NEED_OPCODE_JUMP_TABLES #include "opcode.h" // EXTENDED_ARG -#include "wordcode_helpers.h" // instrsize() #define DEFAULT_BLOCK_SIZE 16 @@ -131,6 +130,43 @@ is_jump(struct instr *i) return i->i_opcode >= SETUP_WITH || is_bit_set_in_table(_PyOpcode_Jump, i->i_opcode); } +static int +instr_size(struct instr *instruction) +{ + int opcode = instruction->i_opcode; + int oparg = instruction->i_oparg; + int extended_args = (0xFFFFFF < oparg) + (0xFFFF < oparg) + (0xFF < oparg); + int caches = _PyOpcode_InlineCacheEntries[opcode]; + return extended_args + 1 + caches; +} + +static void +write_instr(_Py_CODEUNIT *codestr, struct instr *instruction, int ilen) +{ + int opcode = instruction->i_opcode; + int oparg = instruction->i_oparg; + int caches = _PyOpcode_InlineCacheEntries[opcode]; + switch (ilen - caches) { + case 4: + *codestr++ = _Py_MAKECODEUNIT(EXTENDED_ARG, (oparg >> 24) & 0xFF); + /* fall through */ + case 3: + *codestr++ = _Py_MAKECODEUNIT(EXTENDED_ARG, (oparg >> 16) & 0xFF); + /* fall through */ + case 2: + *codestr++ = _Py_MAKECODEUNIT(EXTENDED_ARG, (oparg >> 8) & 0xFF); + /* fall through */ + case 1: + *codestr++ = _Py_MAKECODEUNIT(opcode, oparg & 0xFF); + break; + default: + Py_UNREACHABLE(); + } + while (caches--) { + *codestr++ = _Py_MAKECODEUNIT(CACHE, 0); + } +} + typedef struct basicblock_ { /* Each basicblock in a compilation unit is linked via b_list in the reverse order that the block are allocated. b_list points to the next @@ -854,6 +890,7 @@ stack_effect(int opcode, int oparg, int jump) case NOP: case EXTENDED_ARG: case RESUME: + case CACHE: return 0; /* Stack manipulation */ @@ -7065,8 +7102,9 @@ blocksize(basicblock *b) int i; int size = 0; - for (i = 0; i < b->b_iused; i++) - size += instrsize(b->b_instr[i].i_oparg); + for (i = 0; i < b->b_iused; i++) { + size += instr_size(&b->b_instr[i]); + } return size; } @@ -7330,7 +7368,7 @@ assemble_exception_table(struct assembler *a) start = ioffset; handler = instr->i_except; } - ioffset += instrsize(instr->i_oparg); + ioffset += instr_size(instr); } } if (handler != NULL) { @@ -7459,12 +7497,10 @@ assemble_cnotab(struct assembler* a, struct instr* i, int instr_size) static int assemble_emit(struct assembler *a, struct instr *i) { - int size, arg = 0; Py_ssize_t len = PyBytes_GET_SIZE(a->a_bytecode); _Py_CODEUNIT *code; - arg = i->i_oparg; - size = instrsize(arg); + int size = instr_size(i); if (i->i_lineno && !assemble_lnotab(a, i)) { return 0; } @@ -7482,7 +7518,7 @@ assemble_emit(struct assembler *a, struct instr *i) } code = (_Py_CODEUNIT *)PyBytes_AS_STRING(a->a_bytecode) + a->a_offset; a->a_offset += size; - write_op_arg(code, i->i_opcode, arg, size); + write_instr(code, i, size); return 1; } @@ -7532,7 +7568,7 @@ assemble_jump_offsets(struct assembler *a, struct compiler *c) bsize = b->b_offset; for (i = 0; i < b->b_iused; i++) { struct instr *instr = &b->b_instr[i]; - int isize = instrsize(instr->i_oparg); + int isize = instr_size(instr); /* Relative jumps are computed relative to the instruction pointer after fetching the jump instruction. @@ -7543,7 +7579,7 @@ assemble_jump_offsets(struct assembler *a, struct compiler *c) if (is_relative_jump(instr)) { instr->i_oparg -= bsize; } - if (instrsize(instr->i_oparg) != isize) { + if (instr_size(instr) != isize) { extended_arg_recompile = 1; } } @@ -7555,7 +7591,7 @@ assemble_jump_offsets(struct assembler *a, struct compiler *c) with a better solution. The issue is that in the first loop blocksize() is called - which calls instrsize() which requires i_oparg be set + which calls instr_size() which requires i_oparg be set appropriately. There is a bootstrap problem because i_oparg is calculated in the second loop above. diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index 3339ab1a04327..d463e303e27ac 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -2,19 +2,20 @@ static void *opcode_targets[256] = { &&_unknown_opcode, &&TARGET_POP_TOP, &&TARGET_PUSH_NULL, + &&TARGET_CACHE, &&TARGET_BINARY_OP_ADAPTIVE, &&TARGET_BINARY_OP_ADD_INT, &&TARGET_BINARY_OP_ADD_FLOAT, &&TARGET_BINARY_OP_ADD_UNICODE, &&TARGET_BINARY_OP_INPLACE_ADD_UNICODE, - &&TARGET_BINARY_OP_MULTIPLY_INT, &&TARGET_NOP, &&TARGET_UNARY_POSITIVE, &&TARGET_UNARY_NEGATIVE, &&TARGET_UNARY_NOT, + &&TARGET_BINARY_OP_MULTIPLY_INT, &&TARGET_BINARY_OP_MULTIPLY_FLOAT, - &&TARGET_BINARY_OP_SUBTRACT_INT, &&TARGET_UNARY_INVERT, + &&TARGET_BINARY_OP_SUBTRACT_INT, &&TARGET_BINARY_OP_SUBTRACT_FLOAT, &&TARGET_COMPARE_OP_ADAPTIVE, &&TARGET_COMPARE_OP_FLOAT_JUMP, @@ -23,18 +24,18 @@ static void *opcode_targets[256] = { &&TARGET_BINARY_SUBSCR_ADAPTIVE, &&TARGET_BINARY_SUBSCR_GETITEM, &&TARGET_BINARY_SUBSCR_LIST_INT, - &&TARGET_BINARY_SUBSCR_TUPLE_INT, &&TARGET_BINARY_SUBSCR, + &&TARGET_BINARY_SUBSCR_TUPLE_INT, &&TARGET_BINARY_SUBSCR_DICT, &&TARGET_STORE_SUBSCR_ADAPTIVE, &&TARGET_STORE_SUBSCR_LIST_INT, - &&TARGET_STORE_SUBSCR_DICT, &&TARGET_GET_LEN, &&TARGET_MATCH_MAPPING, &&TARGET_MATCH_SEQUENCE, &&TARGET_MATCH_KEYS, - &&TARGET_CALL_ADAPTIVE, + &&TARGET_STORE_SUBSCR_DICT, &&TARGET_PUSH_EXC_INFO, + &&TARGET_CALL_ADAPTIVE, &&TARGET_CALL_PY_EXACT_ARGS, &&TARGET_CALL_PY_WITH_DEFAULTS, &&TARGET_JUMP_ABSOLUTE_QUICK, @@ -47,40 +48,39 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_GLOBAL_MODULE, &&TARGET_LOAD_GLOBAL_BUILTIN, &&TARGET_LOAD_METHOD_ADAPTIVE, - &&TARGET_LOAD_METHOD_CLASS, &&TARGET_WITH_EXCEPT_START, &&TARGET_GET_AITER, &&TARGET_GET_ANEXT, &&TARGET_BEFORE_ASYNC_WITH, &&TARGET_BEFORE_WITH, &&TARGET_END_ASYNC_FOR, + &&TARGET_LOAD_METHOD_CLASS, &&TARGET_LOAD_METHOD_MODULE, &&TARGET_LOAD_METHOD_NO_DICT, &&TARGET_LOAD_METHOD_WITH_DICT, &&TARGET_LOAD_METHOD_WITH_VALUES, - &&TARGET_PRECALL_ADAPTIVE, &&TARGET_STORE_SUBSCR, &&TARGET_DELETE_SUBSCR, + &&TARGET_PRECALL_ADAPTIVE, &&TARGET_PRECALL_BUILTIN_CLASS, &&TARGET_PRECALL_NO_KW_BUILTIN_O, &&TARGET_PRECALL_NO_KW_BUILTIN_FAST, &&TARGET_PRECALL_BUILTIN_FAST_WITH_KEYWORDS, &&TARGET_PRECALL_NO_KW_LEN, - &&TARGET_PRECALL_NO_KW_ISINSTANCE, &&TARGET_GET_ITER, &&TARGET_GET_YIELD_FROM_ITER, &&TARGET_PRINT_EXPR, &&TARGET_LOAD_BUILD_CLASS, - &&TARGET_PRECALL_NO_KW_LIST_APPEND, + &&TARGET_PRECALL_NO_KW_ISINSTANCE, &&TARGET_GET_AWAITABLE, &&TARGET_LOAD_ASSERTION_ERROR, &&TARGET_RETURN_GENERATOR, + &&TARGET_PRECALL_NO_KW_LIST_APPEND, &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_O, &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS, &&TARGET_PRECALL_NO_KW_STR_1, &&TARGET_PRECALL_NO_KW_TUPLE_1, &&TARGET_PRECALL_NO_KW_TYPE_1, - &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST, &&TARGET_LIST_TO_TUPLE, &&TARGET_RETURN_VALUE, &&TARGET_IMPORT_STAR, @@ -130,7 +130,7 @@ static void *opcode_targets[256] = { &&TARGET_POP_JUMP_IF_NOT_NONE, &&TARGET_POP_JUMP_IF_NONE, &&TARGET_RAISE_VARARGS, - &&TARGET_PRECALL_BOUND_METHOD, + &&TARGET_PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST, &&TARGET_MAKE_FUNCTION, &&TARGET_BUILD_SLICE, &&TARGET_JUMP_NO_INTERRUPT, @@ -139,39 +139,40 @@ static void *opcode_targets[256] = { &&TARGET_LOAD_DEREF, &&TARGET_STORE_DEREF, &&TARGET_DELETE_DEREF, + &&TARGET_PRECALL_BOUND_METHOD, &&TARGET_PRECALL_PYFUNC, - &&TARGET_RESUME_QUICK, &&TARGET_CALL_FUNCTION_EX, - &&TARGET_STORE_ATTR_ADAPTIVE, + &&TARGET_RESUME_QUICK, &&TARGET_EXTENDED_ARG, &&TARGET_LIST_APPEND, &&TARGET_SET_ADD, &&TARGET_MAP_ADD, &&TARGET_LOAD_CLASSDEREF, &&TARGET_COPY_FREE_VARS, - &&TARGET_STORE_ATTR_INSTANCE_VALUE, + &&TARGET_STORE_ATTR_ADAPTIVE, &&TARGET_RESUME, &&TARGET_MATCH_CLASS, + &&TARGET_STORE_ATTR_INSTANCE_VALUE, &&TARGET_STORE_ATTR_SLOT, - &&TARGET_STORE_ATTR_WITH_HINT, &&TARGET_FORMAT_VALUE, &&TARGET_BUILD_CONST_KEY_MAP, &&TARGET_BUILD_STRING, + &&TARGET_STORE_ATTR_WITH_HINT, &&TARGET_UNPACK_SEQUENCE_ADAPTIVE, - &&TARGET_UNPACK_SEQUENCE_LIST, &&TARGET_LOAD_METHOD, - &&TARGET_UNPACK_SEQUENCE_TUPLE, + &&TARGET_UNPACK_SEQUENCE_LIST, &&TARGET_LIST_EXTEND, &&TARGET_SET_UPDATE, &&TARGET_DICT_MERGE, &&TARGET_DICT_UPDATE, &&TARGET_PRECALL, + &&TARGET_UNPACK_SEQUENCE_TUPLE, &&TARGET_UNPACK_SEQUENCE_TWO_TUPLE, &&TARGET_LOAD_FAST__LOAD_FAST, &&TARGET_STORE_FAST__LOAD_FAST, - &&TARGET_LOAD_FAST__LOAD_CONST, &&TARGET_CALL, &&TARGET_KW_NAMES, + &&TARGET_LOAD_FAST__LOAD_CONST, &&TARGET_LOAD_CONST__LOAD_FAST, &&TARGET_STORE_FAST__STORE_FAST, &&TARGET_LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE, @@ -253,6 +254,5 @@ static void *opcode_targets[256] = { &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, - &&_unknown_opcode, &&TARGET_DO_TRACING }; diff --git a/Python/specialize.c b/Python/specialize.c index 6e4eb46443dd0..e1db12b853c7c 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -65,7 +65,6 @@ static uint8_t cache_requirements[256] = { [CALL] = 2, /* _PyAdaptiveEntry and _PyObjectCache/_PyCallCache */ [PRECALL] = 2, /* _PyAdaptiveEntry and _PyObjectCache/_PyCallCache */ [STORE_ATTR] = 1, // _PyAdaptiveEntry - [BINARY_OP] = 1, // _PyAdaptiveEntry [COMPARE_OP] = 1, /* _PyAdaptiveEntry */ [UNPACK_SEQUENCE] = 1, // _PyAdaptiveEntry }; @@ -385,29 +384,34 @@ optimize(SpecializedCacheOrInstruction *quickened, int len) int opcode = _Py_OPCODE(instructions[i]); int oparg = _Py_OPARG(instructions[i]); uint8_t adaptive_opcode = adaptive_opcodes[opcode]; - if (adaptive_opcode && previous_opcode != EXTENDED_ARG) { - int new_oparg = oparg_from_instruction_and_update_offset( - i, opcode, oparg, &cache_offset - ); - if (new_oparg < 0) { - /* Not possible to allocate a cache for this instruction */ - previous_opcode = opcode; - continue; + if (adaptive_opcode) { + if (_PyOpcode_InlineCacheEntries[opcode]) { + instructions[i] = _Py_MAKECODEUNIT(adaptive_opcode, oparg); } - previous_opcode = adaptive_opcode; - int entries_needed = cache_requirements[opcode]; - if (entries_needed) { - /* Initialize the adpative cache entry */ - int cache0_offset = cache_offset-entries_needed; - SpecializedCacheEntry *cache = - _GetSpecializedCacheEntry(instructions, cache0_offset); - cache->adaptive.original_oparg = oparg; - cache->adaptive.counter = 0; - } else { - // oparg is the adaptive cache counter - new_oparg = 0; + else if (previous_opcode != EXTENDED_ARG) { + int new_oparg = oparg_from_instruction_and_update_offset( + i, opcode, oparg, &cache_offset + ); + if (new_oparg < 0) { + /* Not possible to allocate a cache for this instruction */ + previous_opcode = opcode; + continue; + } + previous_opcode = adaptive_opcode; + int entries_needed = cache_requirements[opcode]; + if (entries_needed) { + /* Initialize the adpative cache entry */ + int cache0_offset = cache_offset-entries_needed; + SpecializedCacheEntry *cache = + _GetSpecializedCacheEntry(instructions, cache0_offset); + cache->adaptive.original_oparg = oparg; + cache->adaptive.counter = 0; + } else { + // oparg is the adaptive cache counter + new_oparg = 0; + } + instructions[i] = _Py_MAKECODEUNIT(adaptive_opcode, new_oparg); } - instructions[i] = _Py_MAKECODEUNIT(adaptive_opcode, new_oparg); } else { /* Super instructions don't use the cache, @@ -1922,10 +1926,12 @@ binary_op_fail_kind(int oparg, PyObject *lhs, PyObject *rhs) void _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, - SpecializedCacheEntry *cache) + int oparg) { - _PyAdaptiveEntry *adaptive = &cache->adaptive; - switch (adaptive->original_oparg) { + assert(_PyOpcode_InlineCacheEntries[BINARY_OP] == + INLINE_CACHE_ENTRIES_BINARY_OP); + _PyBinaryOpCache *cache = (_PyBinaryOpCache *)(instr + 1); + switch (oparg) { case NB_ADD: case NB_INPLACE_ADD: if (!Py_IS_TYPE(lhs, Py_TYPE(rhs))) { @@ -1934,20 +1940,18 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, if (PyUnicode_CheckExact(lhs)) { if (_Py_OPCODE(instr[1]) == STORE_FAST && Py_REFCNT(lhs) == 2) { *instr = _Py_MAKECODEUNIT(BINARY_OP_INPLACE_ADD_UNICODE, - _Py_OPARG(*instr)); + oparg); goto success; } - *instr = _Py_MAKECODEUNIT(BINARY_OP_ADD_UNICODE, - _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(BINARY_OP_ADD_UNICODE, oparg); goto success; } if (PyLong_CheckExact(lhs)) { - *instr = _Py_MAKECODEUNIT(BINARY_OP_ADD_INT, _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(BINARY_OP_ADD_INT, oparg); goto success; } if (PyFloat_CheckExact(lhs)) { - *instr = _Py_MAKECODEUNIT(BINARY_OP_ADD_FLOAT, - _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(BINARY_OP_ADD_FLOAT, oparg); goto success; } break; @@ -1957,13 +1961,11 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, break; } if (PyLong_CheckExact(lhs)) { - *instr = _Py_MAKECODEUNIT(BINARY_OP_MULTIPLY_INT, - _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(BINARY_OP_MULTIPLY_INT, oparg); goto success; } if (PyFloat_CheckExact(lhs)) { - *instr = _Py_MAKECODEUNIT(BINARY_OP_MULTIPLY_FLOAT, - _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(BINARY_OP_MULTIPLY_FLOAT, oparg); goto success; } break; @@ -1973,13 +1975,11 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, break; } if (PyLong_CheckExact(lhs)) { - *instr = _Py_MAKECODEUNIT(BINARY_OP_SUBTRACT_INT, - _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(BINARY_OP_SUBTRACT_INT, oparg); goto success; } if (PyFloat_CheckExact(lhs)) { - *instr = _Py_MAKECODEUNIT(BINARY_OP_SUBTRACT_FLOAT, - _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(BINARY_OP_SUBTRACT_FLOAT, oparg); goto success; } break; @@ -1990,18 +1990,17 @@ _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, // back to BINARY_OP (unless we're collecting stats, where it's more // important to get accurate hit counts for the unadaptive version // and each of the different failure types): - *instr = _Py_MAKECODEUNIT(BINARY_OP, adaptive->original_oparg); + *instr = _Py_MAKECODEUNIT(BINARY_OP, oparg); return; #endif } - SPECIALIZATION_FAIL( - BINARY_OP, binary_op_fail_kind(adaptive->original_oparg, lhs, rhs)); + SPECIALIZATION_FAIL(BINARY_OP, binary_op_fail_kind(oparg, lhs, rhs)); STAT_INC(BINARY_OP, failure); - cache_backoff(adaptive); + cache->counter = ADAPTIVE_CACHE_BACKOFF; return; success: STAT_INC(BINARY_OP, success); - adaptive->counter = initial_counter_value(); + cache->counter = initial_counter_value(); } diff --git a/Python/wordcode_helpers.h b/Python/wordcode_helpers.h deleted file mode 100644 index c8f7a0f41f903..0000000000000 --- a/Python/wordcode_helpers.h +++ /dev/null @@ -1,44 +0,0 @@ -/* This file contains code shared by the compiler and the peephole - optimizer. - */ - -#ifdef WORDS_BIGENDIAN -# define PACKOPARG(opcode, oparg) ((_Py_CODEUNIT)(((opcode) << 8) | (oparg))) -#else -# define PACKOPARG(opcode, oparg) ((_Py_CODEUNIT)(((oparg) << 8) | (opcode))) -#endif - -/* Minimum number of code units necessary to encode instruction with - EXTENDED_ARGs */ -static int -instrsize(unsigned int oparg) -{ - return oparg <= 0xff ? 1 : - oparg <= 0xffff ? 2 : - oparg <= 0xffffff ? 3 : - 4; -} - -/* Spits out op/oparg pair using ilen bytes. codestr should be pointed at the - desired location of the first EXTENDED_ARG */ -static void -write_op_arg(_Py_CODEUNIT *codestr, unsigned char opcode, - unsigned int oparg, int ilen) -{ - switch (ilen) { - case 4: - *codestr++ = PACKOPARG(EXTENDED_ARG, (oparg >> 24) & 0xff); - /* fall through */ - case 3: - *codestr++ = PACKOPARG(EXTENDED_ARG, (oparg >> 16) & 0xff); - /* fall through */ - case 2: - *codestr++ = PACKOPARG(EXTENDED_ARG, (oparg >> 8) & 0xff); - /* fall through */ - case 1: - *codestr++ = PACKOPARG(opcode, oparg & 0xff); - break; - default: - Py_UNREACHABLE(); - } -} diff --git a/Tools/scripts/generate_opcode_h.py b/Tools/scripts/generate_opcode_h.py index 0ee4b95bbfdeb..1f21347935d83 100644 --- a/Tools/scripts/generate_opcode_h.py +++ b/Tools/scripts/generate_opcode_h.py @@ -53,6 +53,7 @@ def main(opcode_py, outfile='Include/opcode.h'): code = fp.read() exec(code, opcode) opmap = opcode['opmap'] + opname = opcode['opname'] hasconst = opcode['hasconst'] hasjrel = opcode['hasjrel'] hasjabs = opcode['hasjabs'] @@ -62,7 +63,7 @@ def main(opcode_py, outfile='Include/opcode.h'): used[op] = True with open(outfile, 'w') as fobj: fobj.write(header) - for name in opcode['opname']: + for name in opname: if name in opmap: fobj.write(DEFINE.format(name, opmap[name])) if name == 'POP_EXCEPT': # Special entry for HAVE_ARGUMENT @@ -89,6 +90,12 @@ def main(opcode_py, outfile='Include/opcode.h'): for i, (op, _) in enumerate(opcode["_nb_ops"]): fobj.write(DEFINE.format(op, i)) + fobj.write("\nstatic const uint8_t _PyOpcode_InlineCacheEntries[256] = {\n") + for i, entries in enumerate(opcode["_inline_cache_entries"]): + if entries: + fobj.write(f" [{opname[i]}] = {entries},\n") + fobj.write("};\n") + fobj.write(footer) From webhook-mailer at python.org Fri Feb 25 07:17:54 2022 From: webhook-mailer at python.org (markshannon) Date: Fri, 25 Feb 2022 12:17:54 -0000 Subject: [Python-checkins] bpo-46808: remove NEXT_BLOCK() from compile.c (GH-31448) Message-ID: https://github.com/python/cpython/commit/c579243eb62d3182c84004cd72dcf6ef59100643 commit: c579243eb62d3182c84004cd72dcf6ef59100643 branch: main author: Irit Katriel <1055913+iritkatriel at users.noreply.github.com> committer: markshannon date: 2022-02-25T12:17:50Z summary: bpo-46808: remove NEXT_BLOCK() from compile.c (GH-31448) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-20-23-10-14.bpo-46808.vouNSF.rst M Python/compile.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-20-23-10-14.bpo-46808.vouNSF.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-20-23-10-14.bpo-46808.vouNSF.rst new file mode 100644 index 0000000000000..9b6009021d63a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-20-23-10-14.bpo-46808.vouNSF.rst @@ -0,0 +1 @@ +Remove the ``NEXT_BLOCK`` macro from compile.c, and make the compiler automatically generate implicit blocks when they are needed. diff --git a/Python/compile.c b/Python/compile.c index 40e6263ff7dd8..8ae4d44e9c6f3 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -268,6 +268,9 @@ struct compiler_unit { int u_col_offset; /* the offset of the current stmt */ int u_end_lineno; /* the end line of the current stmt */ int u_end_col_offset; /* the end offset of the current stmt */ + + /* true if we need to create an implicit basicblock before the next instr */ + int u_need_new_implicit_block; }; /* This struct captures the global state of a compilation. @@ -757,23 +760,13 @@ compiler_new_block(struct compiler *c) return b; } -static basicblock * -compiler_next_block(struct compiler *c) -{ - basicblock *block = compiler_new_block(c); - if (block == NULL) - return NULL; - c->u->u_curblock->b_next = block; - c->u->u_curblock = block; - return block; -} - static basicblock * compiler_use_next_block(struct compiler *c, basicblock *block) { assert(block != NULL); c->u->u_curblock->b_next = block; c->u->u_curblock = block; + c->u->u_need_new_implicit_block = 0; return block; } @@ -1141,6 +1134,37 @@ PyCompile_OpcodeStackEffect(int opcode, int oparg) return stack_effect(opcode, oparg, -1); } +static int is_end_of_basic_block(struct instr *instr) +{ + int opcode = instr->i_opcode; + + return is_jump(instr) || + opcode == RETURN_VALUE || + opcode == RAISE_VARARGS || + opcode == RERAISE; +} + +static int +compiler_use_new_implicit_block_if_needed(struct compiler *c) +{ + if (c->u->u_need_new_implicit_block) { + basicblock *b = compiler_new_block(c); + if (b == NULL) { + return -1; + } + compiler_use_next_block(c, b); + } + return 0; +} + +static void +compiler_check_if_end_of_block(struct compiler *c, struct instr *instr) +{ + if (is_end_of_basic_block(instr)) { + c->u->u_need_new_implicit_block = 1; + } +} + /* Add an opcode with no argument. Returns 0 on failure, 1 on success. */ @@ -1149,23 +1173,29 @@ static int compiler_addop_line(struct compiler *c, int opcode, int line, int end_line, int col_offset, int end_col_offset) { - basicblock *b; - struct instr *i; - int off; assert(!HAS_ARG(opcode) || IS_ARTIFICIAL(opcode)); - off = compiler_next_instr(c->u->u_curblock); - if (off < 0) + + if (compiler_use_new_implicit_block_if_needed(c) < 0) { + return -1; + } + + basicblock *b = c->u->u_curblock; + int off = compiler_next_instr(b); + if (off < 0) { return 0; - b = c->u->u_curblock; - i = &b->b_instr[off]; + } + struct instr *i = &b->b_instr[off]; i->i_opcode = opcode; i->i_oparg = 0; - if (opcode == RETURN_VALUE) + if (opcode == RETURN_VALUE) { b->b_return = 1; + } i->i_lineno = line; i->i_end_lineno = end_line; i->i_col_offset = col_offset; i->i_end_col_offset = end_col_offset; + + compiler_check_if_end_of_block(c, i); return 1; } @@ -1377,9 +1407,6 @@ compiler_addop_i_line(struct compiler *c, int opcode, Py_ssize_t oparg, int lineno, int end_lineno, int col_offset, int end_col_offset) { - struct instr *i; - int off; - /* oparg value is unsigned, but a signed C int is usually used to store it in the C code (like Python/ceval.c). @@ -1387,19 +1414,28 @@ compiler_addop_i_line(struct compiler *c, int opcode, Py_ssize_t oparg, The argument of a concrete bytecode instruction is limited to 8-bit. EXTENDED_ARG is used for 16, 24, and 32-bit arguments. */ + assert(HAS_ARG(opcode)); assert(0 <= oparg && oparg <= 2147483647); - off = compiler_next_instr(c->u->u_curblock); - if (off < 0) + if (compiler_use_new_implicit_block_if_needed(c) < 0) { + return -1; + } + + basicblock *b = c->u->u_curblock; + int off = compiler_next_instr(b); + if (off < 0) { return 0; - i = &c->u->u_curblock->b_instr[off]; + } + struct instr *i = &b->b_instr[off]; i->i_opcode = opcode; i->i_oparg = Py_SAFE_DOWNCAST(oparg, Py_ssize_t, int); i->i_lineno = lineno; i->i_end_lineno = end_lineno; i->i_col_offset = col_offset; i->i_end_col_offset = end_col_offset; + + compiler_check_if_end_of_block(c, i); return 1; } @@ -1417,15 +1453,19 @@ compiler_addop_i_noline(struct compiler *c, int opcode, Py_ssize_t oparg) return compiler_addop_i_line(c, opcode, oparg, -1, 0, 0, 0); } -static int add_jump_to_block(basicblock *b, int opcode, +static int add_jump_to_block(struct compiler *c, int opcode, int lineno, int end_lineno, int col_offset, int end_col_offset, basicblock *target) { assert(HAS_ARG(opcode)); - assert(b != NULL); assert(target != NULL); + if (compiler_use_new_implicit_block_if_needed(c) < 0) { + return -1; + } + + basicblock *b = c->u->u_curblock; int off = compiler_next_instr(b); struct instr *i = &b->b_instr[off]; if (off < 0) { @@ -1437,13 +1477,15 @@ static int add_jump_to_block(basicblock *b, int opcode, i->i_end_lineno = end_lineno; i->i_col_offset = col_offset; i->i_end_col_offset = end_col_offset; + + compiler_check_if_end_of_block(c, i); return 1; } static int compiler_addop_j(struct compiler *c, int opcode, basicblock *b) { - return add_jump_to_block(c->u->u_curblock, opcode, c->u->u_lineno, + return add_jump_to_block(c, opcode, c->u->u_lineno, c->u->u_end_lineno, c->u->u_col_offset, c->u->u_end_col_offset, b); } @@ -1451,18 +1493,7 @@ compiler_addop_j(struct compiler *c, int opcode, basicblock *b) static int compiler_addop_j_noline(struct compiler *c, int opcode, basicblock *b) { - return add_jump_to_block(c->u->u_curblock, opcode, -1, 0, 0, 0, b); -} - -/* NEXT_BLOCK() creates an implicit jump from the current block - to the new block. - - The returns inside this macro make it impossible to decref objects - created in the local function. Local objects should use the arena. -*/ -#define NEXT_BLOCK(C) { \ - if (compiler_next_block((C)) == NULL) \ - return 0; \ + return add_jump_to_block(c, opcode, -1, 0, 0, 0, b); } #define ADDOP(C, OP) { \ @@ -2823,12 +2854,10 @@ compiler_jump_if(struct compiler *c, expr_ty e, basicblock *next, int cond) ADDOP_I(c, COPY, 2); ADDOP_COMPARE(c, asdl_seq_GET(e->v.Compare.ops, i)); ADDOP_JUMP(c, POP_JUMP_IF_FALSE, cleanup); - NEXT_BLOCK(c); } VISIT(c, expr, (expr_ty)asdl_seq_GET(e->v.Compare.comparators, n)); ADDOP_COMPARE(c, asdl_seq_GET(e->v.Compare.ops, n)); ADDOP_JUMP(c, cond ? POP_JUMP_IF_TRUE : POP_JUMP_IF_FALSE, next); - NEXT_BLOCK(c); basicblock *end = compiler_new_block(c); if (end == NULL) return 0; @@ -2852,7 +2881,6 @@ compiler_jump_if(struct compiler *c, expr_ty e, basicblock *next, int cond) /* general implementation */ VISIT(c, expr, e); ADDOP_JUMP(c, cond ? POP_JUMP_IF_TRUE : POP_JUMP_IF_FALSE, next); - NEXT_BLOCK(c); return 1; } @@ -3128,7 +3156,6 @@ compiler_return(struct compiler *c, stmt_ty s) ADDOP_LOAD_CONST(c, s->v.Return.value->v.Constant.value); } ADDOP(c, RETURN_VALUE); - NEXT_BLOCK(c); return 1; } @@ -3149,7 +3176,6 @@ compiler_break(struct compiler *c) return 0; } ADDOP_JUMP(c, JUMP_ABSOLUTE, loop->fb_exit); - NEXT_BLOCK(c); return 1; } @@ -3166,7 +3192,6 @@ compiler_continue(struct compiler *c) return compiler_error(c, "'continue' not properly in loop"); } ADDOP_JUMP(c, JUMP_ABSOLUTE, loop->fb_block); - NEXT_BLOCK(c) return 1; } @@ -3348,7 +3373,6 @@ compiler_try_except(struct compiler *c, stmt_ty s) compiler_pop_fblock(c, TRY_EXCEPT, body); ADDOP_NOLINE(c, POP_BLOCK); if (s->v.Try.orelse && asdl_seq_LEN(s->v.Try.orelse)) { - NEXT_BLOCK(c); VISIT_SEQ(c, stmt, s->v.Try.orelse); } ADDOP_JUMP_NOLINE(c, JUMP_FORWARD, end); @@ -3374,7 +3398,6 @@ compiler_try_except(struct compiler *c, stmt_ty s) if (handler->v.ExceptHandler.type) { VISIT(c, expr, handler->v.ExceptHandler.type); ADDOP_JUMP(c, JUMP_IF_NOT_EXC_MATCH, except); - NEXT_BLOCK(c); } if (handler->v.ExceptHandler.name) { basicblock *cleanup_end, *cleanup_body; @@ -3580,7 +3603,6 @@ compiler_try_star_except(struct compiler *c, stmt_ty s) if (handler->v.ExceptHandler.type) { VISIT(c, expr, handler->v.ExceptHandler.type); ADDOP_JUMP(c, JUMP_IF_NOT_EG_MATCH, except); - NEXT_BLOCK(c); } basicblock *cleanup_end = compiler_new_block(c); @@ -3665,7 +3687,6 @@ compiler_try_star_except(struct compiler *c, stmt_ty s) ADDOP(c, PREP_RERAISE_STAR); ADDOP_I(c, COPY, 1); ADDOP_JUMP(c, POP_JUMP_IF_NOT_NONE, reraise); - NEXT_BLOCK(c); /* Nothing to reraise */ ADDOP(c, POP_TOP); @@ -3957,7 +3978,6 @@ compiler_visit_stmt(struct compiler *c, stmt_ty s) } } ADDOP_I(c, RAISE_VARARGS, (int)n); - NEXT_BLOCK(c); break; case Try_kind: return compiler_try(c, s); @@ -4503,7 +4523,6 @@ compiler_compare(struct compiler *c, expr_ty e) ADDOP_I(c, COPY, 2); ADDOP_COMPARE(c, asdl_seq_GET(e->v.Compare.ops, i)); ADDOP_JUMP(c, JUMP_IF_FALSE_OR_POP, cleanup); - NEXT_BLOCK(c); } VISIT(c, expr, (expr_ty)asdl_seq_GET(e->v.Compare.comparators, n)); ADDOP_COMPARE(c, asdl_seq_GET(e->v.Compare.ops, n)); @@ -5093,7 +5112,6 @@ compiler_sync_comprehension_generator(struct compiler *c, depth++; compiler_use_next_block(c, start); ADDOP_JUMP(c, FOR_ITER, anchor); - NEXT_BLOCK(c); } VISIT(c, expr, gen->target); @@ -5103,7 +5121,6 @@ compiler_sync_comprehension_generator(struct compiler *c, expr_ty e = (expr_ty)asdl_seq_GET(gen->ifs, i); if (!compiler_jump_if(c, e, if_cleanup, 0)) return 0; - NEXT_BLOCK(c); } if (++gen_index < asdl_seq_LEN(generators)) @@ -5198,7 +5215,6 @@ compiler_async_comprehension_generator(struct compiler *c, expr_ty e = (expr_ty)asdl_seq_GET(gen->ifs, i); if (!compiler_jump_if(c, e, if_cleanup, 0)) return 0; - NEXT_BLOCK(c); } depth++; @@ -5410,7 +5426,6 @@ compiler_with_except_finish(struct compiler *c, basicblock * cleanup) { if (exit == NULL) return 0; ADDOP_JUMP(c, POP_JUMP_IF_TRUE, exit); - NEXT_BLOCK(c); ADDOP_I(c, RERAISE, 2); compiler_use_next_block(c, cleanup); POP_EXCEPT_AND_RERAISE(c); @@ -6149,7 +6164,6 @@ jump_to_fail_pop(struct compiler *c, pattern_context *pc, int op) Py_ssize_t pops = pc->on_top + PyList_GET_SIZE(pc->stores); RETURN_IF_FALSE(ensure_fail_pop(c, pc, pops)); ADDOP_JUMP(c, op, pc->fail_pop[pops]); - NEXT_BLOCK(c); return 1; } @@ -6159,7 +6173,6 @@ emit_and_reset_fail_pop(struct compiler *c, pattern_context *pc) { if (!pc->fail_pop_size) { assert(pc->fail_pop == NULL); - NEXT_BLOCK(c); return 1; } while (--pc->fail_pop_size) { @@ -6662,7 +6675,6 @@ compiler_pattern_or(struct compiler *c, pattern_ty p, pattern_context *pc) } assert(control); if (!compiler_addop_j(c, JUMP_FORWARD, end) || - !compiler_next_block(c) || !emit_and_reset_fail_pop(c, pc)) { goto error; @@ -8136,10 +8148,7 @@ assemble(struct compiler *c, int addNone) PyCodeObject *co = NULL; PyObject *consts = NULL; - /* Make sure every block that falls off the end returns None. - XXX NEXT_BLOCK() isn't quite right, because if the last - block ends with a jump or return b_next shouldn't set. - */ + /* Make sure every block that falls off the end returns None. */ if (!c->u->u_curblock->b_return) { UNSET_LOC(c); if (addNone) From webhook-mailer at python.org Fri Feb 25 09:41:41 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 14:41:41 -0000 Subject: [Python-checkins] bpo-45316: Move private PyCode C API to internal C API (GH-31576) Message-ID: https://github.com/python/cpython/commit/4a0c7a1aacd08cead7717479620e62359c828e88 commit: 4a0c7a1aacd08cead7717479620e62359c828e88 branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T15:41:32+01:00 summary: bpo-45316: Move private PyCode C API to internal C API (GH-31576) Rename private functions (no exported), add an underscore prefix: * PyLineTable_InitAddressRange() => _PyLineTable_InitAddressRange() * PyLineTable_NextAddressRange() => _PyLineTable_NextAddressRange() * PyLineTable_PreviousAddressRange() => _PyLineTable_PreviousAddressRange() Move private functions to the internal C API: * _PyCode_Addr2EndLine() * _PyCode_Addr2EndOffset() * _PyCode_Addr2Offset() * _PyCode_InitAddressRange() * _PyCode_InitEndAddressRange( * _PyLineTable_InitAddressRange() * _PyLineTable_NextAddressRange() * _PyLineTable_PreviousAddressRange() No longer export the following internal functions: * _PyCode_GetVarnames() * _PyCode_GetCellvars() * _PyCode_GetFreevars() * _Py_GetSpecializationStats() Add "extern" to pycore_code.h functions to identify them more easiliy (they are still not exported). files: M Include/cpython/code.h M Include/internal/pycore_code.h M Objects/codeobject.c M Objects/frameobject.c diff --git a/Include/cpython/code.h b/Include/cpython/code.h index 39a44ce3d7e61..21f8fe7ddad4a 100644 --- a/Include/cpython/code.h +++ b/Include/cpython/code.h @@ -174,13 +174,6 @@ PyAPI_FUNC(int) PyCode_Addr2Line(PyCodeObject *, int); PyAPI_FUNC(int) PyCode_Addr2Location(PyCodeObject *, int, int *, int *, int *, int *); -/* Return the ending source code line number from a bytecode index. */ -PyAPI_FUNC(int) _PyCode_Addr2EndLine(PyCodeObject *, int); -/* Return the starting source code column offset from a bytecode index. */ -PyAPI_FUNC(int) _PyCode_Addr2Offset(PyCodeObject *, int); -/* Return the ending source code column offset from a bytecode index. */ -PyAPI_FUNC(int) _PyCode_Addr2EndOffset(PyCodeObject *, int); - /* for internal use only */ struct _opaque { int computed_line; @@ -217,15 +210,3 @@ PyAPI_FUNC(int) _PyCode_GetExtra(PyObject *code, Py_ssize_t index, void **extra); PyAPI_FUNC(int) _PyCode_SetExtra(PyObject *code, Py_ssize_t index, void *extra); - -/** API for initializing the line number tables. */ -int _PyCode_InitAddressRange(PyCodeObject* co, PyCodeAddressRange *bounds); -int _PyCode_InitEndAddressRange(PyCodeObject* co, PyCodeAddressRange* bounds); - -/** Out of process API for initializing the line number table. */ -void PyLineTable_InitAddressRange(const char *linetable, Py_ssize_t length, int firstlineno, PyCodeAddressRange *range); - -/** API for traversing the line number table. */ -int PyLineTable_NextAddressRange(PyCodeAddressRange *range); -int PyLineTable_PreviousAddressRange(PyCodeAddressRange *range); - diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index 2791495e704e1..d83df5e300468 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -259,9 +259,35 @@ PyAPI_FUNC(PyCodeObject *) _PyCode_New(struct _PyCodeConstructor *); /* Private API */ /* Getters for internal PyCodeObject data. */ -PyAPI_FUNC(PyObject *) _PyCode_GetVarnames(PyCodeObject *); -PyAPI_FUNC(PyObject *) _PyCode_GetCellvars(PyCodeObject *); -PyAPI_FUNC(PyObject *) _PyCode_GetFreevars(PyCodeObject *); +extern PyObject* _PyCode_GetVarnames(PyCodeObject *); +extern PyObject* _PyCode_GetCellvars(PyCodeObject *); +extern PyObject* _PyCode_GetFreevars(PyCodeObject *); + +/* Return the ending source code line number from a bytecode index. */ +extern int _PyCode_Addr2EndLine(PyCodeObject *, int); + +/* Return the ending source code line number from a bytecode index. */ +extern int _PyCode_Addr2EndLine(PyCodeObject *, int); +/* Return the starting source code column offset from a bytecode index. */ +extern int _PyCode_Addr2Offset(PyCodeObject *, int); +/* Return the ending source code column offset from a bytecode index. */ +extern int _PyCode_Addr2EndOffset(PyCodeObject *, int); + +/** API for initializing the line number tables. */ +extern int _PyCode_InitAddressRange(PyCodeObject* co, PyCodeAddressRange *bounds); +extern int _PyCode_InitEndAddressRange(PyCodeObject* co, PyCodeAddressRange* bounds); + +/** Out of process API for initializing the line number table. */ +extern void _PyLineTable_InitAddressRange( + const char *linetable, + Py_ssize_t length, + int firstlineno, + PyCodeAddressRange *range); + +/** API for traversing the line number table. */ +extern int _PyLineTable_NextAddressRange(PyCodeAddressRange *range); +extern int _PyLineTable_PreviousAddressRange(PyCodeAddressRange *range); + #define ADAPTIVE_CACHE_BACKOFF 64 @@ -272,26 +298,26 @@ cache_backoff(_PyAdaptiveEntry *entry) { /* Specialization functions */ -int _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); -int _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); -int _Py_Specialize_LoadGlobal(PyObject *globals, PyObject *builtins, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); -int _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); -int _Py_Specialize_BinarySubscr(PyObject *sub, PyObject *container, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); -int _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *instr); -int _Py_Specialize_Call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, +extern int _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); +extern int _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); +extern int _Py_Specialize_LoadGlobal(PyObject *globals, PyObject *builtins, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); +extern int _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); +extern int _Py_Specialize_BinarySubscr(PyObject *sub, PyObject *container, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); +extern int _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *instr); +extern int _Py_Specialize_Call(PyObject *callable, _Py_CODEUNIT *instr, int nargs, PyObject *kwnames, SpecializedCacheEntry *cache); -int _Py_Specialize_Precall(PyObject *callable, _Py_CODEUNIT *instr, int nargs, +extern int _Py_Specialize_Precall(PyObject *callable, _Py_CODEUNIT *instr, int nargs, PyObject *kwnames, SpecializedCacheEntry *cache, PyObject *builtins); -void _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, +extern void _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, int oparg); -void _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); -void _Py_Specialize_UnpackSequence(PyObject *seq, _Py_CODEUNIT *instr, +extern void _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); +extern void _Py_Specialize_UnpackSequence(PyObject *seq, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); /* Deallocator function for static codeobjects used in deepfreeze.py */ -void _PyStaticCode_Dealloc(PyCodeObject *co); +extern void _PyStaticCode_Dealloc(PyCodeObject *co); /* Function to intern strings of codeobjects */ -void _PyStaticCode_InternStrings(PyCodeObject *co); +extern void _PyStaticCode_InternStrings(PyCodeObject *co); #ifdef Py_STATS @@ -343,9 +369,9 @@ extern PyStats _py_stats; #define CALL_STAT_INC(name) _py_stats.call_stats.name++ #define OBJECT_STAT_INC(name) _py_stats.object_stats.name++ -void _Py_PrintSpecializationStats(int to_file); +extern void _Py_PrintSpecializationStats(int to_file); -PyAPI_FUNC(PyObject*) _Py_GetSpecializationStats(void); +extern PyObject* _Py_GetSpecializationStats(void); #else #define STAT_INC(opname, name) ((void)0) diff --git a/Objects/codeobject.c b/Objects/codeobject.c index f166204de2d8f..f947595803aed 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -676,7 +676,7 @@ _PyCode_Addr2EndOffset(PyCodeObject* co, int addrq) } void -PyLineTable_InitAddressRange(const char *linetable, Py_ssize_t length, int firstlineno, PyCodeAddressRange *range) +_PyLineTable_InitAddressRange(const char *linetable, Py_ssize_t length, int firstlineno, PyCodeAddressRange *range) { range->opaque.lo_next = linetable; range->opaque.limit = range->opaque.lo_next + length; @@ -691,7 +691,7 @@ _PyCode_InitAddressRange(PyCodeObject* co, PyCodeAddressRange *bounds) { const char *linetable = PyBytes_AS_STRING(co->co_linetable); Py_ssize_t length = PyBytes_GET_SIZE(co->co_linetable); - PyLineTable_InitAddressRange(linetable, length, co->co_firstlineno, bounds); + _PyLineTable_InitAddressRange(linetable, length, co->co_firstlineno, bounds); return bounds->ar_line; } @@ -700,7 +700,7 @@ _PyCode_InitEndAddressRange(PyCodeObject* co, PyCodeAddressRange* bounds) { char* linetable = PyBytes_AS_STRING(co->co_endlinetable); Py_ssize_t length = PyBytes_GET_SIZE(co->co_endlinetable); - PyLineTable_InitAddressRange(linetable, length, co->co_firstlineno, bounds); + _PyLineTable_InitAddressRange(linetable, length, co->co_firstlineno, bounds); return bounds->ar_line; } @@ -710,12 +710,12 @@ int _PyCode_CheckLineNumber(int lasti, PyCodeAddressRange *bounds) { while (bounds->ar_end <= lasti) { - if (!PyLineTable_NextAddressRange(bounds)) { + if (!_PyLineTable_NextAddressRange(bounds)) { return -1; } } while (bounds->ar_start > lasti) { - if (!PyLineTable_PreviousAddressRange(bounds)) { + if (!_PyLineTable_PreviousAddressRange(bounds)) { return -1; } } @@ -765,7 +765,7 @@ at_end(PyCodeAddressRange *bounds) { } int -PyLineTable_PreviousAddressRange(PyCodeAddressRange *range) +_PyLineTable_PreviousAddressRange(PyCodeAddressRange *range) { if (range->ar_start <= 0) { return 0; @@ -779,7 +779,7 @@ PyLineTable_PreviousAddressRange(PyCodeAddressRange *range) } int -PyLineTable_NextAddressRange(PyCodeAddressRange *range) +_PyLineTable_NextAddressRange(PyCodeAddressRange *range) { if (at_end(range)) { return 0; @@ -847,7 +847,7 @@ decode_linetable(PyCodeObject *code) return NULL; } _PyCode_InitAddressRange(code, &bounds); - while (PyLineTable_NextAddressRange(&bounds)) { + while (_PyLineTable_NextAddressRange(&bounds)) { if (bounds.opaque.computed_line != line) { int bdelta = bounds.ar_start - code_offset; int ldelta = bounds.opaque.computed_line - line; @@ -883,7 +883,7 @@ static PyObject * lineiter_next(lineiterator *li) { PyCodeAddressRange *bounds = &li->li_line; - if (!PyLineTable_NextAddressRange(bounds)) { + if (!_PyLineTable_NextAddressRange(bounds)) { return NULL; } PyObject *start = NULL; diff --git a/Objects/frameobject.c b/Objects/frameobject.c index dc210030e8a15..c9445e3bb36da 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -377,9 +377,9 @@ marklines(PyCodeObject *code, int len) linestarts[i] = -1; } - while (PyLineTable_NextAddressRange(&bounds)) { - assert(bounds.ar_start/(int)sizeof(_Py_CODEUNIT) < len); - linestarts[bounds.ar_start/sizeof(_Py_CODEUNIT)] = bounds.ar_line; + while (_PyLineTable_NextAddressRange(&bounds)) { + assert(bounds.ar_start / (int)sizeof(_Py_CODEUNIT) < len); + linestarts[bounds.ar_start / sizeof(_Py_CODEUNIT)] = bounds.ar_line; } return linestarts; } From webhook-mailer at python.org Fri Feb 25 09:42:00 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 14:42:00 -0000 Subject: [Python-checkins] bpo-45316: Move private PyDict functions to internal C API (GH-31577) Message-ID: https://github.com/python/cpython/commit/8ddbdd9e96e64b42c87dcfe4e38383cf0694988a commit: 8ddbdd9e96e64b42c87dcfe4e38383cf0694988a branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T15:41:55+01:00 summary: bpo-45316: Move private PyDict functions to internal C API (GH-31577) Move the following private unexported functions to the internal C API headers: * _PyDictKeys_GetVersionForCurrentState() * _PyDictKeys_StringLookup() * _PyDict_FromKeys() * _PyDict_GetItemHint() * _PyDict_KeysSize() * _PyDict_LoadGlobal() * _PyDict_NewKeysForClass() * _PyDict_Pop_KnownHash() * _PyDict_SetItem_Take2() * _PyObjectDict_SetItem() * _PyObject_MakeDictFromInstanceAttributes() * _Py_dict_lookup() files: M Include/cpython/dictobject.h M Include/internal/pycore_dict.h diff --git a/Include/cpython/dictobject.h b/Include/cpython/dictobject.h index 68b4593acf6c4..033eaeb4c9751 100644 --- a/Include/cpython/dictobject.h +++ b/Include/cpython/dictobject.h @@ -42,7 +42,6 @@ PyAPI_FUNC(int) _PyDict_DelItem_KnownHash(PyObject *mp, PyObject *key, Py_hash_t hash); PyAPI_FUNC(int) _PyDict_DelItemIf(PyObject *mp, PyObject *key, int (*predicate)(PyObject *value)); -PyDictKeysObject *_PyDict_NewKeysForClass(void); PyAPI_FUNC(int) _PyDict_Next( PyObject *mp, Py_ssize_t *pos, PyObject **key, PyObject **value, Py_hash_t *hash); @@ -53,11 +52,8 @@ PyAPI_FUNC(int) _PyDict_ContainsId(PyObject *, _Py_Identifier *); PyAPI_FUNC(PyObject *) _PyDict_NewPresized(Py_ssize_t minused); PyAPI_FUNC(void) _PyDict_MaybeUntrack(PyObject *mp); PyAPI_FUNC(int) _PyDict_HasOnlyStringKeys(PyObject *mp); -Py_ssize_t _PyDict_KeysSize(PyDictKeysObject *keys); PyAPI_FUNC(Py_ssize_t) _PyDict_SizeOf(PyDictObject *); PyAPI_FUNC(PyObject *) _PyDict_Pop(PyObject *, PyObject *, PyObject *); -PyObject *_PyDict_Pop_KnownHash(PyObject *, PyObject *, Py_hash_t, PyObject *); -PyObject *_PyDict_FromKeys(PyObject *, PyObject *, PyObject *); #define _PyDict_HasSplitTable(d) ((d)->ma_values != NULL) /* Like PyDict_Merge, but override can be 0, 1 or 2. If override is 0, @@ -71,10 +67,6 @@ PyAPI_FUNC(int) _PyDict_SetItemId(PyObject *dp, _Py_Identifier *key, PyObject *i PyAPI_FUNC(int) _PyDict_DelItemId(PyObject *mp, _Py_Identifier *key); PyAPI_FUNC(void) _PyDict_DebugMallocStats(FILE *out); -int _PyObjectDict_SetItem(PyTypeObject *tp, PyObject **dictptr, PyObject *name, PyObject *value); -PyObject *_PyDict_LoadGlobal(PyDictObject *, PyDictObject *, PyObject *); -Py_ssize_t _PyDict_GetItemHint(PyDictObject *, PyObject *, Py_ssize_t, PyObject **); - /* _PyDictView */ typedef struct { @@ -84,9 +76,3 @@ typedef struct { PyAPI_FUNC(PyObject *) _PyDictView_New(PyObject *, PyTypeObject *); PyAPI_FUNC(PyObject *) _PyDictView_Intersect(PyObject* self, PyObject *other); - -/* Gets a version number unique to the current state of the keys of dict, if possible. - * Returns the version number, or zero if it was not possible to get a version number. */ -uint32_t _PyDictKeys_GetVersionForCurrentState(PyDictKeysObject *dictkeys); - -Py_ssize_t _PyDictKeys_StringLookup(PyDictKeysObject* dictkeys, PyObject *key); diff --git a/Include/internal/pycore_dict.h b/Include/internal/pycore_dict.h index 9cd652b2fb6be..68f6663dc6445 100644 --- a/Include/internal/pycore_dict.h +++ b/Include/internal/pycore_dict.h @@ -43,13 +43,29 @@ typedef struct { PyObject *me_value; /* This field is only meaningful for combined tables */ } PyDictKeyEntry; +extern PyDictKeysObject *_PyDict_NewKeysForClass(void); +extern PyObject *_PyDict_FromKeys(PyObject *, PyObject *, PyObject *); + +/* Gets a version number unique to the current state of the keys of dict, if possible. + * Returns the version number, or zero if it was not possible to get a version number. */ +extern uint32_t _PyDictKeys_GetVersionForCurrentState(PyDictKeysObject *dictkeys); + +extern Py_ssize_t _PyDict_KeysSize(PyDictKeysObject *keys); + /* _Py_dict_lookup() returns index of entry which can be used like DK_ENTRIES(dk)[index]. * -1 when no entry found, -3 when compare raises error. */ -Py_ssize_t _Py_dict_lookup(PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject **value_addr); +extern Py_ssize_t _Py_dict_lookup(PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject **value_addr); + +extern Py_ssize_t _PyDict_GetItemHint(PyDictObject *, PyObject *, Py_ssize_t, PyObject **); +extern Py_ssize_t _PyDictKeys_StringLookup(PyDictKeysObject* dictkeys, PyObject *key); +extern PyObject *_PyDict_LoadGlobal(PyDictObject *, PyDictObject *, PyObject *); /* Consumes references to key and value */ -int _PyDict_SetItem_Take2(PyDictObject *op, PyObject *key, PyObject *value); +extern int _PyDict_SetItem_Take2(PyDictObject *op, PyObject *key, PyObject *value); +extern int _PyObjectDict_SetItem(PyTypeObject *tp, PyObject **dictptr, PyObject *name, PyObject *value); + +extern PyObject *_PyDict_Pop_KnownHash(PyObject *, PyObject *, Py_hash_t, PyObject *); #define DKIX_EMPTY (-1) #define DKIX_DUMMY (-2) /* Used internally */ @@ -138,7 +154,7 @@ extern uint64_t _pydict_global_version; #define DICT_NEXT_VERSION() (++_pydict_global_version) -PyObject *_PyObject_MakeDictFromInstanceAttributes(PyObject *obj, PyDictValues *values); +extern PyObject *_PyObject_MakeDictFromInstanceAttributes(PyObject *obj, PyDictValues *values); static inline void _PyDictValues_AddToInsertionOrder(PyDictValues *values, Py_ssize_t ix) From webhook-mailer at python.org Fri Feb 25 09:47:15 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 14:47:15 -0000 Subject: [Python-checkins] bpo-46852: Rename float.__set_format__() to float.__setformat__() (GH-31558) (GH-31578) Message-ID: https://github.com/python/cpython/commit/0848da19ce8ea037ab1cfc569778e94bf8e3b24a commit: 0848da19ce8ea037ab1cfc569778e94bf8e3b24a branch: 3.10 author: Victor Stinner committer: vstinner date: 2022-02-25T15:47:07+01:00 summary: bpo-46852: Rename float.__set_format__() to float.__setformat__() (GH-31558) (GH-31578) Rename the private undocumented float.__set_format__() method to float.__setformat__() to fix a typo introduced in Python 3.7. The method is only used by test_float. The change enables again test_float tests on the float format which were previously skipped because of the typo. The typo was introduced in Python 3.7 by bpo-20185 in commit b5c51d3dd95bbfde533655fb86ac0f96f771ba7b. (cherry picked from commit 7d03c8be5af2f1559dbc35b775b3116dfd63cfb6) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst M Objects/clinic/floatobject.c.h M Objects/floatobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst new file mode 100644 index 0000000000000..65b826473b915 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst @@ -0,0 +1,3 @@ +Rename the private undocumented ``float.__set_format__()`` method to +``float.__setformat__()`` to fix a typo introduced in Python 3.7. The method +is only used by test_float. Patch by Victor Stinner. diff --git a/Objects/clinic/floatobject.c.h b/Objects/clinic/floatobject.c.h index 5643f0e3ac650..494c0a227139a 100644 --- a/Objects/clinic/floatobject.c.h +++ b/Objects/clinic/floatobject.c.h @@ -289,8 +289,8 @@ float___getformat__(PyTypeObject *type, PyObject *arg) return return_value; } -PyDoc_STRVAR(float___set_format____doc__, -"__set_format__($type, typestr, fmt, /)\n" +PyDoc_STRVAR(float___setformat____doc__, +"__setformat__($type, typestr, fmt, /)\n" "--\n" "\n" "You probably don\'t want to use this function.\n" @@ -307,25 +307,25 @@ PyDoc_STRVAR(float___set_format____doc__, "Override the automatic determination of C-level floating point type.\n" "This affects how floats are converted to and from binary strings."); -#define FLOAT___SET_FORMAT___METHODDEF \ - {"__set_format__", (PyCFunction)(void(*)(void))float___set_format__, METH_FASTCALL|METH_CLASS, float___set_format____doc__}, +#define FLOAT___SETFORMAT___METHODDEF \ + {"__setformat__", (PyCFunction)(void(*)(void))float___setformat__, METH_FASTCALL|METH_CLASS, float___setformat____doc__}, static PyObject * -float___set_format___impl(PyTypeObject *type, const char *typestr, - const char *fmt); +float___setformat___impl(PyTypeObject *type, const char *typestr, + const char *fmt); static PyObject * -float___set_format__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs) +float___setformat__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; const char *typestr; const char *fmt; - if (!_PyArg_CheckPositional("__set_format__", nargs, 2, 2)) { + if (!_PyArg_CheckPositional("__setformat__", nargs, 2, 2)) { goto exit; } if (!PyUnicode_Check(args[0])) { - _PyArg_BadArgument("__set_format__", "argument 1", "str", args[0]); + _PyArg_BadArgument("__setformat__", "argument 1", "str", args[0]); goto exit; } Py_ssize_t typestr_length; @@ -338,7 +338,7 @@ float___set_format__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs goto exit; } if (!PyUnicode_Check(args[1])) { - _PyArg_BadArgument("__set_format__", "argument 2", "str", args[1]); + _PyArg_BadArgument("__setformat__", "argument 2", "str", args[1]); goto exit; } Py_ssize_t fmt_length; @@ -350,7 +350,7 @@ float___set_format__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs PyErr_SetString(PyExc_ValueError, "embedded null character"); goto exit; } - return_value = float___set_format___impl(type, typestr, fmt); + return_value = float___setformat___impl(type, typestr, fmt); exit: return return_value; @@ -387,4 +387,4 @@ float___format__(PyObject *self, PyObject *arg) exit: return return_value; } -/*[clinic end generated code: output=bb079c3e130e4ce6 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=f4aae29054273cb5 input=a9049054013a1b77]*/ diff --git a/Objects/floatobject.c b/Objects/floatobject.c index e4ce7e74d2c57..2e02f37f4a580 100644 --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -1748,7 +1748,7 @@ float___getformat___impl(PyTypeObject *type, const char *typestr) /*[clinic input] @classmethod -float.__set_format__ +float.__setformat__ typestr: str Must be 'double' or 'float'. @@ -1767,9 +1767,9 @@ This affects how floats are converted to and from binary strings. [clinic start generated code]*/ static PyObject * -float___set_format___impl(PyTypeObject *type, const char *typestr, - const char *fmt) -/*[clinic end generated code: output=504460f5dc85acbd input=5306fa2b81a997e4]*/ +float___setformat___impl(PyTypeObject *type, const char *typestr, + const char *fmt) +/*[clinic end generated code: output=06864de1fb5f1f04 input=c0e9e04dd87f9988]*/ { float_format_type f; float_format_type detected; @@ -1871,7 +1871,7 @@ static PyMethodDef float_methods[] = { FLOAT_IS_INTEGER_METHODDEF FLOAT___GETNEWARGS___METHODDEF FLOAT___GETFORMAT___METHODDEF - FLOAT___SET_FORMAT___METHODDEF + FLOAT___SETFORMAT___METHODDEF FLOAT___FORMAT___METHODDEF {NULL, NULL} /* sentinel */ }; From webhook-mailer at python.org Fri Feb 25 09:49:58 2022 From: webhook-mailer at python.org (rhettinger) Date: Fri, 25 Feb 2022 14:49:58 -0000 Subject: [Python-checkins] Update dict/OrderedDict differences with code equivalents. (GH-31563) Message-ID: https://github.com/python/cpython/commit/26aba295a9c1bcb0812fe44bd7e68ddd1d8a6828 commit: 26aba295a9c1bcb0812fe44bd7e68ddd1d8a6828 branch: main author: Raymond Hettinger committer: rhettinger date: 2022-02-25T08:49:53-06:00 summary: Update dict/OrderedDict differences with code equivalents. (GH-31563) files: M Doc/library/collections.rst diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index c45d90cbbc1c7..67b64ddda7a2c 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -1092,18 +1092,35 @@ Some differences from :class:`dict` still remain: Space efficiency, iteration speed, and the performance of update operations were secondary. -* Algorithmically, :class:`OrderedDict` can handle frequent reordering - operations better than :class:`dict`. This makes it suitable for tracking - recent accesses (for example in an `LRU cache - `_). +* The :class:`OrderedDict` algorithm can handle frequent reordering operations + better than :class:`dict`. As shown in the recipes below, this makes it + suitable for implementing various kinds of LRU caches. * The equality operation for :class:`OrderedDict` checks for matching order. + A regular :class:`dict` can emulate the order sensitive equality test with + ``p == q and all(k1 == k2 for k1, k2 in zip(p, q))``. + * The :meth:`popitem` method of :class:`OrderedDict` has a different signature. It accepts an optional argument to specify which item is popped. -* :class:`OrderedDict` has a :meth:`move_to_end` method to - efficiently reposition an element to an endpoint. + A regular :class:`dict` can emulate OrderedDict's ``od.popitem(last=True)`` + with ``d.popitem()`` which is guaranteed to pop the rightmost (last) item. + + A regular :class:`dict` can emulate OrderedDict's ``od.popitem(last=False)`` + with ``(k := next(iter(d)), d.pop(k))`` which will return and remove the + leftmost (first) item if it exists. + +* :class:`OrderedDict` has a :meth:`move_to_end` method to efficiently + reposition an element to an endpoint. + + A regular :class:`dict` can emulate OrderedDict's ``od.move_to_end(k, + last=True)`` with ``d[k] = d.pop(k)`` which will move the key and its + associated value to the rightmost (last) position. + + A regular :class:`dict` does not have an efficient equivalent for + OrderedDict's ``od.move_to_end(k, last=False)`` which moves the key + and its associated value to the leftmost (first) position. * Until Python 3.8, :class:`dict` lacked a :meth:`__reversed__` method. From webhook-mailer at python.org Fri Feb 25 10:07:19 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 15:07:19 -0000 Subject: [Python-checkins] bpo-45316: Move private functions to internal C API (GH-31579) Message-ID: https://github.com/python/cpython/commit/8f2a337a80a283c66e1a4252839792fa229d2763 commit: 8f2a337a80a283c66e1a4252839792fa229d2763 branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T16:07:14+01:00 summary: bpo-45316: Move private functions to internal C API (GH-31579) Move the unexported private functions to the internal C API: * pycore_frame.h: _PyFrame_New_NoTrack() * pycore_function.h: _PyFunction_GetVersionForCurrentState() * pycore_genobject.h: _PyAsyncGenValueWrapperNew() * pycore_genobject.h: _PyCoro_GetAwaitableIter() * pycore_genobject.h: _PyGen_yf() files: M Include/cpython/frameobject.h M Include/cpython/funcobject.h M Include/cpython/genobject.h M Include/internal/pycore_frame.h M Include/internal/pycore_function.h M Include/internal/pycore_genobject.h M Python/specialize.c diff --git a/Include/cpython/frameobject.h b/Include/cpython/frameobject.h index e69209686bee0..ebaecbed1b487 100644 --- a/Include/cpython/frameobject.h +++ b/Include/cpython/frameobject.h @@ -13,11 +13,6 @@ PyAPI_DATA(PyTypeObject) PyFrame_Type; PyAPI_FUNC(PyFrameObject *) PyFrame_New(PyThreadState *, PyCodeObject *, PyObject *, PyObject *); -/* only internal use */ -PyFrameObject* -_PyFrame_New_NoTrack(PyCodeObject *code); - - /* The rest of the interface is specific for frame objects */ /* Conversions between "fast locals" and locals in dictionary */ diff --git a/Include/cpython/funcobject.h b/Include/cpython/funcobject.h index 9f0560fb72503..99ac6008f8b61 100644 --- a/Include/cpython/funcobject.h +++ b/Include/cpython/funcobject.h @@ -82,8 +82,6 @@ PyAPI_FUNC(PyObject *) _PyFunction_Vectorcall( size_t nargsf, PyObject *kwnames); -uint32_t _PyFunction_GetVersionForCurrentState(PyFunctionObject *func); - /* Macros for direct access to these values. Type checks are *not* done, so use with care. */ #define PyFunction_GET_CODE(func) \ diff --git a/Include/cpython/genobject.h b/Include/cpython/genobject.h index 838ca6cd24691..b485ac6183e2e 100644 --- a/Include/cpython/genobject.h +++ b/Include/cpython/genobject.h @@ -45,7 +45,6 @@ PyAPI_FUNC(PyObject *) PyGen_NewWithQualName(PyFrameObject *, PyObject *name, PyObject *qualname); PyAPI_FUNC(int) _PyGen_SetStopIterationValue(PyObject *); PyAPI_FUNC(int) _PyGen_FetchStopIterationValue(PyObject **); -PyObject *_PyGen_yf(PyGenObject *); PyAPI_FUNC(void) _PyGen_Finalize(PyObject *self); @@ -59,7 +58,6 @@ PyAPI_DATA(PyTypeObject) PyCoro_Type; PyAPI_DATA(PyTypeObject) _PyCoroWrapper_Type; #define PyCoro_CheckExact(op) Py_IS_TYPE(op, &PyCoro_Type) -PyObject *_PyCoro_GetAwaitableIter(PyObject *o); PyAPI_FUNC(PyObject *) PyCoro_New(PyFrameObject *, PyObject *name, PyObject *qualname); @@ -80,8 +78,6 @@ PyAPI_FUNC(PyObject *) PyAsyncGen_New(PyFrameObject *, #define PyAsyncGen_CheckExact(op) Py_IS_TYPE(op, &PyAsyncGen_Type) -PyObject *_PyAsyncGenValueWrapperNew(PyObject *); - #undef _PyGenObject_HEAD diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h index 09d41222b61fc..f8b8e001025af 100644 --- a/Include/internal/pycore_frame.h +++ b/Include/internal/pycore_frame.h @@ -19,6 +19,8 @@ struct _frame { PyObject *_f_frame_data[1]; }; +extern PyFrameObject* _PyFrame_New_NoTrack(PyCodeObject *code); + /* runtime lifecycle */ extern void _PyFrame_Fini(PyInterpreterState *interp); diff --git a/Include/internal/pycore_function.h b/Include/internal/pycore_function.h index dc4422df3eb53..1c87aa31ddb61 100644 --- a/Include/internal/pycore_function.h +++ b/Include/internal/pycore_function.h @@ -1,11 +1,18 @@ #ifndef Py_INTERNAL_FUNCTION_H #define Py_INTERNAL_FUNCTION_H +#ifdef __cplusplus +extern "C" { +#endif +#ifndef Py_BUILD_CORE +# error "this header requires Py_BUILD_CORE define" +#endif -#include "Python.h" - -PyFunctionObject * -_PyFunction_FromConstructor(PyFrameConstructor *constr); +extern PyFunctionObject* _PyFunction_FromConstructor(PyFrameConstructor *constr); +extern uint32_t _PyFunction_GetVersionForCurrentState(PyFunctionObject *func); +#ifdef __cplusplus +} +#endif #endif /* !Py_INTERNAL_FUNCTION_H */ diff --git a/Include/internal/pycore_genobject.h b/Include/internal/pycore_genobject.h index 74a676df4ad53..42db0d87d1f40 100644 --- a/Include/internal/pycore_genobject.h +++ b/Include/internal/pycore_genobject.h @@ -8,6 +8,9 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +extern PyObject *_PyGen_yf(PyGenObject *); +extern PyObject *_PyCoro_GetAwaitableIter(PyObject *o); +extern PyObject *_PyAsyncGenValueWrapperNew(PyObject *); /* runtime lifecycle */ diff --git a/Python/specialize.c b/Python/specialize.c index e1db12b853c7c..1624f1955d42a 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -1,6 +1,7 @@ #include "Python.h" #include "pycore_code.h" #include "pycore_dict.h" +#include "pycore_function.h" // _PyFunction_GetVersionForCurrentState() #include "pycore_global_strings.h" // _Py_ID() #include "pycore_long.h" #include "pycore_moduleobject.h" @@ -1928,7 +1929,7 @@ void _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, int oparg) { - assert(_PyOpcode_InlineCacheEntries[BINARY_OP] == + assert(_PyOpcode_InlineCacheEntries[BINARY_OP] == INLINE_CACHE_ENTRIES_BINARY_OP); _PyBinaryOpCache *cache = (_PyBinaryOpCache *)(instr + 1); switch (oparg) { From webhook-mailer at python.org Fri Feb 25 10:13:54 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 15:13:54 -0000 Subject: [Python-checkins] bpo-46852: Rename float.__set_format__() to float.__setformat__() (GH-31558) (GH-31581) Message-ID: https://github.com/python/cpython/commit/a549cd1fc55888e2e287714b25e2cb2251913909 commit: a549cd1fc55888e2e287714b25e2cb2251913909 branch: 3.9 author: Victor Stinner committer: vstinner date: 2022-02-25T16:13:34+01:00 summary: bpo-46852: Rename float.__set_format__() to float.__setformat__() (GH-31558) (GH-31581) Rename the private undocumented float.__set_format__() method to float.__setformat__() to fix a typo introduced in Python 3.7. The method is only used by test_float. The change enables again test_float tests on the float format which were previously skipped because of the typo. The typo was introduced in Python 3.7 by bpo-20185 in commit b5c51d3dd95bbfde533655fb86ac0f96f771ba7b. (cherry picked from commit 7d03c8be5af2f1559dbc35b775b3116dfd63cfb6) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst M Objects/clinic/floatobject.c.h M Objects/floatobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst new file mode 100644 index 0000000000000..65b826473b915 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-02-01-42.bpo-46852._3zg8D.rst @@ -0,0 +1,3 @@ +Rename the private undocumented ``float.__set_format__()`` method to +``float.__setformat__()`` to fix a typo introduced in Python 3.7. The method +is only used by test_float. Patch by Victor Stinner. diff --git a/Objects/clinic/floatobject.c.h b/Objects/clinic/floatobject.c.h index b7554832b5a8a..6ecdd9e66e95f 100644 --- a/Objects/clinic/floatobject.c.h +++ b/Objects/clinic/floatobject.c.h @@ -289,8 +289,8 @@ float___getformat__(PyTypeObject *type, PyObject *arg) return return_value; } -PyDoc_STRVAR(float___set_format____doc__, -"__set_format__($type, typestr, fmt, /)\n" +PyDoc_STRVAR(float___setformat____doc__, +"__setformat__($type, typestr, fmt, /)\n" "--\n" "\n" "You probably don\'t want to use this function.\n" @@ -307,25 +307,25 @@ PyDoc_STRVAR(float___set_format____doc__, "Override the automatic determination of C-level floating point type.\n" "This affects how floats are converted to and from binary strings."); -#define FLOAT___SET_FORMAT___METHODDEF \ - {"__set_format__", (PyCFunction)(void(*)(void))float___set_format__, METH_FASTCALL|METH_CLASS, float___set_format____doc__}, +#define FLOAT___SETFORMAT___METHODDEF \ + {"__setformat__", (PyCFunction)(void(*)(void))float___setformat__, METH_FASTCALL|METH_CLASS, float___setformat____doc__}, static PyObject * -float___set_format___impl(PyTypeObject *type, const char *typestr, - const char *fmt); +float___setformat___impl(PyTypeObject *type, const char *typestr, + const char *fmt); static PyObject * -float___set_format__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs) +float___setformat__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; const char *typestr; const char *fmt; - if (!_PyArg_CheckPositional("__set_format__", nargs, 2, 2)) { + if (!_PyArg_CheckPositional("__setformat__", nargs, 2, 2)) { goto exit; } if (!PyUnicode_Check(args[0])) { - _PyArg_BadArgument("__set_format__", "argument 1", "str", args[0]); + _PyArg_BadArgument("__setformat__", "argument 1", "str", args[0]); goto exit; } Py_ssize_t typestr_length; @@ -338,7 +338,7 @@ float___set_format__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs goto exit; } if (!PyUnicode_Check(args[1])) { - _PyArg_BadArgument("__set_format__", "argument 2", "str", args[1]); + _PyArg_BadArgument("__setformat__", "argument 2", "str", args[1]); goto exit; } Py_ssize_t fmt_length; @@ -350,7 +350,7 @@ float___set_format__(PyTypeObject *type, PyObject *const *args, Py_ssize_t nargs PyErr_SetString(PyExc_ValueError, "embedded null character"); goto exit; } - return_value = float___set_format___impl(type, typestr, fmt); + return_value = float___setformat___impl(type, typestr, fmt); exit: return return_value; @@ -387,4 +387,4 @@ float___format__(PyObject *self, PyObject *arg) exit: return return_value; } -/*[clinic end generated code: output=25fbbe253f44e2df input=a9049054013a1b77]*/ +/*[clinic end generated code: output=a6af179ec5f83fba input=a9049054013a1b77]*/ diff --git a/Objects/floatobject.c b/Objects/floatobject.c index 8538a051b1982..6ac6127ae5145 100644 --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -1702,7 +1702,7 @@ float___getformat___impl(PyTypeObject *type, const char *typestr) /*[clinic input] @classmethod -float.__set_format__ +float.__setformat__ typestr: str Must be 'double' or 'float'. @@ -1721,9 +1721,9 @@ This affects how floats are converted to and from binary strings. [clinic start generated code]*/ static PyObject * -float___set_format___impl(PyTypeObject *type, const char *typestr, - const char *fmt) -/*[clinic end generated code: output=504460f5dc85acbd input=5306fa2b81a997e4]*/ +float___setformat___impl(PyTypeObject *type, const char *typestr, + const char *fmt) +/*[clinic end generated code: output=06864de1fb5f1f04 input=c0e9e04dd87f9988]*/ { float_format_type f; float_format_type detected; @@ -1825,7 +1825,7 @@ static PyMethodDef float_methods[] = { FLOAT_IS_INTEGER_METHODDEF FLOAT___GETNEWARGS___METHODDEF FLOAT___GETFORMAT___METHODDEF - FLOAT___SET_FORMAT___METHODDEF + FLOAT___SETFORMAT___METHODDEF FLOAT___FORMAT___METHODDEF {NULL, NULL} /* sentinel */ }; From webhook-mailer at python.org Fri Feb 25 10:16:07 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 25 Feb 2022 15:16:07 -0000 Subject: [Python-checkins] Update dict/OrderedDict differences with code equivalents. (GH-31563) Message-ID: https://github.com/python/cpython/commit/8b37a0c522a11bd4a1ae66a132fa6c2789c65213 commit: 8b37a0c522a11bd4a1ae66a132fa6c2789c65213 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-25T07:15:58-08:00 summary: Update dict/OrderedDict differences with code equivalents. (GH-31563) (cherry picked from commit 26aba295a9c1bcb0812fe44bd7e68ddd1d8a6828) Co-authored-by: Raymond Hettinger files: M Doc/library/collections.rst diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index c45d90cbbc1c7..67b64ddda7a2c 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -1092,18 +1092,35 @@ Some differences from :class:`dict` still remain: Space efficiency, iteration speed, and the performance of update operations were secondary. -* Algorithmically, :class:`OrderedDict` can handle frequent reordering - operations better than :class:`dict`. This makes it suitable for tracking - recent accesses (for example in an `LRU cache - `_). +* The :class:`OrderedDict` algorithm can handle frequent reordering operations + better than :class:`dict`. As shown in the recipes below, this makes it + suitable for implementing various kinds of LRU caches. * The equality operation for :class:`OrderedDict` checks for matching order. + A regular :class:`dict` can emulate the order sensitive equality test with + ``p == q and all(k1 == k2 for k1, k2 in zip(p, q))``. + * The :meth:`popitem` method of :class:`OrderedDict` has a different signature. It accepts an optional argument to specify which item is popped. -* :class:`OrderedDict` has a :meth:`move_to_end` method to - efficiently reposition an element to an endpoint. + A regular :class:`dict` can emulate OrderedDict's ``od.popitem(last=True)`` + with ``d.popitem()`` which is guaranteed to pop the rightmost (last) item. + + A regular :class:`dict` can emulate OrderedDict's ``od.popitem(last=False)`` + with ``(k := next(iter(d)), d.pop(k))`` which will return and remove the + leftmost (first) item if it exists. + +* :class:`OrderedDict` has a :meth:`move_to_end` method to efficiently + reposition an element to an endpoint. + + A regular :class:`dict` can emulate OrderedDict's ``od.move_to_end(k, + last=True)`` with ``d[k] = d.pop(k)`` which will move the key and its + associated value to the rightmost (last) position. + + A regular :class:`dict` does not have an efficient equivalent for + OrderedDict's ``od.move_to_end(k, last=False)`` which moves the key + and its associated value to the leftmost (first) position. * Until Python 3.8, :class:`dict` lacked a :meth:`__reversed__` method. From webhook-mailer at python.org Fri Feb 25 10:19:35 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 15:19:35 -0000 Subject: [Python-checkins] bpo-45316: Move _PyArg_Fini() to internal C API (GH-31580) Message-ID: https://github.com/python/cpython/commit/f780d9690f1a009a56ac0c653ec9608e6b2aeff4 commit: f780d9690f1a009a56ac0c653ec9608e6b2aeff4 branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T16:19:30+01:00 summary: bpo-45316: Move _PyArg_Fini() to internal C API (GH-31580) Move the private unexported _PyArg_Fini() function to the internal C API: to the pycore_pylifecycle.h header file. files: M Include/internal/pycore_pylifecycle.h M Include/modsupport.h M Python/getargs.c M Python/pylifecycle.c diff --git a/Include/internal/pycore_pylifecycle.h b/Include/internal/pycore_pylifecycle.h index deac6ee3d3f41..00d13b85d2c25 100644 --- a/Include/internal/pycore_pylifecycle.h +++ b/Include/internal/pycore_pylifecycle.h @@ -84,6 +84,7 @@ extern void _PyAST_Fini(PyInterpreterState *interp); extern void _PyAtExit_Fini(PyInterpreterState *interp); extern void _PyThread_FiniType(PyInterpreterState *interp); extern void _Py_Deepfreeze_Fini(void); +extern void _PyArg_Fini(void); extern PyStatus _PyGILState_Init(_PyRuntimeState *runtime); extern PyStatus _PyGILState_SetTstate(PyThreadState *tstate); diff --git a/Include/modsupport.h b/Include/modsupport.h index 6cc2ad01ebe96..3cfefb413ca55 100644 --- a/Include/modsupport.h +++ b/Include/modsupport.h @@ -141,8 +141,6 @@ PyAPI_FUNC(PyObject * const *) _PyArg_UnpackKeywordsWithVararg( (minpos) <= (nargs) && (nargs) <= (maxpos) && args != NULL) ? (args) : \ _PyArg_UnpackKeywords((args), (nargs), (kwargs), (kwnames), (parser), \ (minpos), (maxpos), (minkw), (buf))) - -void _PyArg_Fini(void); #endif /* Py_LIMITED_API */ // Add an attribute with name 'name' and value 'obj' to the module 'mod. diff --git a/Python/getargs.c b/Python/getargs.c index 3fab3b5efefc2..a2ea4d7da45f8 100644 --- a/Python/getargs.c +++ b/Python/getargs.c @@ -3,6 +3,7 @@ #include "Python.h" #include "pycore_tuple.h" // _PyTuple_ITEMS() +#include "pycore_pylifecycle.h" // _PyArg_Fini #include #include diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 9dd769a0e87b1..a671bcaf42c66 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -829,7 +829,7 @@ pycore_interp_init(PyThreadState *tstate) // Intern strings in deep-frozen modules first so that others // can use it instead of creating a heap allocated string. _Py_Deepfreeze_Init(); - + status = pycore_init_types(interp); if (_PyStatus_EXCEPTION(status)) { goto done; From webhook-mailer at python.org Fri Feb 25 10:22:06 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 15:22:06 -0000 Subject: [Python-checkins] bpo-46836: Rename InterpreterFrame to _PyInterpreterFrame (GH-31583) Message-ID: https://github.com/python/cpython/commit/87af12bff33b3e7546fa26158b7d8680ecb6ecec commit: 87af12bff33b3e7546fa26158b7d8680ecb6ecec branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T16:22:00+01:00 summary: bpo-46836: Rename InterpreterFrame to _PyInterpreterFrame (GH-31583) Rename also struct _interpreter_frame to struct _PyInterpreterFrame. Reduce risk of name conflicts if a project includes pycore_frame.h. files: M Include/cpython/ceval.h M Include/cpython/pystate.h M Include/internal/pycore_ceval.h M Include/internal/pycore_frame.h M Modules/_tracemalloc.c M Modules/_xxsubinterpretersmodule.c M Modules/signalmodule.c M Objects/frameobject.c M Objects/genobject.c M Objects/typeobject.c M Python/ceval.c M Python/frame.c M Python/pystate.c M Python/sysmodule.c M Python/traceback.c diff --git a/Include/cpython/ceval.h b/Include/cpython/ceval.h index aedc7367a18f6..5a904bd3f08e9 100644 --- a/Include/cpython/ceval.h +++ b/Include/cpython/ceval.h @@ -22,7 +22,7 @@ PyAPI_FUNC(PyObject *) _PyEval_GetBuiltinId(_Py_Identifier *); flag was set, else return 0. */ PyAPI_FUNC(int) PyEval_MergeCompilerFlags(PyCompilerFlags *cf); -PyAPI_FUNC(PyObject *) _PyEval_EvalFrameDefault(PyThreadState *tstate, struct _interpreter_frame *f, int exc); +PyAPI_FUNC(PyObject *) _PyEval_EvalFrameDefault(PyThreadState *tstate, struct _PyInterpreterFrame *f, int exc); PyAPI_FUNC(void) _PyEval_SetSwitchInterval(unsigned long microseconds); PyAPI_FUNC(unsigned long) _PyEval_GetSwitchInterval(void); diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index 248320cceb716..0d38604636b2b 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -46,7 +46,7 @@ typedef struct _cframe { */ int use_tracing; /* Pointer to the currently executing frame (it can be NULL) */ - struct _interpreter_frame *current_frame; + struct _PyInterpreterFrame *current_frame; struct _cframe *previous; } CFrame; @@ -258,7 +258,7 @@ PyAPI_FUNC(void) PyThreadState_DeleteCurrent(void); /* Frame evaluation API */ -typedef PyObject* (*_PyFrameEvalFunction)(PyThreadState *tstate, struct _interpreter_frame *, int); +typedef PyObject* (*_PyFrameEvalFunction)(PyThreadState *tstate, struct _PyInterpreterFrame *, int); PyAPI_FUNC(_PyFrameEvalFunction) _PyInterpreterState_GetEvalFrameFunc( PyInterpreterState *interp); diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h index 53d0b5c4549bc..70178e38650cf 100644 --- a/Include/internal/pycore_ceval.h +++ b/Include/internal/pycore_ceval.h @@ -47,7 +47,7 @@ extern PyObject *_PyEval_BuiltinsFromGlobals( static inline PyObject* -_PyEval_EvalFrame(PyThreadState *tstate, struct _interpreter_frame *frame, int throwflag) +_PyEval_EvalFrame(PyThreadState *tstate, struct _PyInterpreterFrame *frame, int throwflag) { if (tstate->interp->eval_frame == NULL) { return _PyEval_EvalFrameDefault(tstate, frame, throwflag); @@ -116,7 +116,7 @@ static inline void _Py_LeaveRecursiveCall_inline(void) { #define Py_LeaveRecursiveCall() _Py_LeaveRecursiveCall_inline() -struct _interpreter_frame *_PyEval_GetFrame(void); +struct _PyInterpreterFrame *_PyEval_GetFrame(void); PyObject *_Py_MakeCoro(PyFunctionObject *func); diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h index f8b8e001025af..20e81b7849b20 100644 --- a/Include/internal/pycore_frame.h +++ b/Include/internal/pycore_frame.h @@ -9,7 +9,7 @@ extern "C" { struct _frame { PyObject_HEAD PyFrameObject *f_back; /* previous frame, or NULL */ - struct _interpreter_frame *f_frame; /* points to the frame data */ + struct _PyInterpreterFrame *f_frame; /* points to the frame data */ PyObject *f_trace; /* Trace function */ int f_lineno; /* Current line number. Only valid if non-zero */ char f_trace_lines; /* Emit per-line trace events? */ @@ -48,63 +48,63 @@ typedef signed char PyFrameState; unless it's -1 in which case next_instr should be first_instr. */ -typedef struct _interpreter_frame { +typedef struct _PyInterpreterFrame { PyFunctionObject *f_func; /* Strong reference */ PyObject *f_globals; /* Borrowed reference */ PyObject *f_builtins; /* Borrowed reference */ PyObject *f_locals; /* Strong reference, may be NULL */ PyCodeObject *f_code; /* Strong reference */ PyFrameObject *frame_obj; /* Strong reference, may be NULL */ - struct _interpreter_frame *previous; + struct _PyInterpreterFrame *previous; int f_lasti; /* Last instruction if called */ int stacktop; /* Offset of TOS from localsplus */ PyFrameState f_state; /* What state the frame is in */ bool is_entry; // Whether this is the "root" frame for the current CFrame. bool is_generator; PyObject *localsplus[1]; -} InterpreterFrame; +} _PyInterpreterFrame; -static inline int _PyFrame_IsRunnable(InterpreterFrame *f) { +static inline int _PyFrame_IsRunnable(_PyInterpreterFrame *f) { return f->f_state < FRAME_EXECUTING; } -static inline int _PyFrame_IsExecuting(InterpreterFrame *f) { +static inline int _PyFrame_IsExecuting(_PyInterpreterFrame *f) { return f->f_state == FRAME_EXECUTING; } -static inline int _PyFrameHasCompleted(InterpreterFrame *f) { +static inline int _PyFrameHasCompleted(_PyInterpreterFrame *f) { return f->f_state > FRAME_EXECUTING; } -static inline PyObject **_PyFrame_Stackbase(InterpreterFrame *f) { +static inline PyObject **_PyFrame_Stackbase(_PyInterpreterFrame *f) { return f->localsplus + f->f_code->co_nlocalsplus; } -static inline PyObject *_PyFrame_StackPeek(InterpreterFrame *f) { +static inline PyObject *_PyFrame_StackPeek(_PyInterpreterFrame *f) { assert(f->stacktop > f->f_code->co_nlocalsplus); assert(f->localsplus[f->stacktop-1] != NULL); return f->localsplus[f->stacktop-1]; } -static inline PyObject *_PyFrame_StackPop(InterpreterFrame *f) { +static inline PyObject *_PyFrame_StackPop(_PyInterpreterFrame *f) { assert(f->stacktop > f->f_code->co_nlocalsplus); f->stacktop--; return f->localsplus[f->stacktop]; } -static inline void _PyFrame_StackPush(InterpreterFrame *f, PyObject *value) { +static inline void _PyFrame_StackPush(_PyInterpreterFrame *f, PyObject *value) { f->localsplus[f->stacktop] = value; f->stacktop++; } -#define FRAME_SPECIALS_SIZE ((sizeof(InterpreterFrame)-1)/sizeof(PyObject *)) +#define FRAME_SPECIALS_SIZE ((sizeof(_PyInterpreterFrame)-1)/sizeof(PyObject *)) -void _PyFrame_Copy(InterpreterFrame *src, InterpreterFrame *dest); +void _PyFrame_Copy(_PyInterpreterFrame *src, _PyInterpreterFrame *dest); /* Consumes reference to func */ static inline void _PyFrame_InitializeSpecials( - InterpreterFrame *frame, PyFunctionObject *func, + _PyInterpreterFrame *frame, PyFunctionObject *func, PyObject *locals, int nlocalsplus) { frame->f_func = func; @@ -124,19 +124,19 @@ _PyFrame_InitializeSpecials( * that precedes this frame. */ static inline PyObject** -_PyFrame_GetLocalsArray(InterpreterFrame *frame) +_PyFrame_GetLocalsArray(_PyInterpreterFrame *frame) { return frame->localsplus; } static inline PyObject** -_PyFrame_GetStackPointer(InterpreterFrame *frame) +_PyFrame_GetStackPointer(_PyInterpreterFrame *frame) { return frame->localsplus+frame->stacktop; } static inline void -_PyFrame_SetStackPointer(InterpreterFrame *frame, PyObject **stack_pointer) +_PyFrame_SetStackPointer(_PyInterpreterFrame *frame, PyObject **stack_pointer) { frame->stacktop = (int)(stack_pointer - frame->localsplus); } @@ -144,13 +144,13 @@ _PyFrame_SetStackPointer(InterpreterFrame *frame, PyObject **stack_pointer) /* For use by _PyFrame_GetFrameObject Do not call directly. */ PyFrameObject * -_PyFrame_MakeAndSetFrameObject(InterpreterFrame *frame); +_PyFrame_MakeAndSetFrameObject(_PyInterpreterFrame *frame); /* Gets the PyFrameObject for this frame, lazily * creating it if necessary. * Returns a borrowed referennce */ static inline PyFrameObject * -_PyFrame_GetFrameObject(InterpreterFrame *frame) +_PyFrame_GetFrameObject(_PyInterpreterFrame *frame) { PyFrameObject *res = frame->frame_obj; if (res != NULL) { @@ -160,7 +160,7 @@ _PyFrame_GetFrameObject(InterpreterFrame *frame) } /* Clears all references in the frame. - * If take is non-zero, then the InterpreterFrame frame + * If take is non-zero, then the _PyInterpreterFrame frame * may be transferred to the frame object it references * instead of being cleared. Either way * the caller no longer owns the references @@ -169,21 +169,21 @@ _PyFrame_GetFrameObject(InterpreterFrame *frame) * frames like the ones in generators and coroutines. */ void -_PyFrame_Clear(InterpreterFrame * frame); +_PyFrame_Clear(_PyInterpreterFrame * frame); int -_PyFrame_Traverse(InterpreterFrame *frame, visitproc visit, void *arg); +_PyFrame_Traverse(_PyInterpreterFrame *frame, visitproc visit, void *arg); int -_PyFrame_FastToLocalsWithError(InterpreterFrame *frame); +_PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame); void -_PyFrame_LocalsToFast(InterpreterFrame *frame, int clear); +_PyFrame_LocalsToFast(_PyInterpreterFrame *frame, int clear); -extern InterpreterFrame * +extern _PyInterpreterFrame * _PyThreadState_BumpFramePointerSlow(PyThreadState *tstate, size_t size); -static inline InterpreterFrame * +static inline _PyInterpreterFrame * _PyThreadState_BumpFramePointer(PyThreadState *tstate, size_t size) { PyObject **base = tstate->datastack_top; @@ -192,16 +192,16 @@ _PyThreadState_BumpFramePointer(PyThreadState *tstate, size_t size) assert(tstate->datastack_limit); if (top < tstate->datastack_limit) { tstate->datastack_top = top; - return (InterpreterFrame *)base; + return (_PyInterpreterFrame *)base; } } return _PyThreadState_BumpFramePointerSlow(tstate, size); } -void _PyThreadState_PopFrame(PyThreadState *tstate, InterpreterFrame *frame); +void _PyThreadState_PopFrame(PyThreadState *tstate, _PyInterpreterFrame *frame); /* Consume reference to func */ -InterpreterFrame * +_PyInterpreterFrame * _PyFrame_Push(PyThreadState *tstate, PyFunctionObject *func); #ifdef __cplusplus diff --git a/Modules/_tracemalloc.c b/Modules/_tracemalloc.c index 53e329989abf2..738d54530c967 100644 --- a/Modules/_tracemalloc.c +++ b/Modules/_tracemalloc.c @@ -305,7 +305,7 @@ hashtable_compare_traceback(const void *key1, const void *key2) static void -tracemalloc_get_frame(InterpreterFrame *pyframe, frame_t *frame) +tracemalloc_get_frame(_PyInterpreterFrame *pyframe, frame_t *frame) { frame->filename = &_Py_STR(anon_unknown); int lineno = PyCode_Addr2Line(pyframe->f_code, pyframe->f_lasti*sizeof(_Py_CODEUNIT)); @@ -399,7 +399,7 @@ traceback_get_frames(traceback_t *traceback) return; } - InterpreterFrame *pyframe = tstate->cframe->current_frame; + _PyInterpreterFrame *pyframe = tstate->cframe->current_frame; for (; pyframe != NULL;) { if (traceback->nframe < _Py_tracemalloc_config.max_nframe) { tracemalloc_get_frame(pyframe, &traceback->frames[traceback->nframe]); @@ -410,7 +410,7 @@ traceback_get_frames(traceback_t *traceback) traceback->total_nframe++; } - InterpreterFrame *back = pyframe->previous; + _PyInterpreterFrame *back = pyframe->previous; pyframe = back; } } diff --git a/Modules/_xxsubinterpretersmodule.c b/Modules/_xxsubinterpretersmodule.c index 1507abf311430..846b24d5efa9a 100644 --- a/Modules/_xxsubinterpretersmodule.c +++ b/Modules/_xxsubinterpretersmodule.c @@ -1839,7 +1839,7 @@ _is_running(PyInterpreterState *interp) } assert(!PyErr_Occurred()); - InterpreterFrame *frame = tstate->cframe->current_frame; + _PyInterpreterFrame *frame = tstate->cframe->current_frame; if (frame == NULL) { return 0; } diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c index 4d2ca5776411e..26a1f48470ed2 100644 --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -8,7 +8,7 @@ #include "pycore_call.h" // _PyObject_Call() #include "pycore_ceval.h" // _PyEval_SignalReceived() #include "pycore_fileutils.h" // _Py_BEGIN_SUPPRESS_IPH -#include "pycore_frame.h" // InterpreterFrame +#include "pycore_frame.h" // _PyInterpreterFrame #include "pycore_moduleobject.h" // _PyModule_GetState() #include "pycore_pyerrors.h" // _PyErr_SetString() #include "pycore_pylifecycle.h" // NSIG @@ -1809,7 +1809,7 @@ _PyErr_CheckSignalsTstate(PyThreadState *tstate) */ _Py_atomic_store(&is_tripped, 0); - InterpreterFrame *frame = tstate->cframe->current_frame; + _PyInterpreterFrame *frame = tstate->cframe->current_frame; signal_state_t *state = &signal_global_state; for (int i = 1; i < NSIG; i++) { if (!_Py_atomic_load_relaxed(&Handlers[i].tripped)) { diff --git a/Objects/frameobject.c b/Objects/frameobject.c index c9445e3bb36da..eb7fdb30cd75e 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -629,8 +629,8 @@ frame_dealloc(PyFrameObject *f) /* Kill all local variables including specials, if we own them */ if (f->f_owns_frame) { f->f_owns_frame = 0; - assert(f->f_frame == (InterpreterFrame *)f->_f_frame_data); - InterpreterFrame *frame = (InterpreterFrame *)f->_f_frame_data; + assert(f->f_frame == (_PyInterpreterFrame *)f->_f_frame_data); + _PyInterpreterFrame *frame = (_PyInterpreterFrame *)f->_f_frame_data; /* Don't clear code object until the end */ co = frame->f_code; frame->f_code = NULL; @@ -707,7 +707,7 @@ static PyObject * frame_sizeof(PyFrameObject *f, PyObject *Py_UNUSED(ignored)) { Py_ssize_t res; - res = offsetof(PyFrameObject, _f_frame_data) + offsetof(InterpreterFrame, localsplus); + res = offsetof(PyFrameObject, _f_frame_data) + offsetof(_PyInterpreterFrame, localsplus); PyCodeObject *code = f->f_frame->f_code; res += (code->co_nlocalsplus+code->co_stacksize) * sizeof(PyObject *); return PyLong_FromSsize_t(res); @@ -738,7 +738,7 @@ PyTypeObject PyFrame_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "frame", offsetof(PyFrameObject, _f_frame_data) + - offsetof(InterpreterFrame, localsplus), + offsetof(_PyInterpreterFrame, localsplus), sizeof(PyObject *), (destructor)frame_dealloc, /* tp_dealloc */ 0, /* tp_vectorcall_offset */ @@ -771,7 +771,7 @@ PyTypeObject PyFrame_Type = { }; static void -init_frame(InterpreterFrame *frame, PyFunctionObject *func, PyObject *locals) +init_frame(_PyInterpreterFrame *frame, PyFunctionObject *func, PyObject *locals) { /* _PyFrame_InitializeSpecials consumes reference to func */ Py_INCREF(func); @@ -827,8 +827,8 @@ PyFrame_New(PyThreadState *tstate, PyCodeObject *code, Py_DECREF(func); return NULL; } - init_frame((InterpreterFrame *)f->_f_frame_data, func, locals); - f->f_frame = (InterpreterFrame *)f->_f_frame_data; + init_frame((_PyInterpreterFrame *)f->_f_frame_data, func, locals); + f->f_frame = (_PyInterpreterFrame *)f->_f_frame_data; f->f_owns_frame = 1; Py_DECREF(func); _PyObject_GC_TRACK(f); @@ -836,7 +836,7 @@ PyFrame_New(PyThreadState *tstate, PyCodeObject *code, } static int -_PyFrame_OpAlreadyRan(InterpreterFrame *frame, int opcode, int oparg) +_PyFrame_OpAlreadyRan(_PyInterpreterFrame *frame, int opcode, int oparg) { const _Py_CODEUNIT *code = (const _Py_CODEUNIT *)PyBytes_AS_STRING(frame->f_code->co_code); @@ -849,7 +849,7 @@ _PyFrame_OpAlreadyRan(InterpreterFrame *frame, int opcode, int oparg) } int -_PyFrame_FastToLocalsWithError(InterpreterFrame *frame) { +_PyFrame_FastToLocalsWithError(_PyInterpreterFrame *frame) { /* Merge fast locals into f->f_locals */ PyObject *locals; PyObject **fast; @@ -960,7 +960,7 @@ PyFrame_FastToLocals(PyFrameObject *f) } void -_PyFrame_LocalsToFast(InterpreterFrame *frame, int clear) +_PyFrame_LocalsToFast(_PyInterpreterFrame *frame, int clear) { /* Merge locals into fast locals */ PyObject *locals; diff --git a/Objects/genobject.c b/Objects/genobject.c index b34931a95a680..bfa1ea5c45f66 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -7,7 +7,7 @@ #include "pycore_object.h" // _PyObject_GC_UNTRACK() #include "pycore_pyerrors.h" // _PyErr_ClearExcState() #include "pycore_pystate.h" // _PyThreadState_GET() -#include "pycore_frame.h" // InterpreterFrame +#include "pycore_frame.h" // _PyInterpreterFrame #include "frameobject.h" // PyFrameObject #include "structmember.h" // PyMemberDef #include "opcode.h" // SEND @@ -36,7 +36,7 @@ gen_traverse(PyGenObject *gen, visitproc visit, void *arg) Py_VISIT(gen->gi_name); Py_VISIT(gen->gi_qualname); if (gen->gi_frame_valid) { - InterpreterFrame *frame = (InterpreterFrame *)(gen->gi_iframe); + _PyInterpreterFrame *frame = (_PyInterpreterFrame *)(gen->gi_iframe); assert(frame->frame_obj == NULL || frame->frame_obj->f_owns_frame == 0); int err = _PyFrame_Traverse(frame, visit, arg); if (err) { @@ -55,7 +55,7 @@ _PyGen_Finalize(PyObject *self) PyObject *res = NULL; PyObject *error_type, *error_value, *error_traceback; - if (gen->gi_frame_valid == 0 || _PyFrameHasCompleted((InterpreterFrame *)gen->gi_iframe)) { + if (gen->gi_frame_valid == 0 || _PyFrameHasCompleted((_PyInterpreterFrame *)gen->gi_iframe)) { /* Generator isn't paused, so no need to close */ return; } @@ -87,7 +87,7 @@ _PyGen_Finalize(PyObject *self) issue a RuntimeWarning. */ if (gen->gi_code != NULL && ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE && - ((InterpreterFrame *)gen->gi_iframe)->f_state == FRAME_CREATED) + ((_PyInterpreterFrame *)gen->gi_iframe)->f_state == FRAME_CREATED) { _PyErr_WarnUnawaitedCoroutine((PyObject *)gen); } @@ -131,7 +131,7 @@ gen_dealloc(PyGenObject *gen) Py_CLEAR(((PyAsyncGenObject*)gen)->ag_origin_or_finalizer); } if (gen->gi_frame_valid) { - InterpreterFrame *frame = (InterpreterFrame *)gen->gi_iframe; + _PyInterpreterFrame *frame = (_PyInterpreterFrame *)gen->gi_iframe; gen->gi_frame_valid = 0; frame->is_generator = false; frame->previous = NULL; @@ -152,7 +152,7 @@ gen_send_ex2(PyGenObject *gen, PyObject *arg, PyObject **presult, int exc, int closing) { PyThreadState *tstate = _PyThreadState_GET(); - InterpreterFrame *frame = (InterpreterFrame *)gen->gi_iframe; + _PyInterpreterFrame *frame = (_PyInterpreterFrame *)gen->gi_iframe; PyObject *result; *presult = NULL; @@ -348,7 +348,7 @@ _PyGen_yf(PyGenObject *gen) PyObject *yf = NULL; if (gen->gi_frame_valid) { - InterpreterFrame *frame = (InterpreterFrame *)gen->gi_iframe; + _PyInterpreterFrame *frame = (_PyInterpreterFrame *)gen->gi_iframe; PyObject *bytecode = gen->gi_code->co_code; unsigned char *code = (unsigned char *)PyBytes_AS_STRING(bytecode); @@ -377,7 +377,7 @@ gen_close(PyGenObject *gen, PyObject *args) int err = 0; if (yf) { - InterpreterFrame *frame = (InterpreterFrame *)gen->gi_iframe; + _PyInterpreterFrame *frame = (_PyInterpreterFrame *)gen->gi_iframe; PyFrameState state = frame->f_state; frame->f_state = FRAME_EXECUTING; err = gen_close_iter(yf); @@ -418,7 +418,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, PyObject *yf = _PyGen_yf(gen); if (yf) { - InterpreterFrame *frame = (InterpreterFrame *)gen->gi_iframe; + _PyInterpreterFrame *frame = (_PyInterpreterFrame *)gen->gi_iframe; PyObject *ret; int err; if (PyErr_GivenExceptionMatches(typ, PyExc_GeneratorExit) && @@ -445,7 +445,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, will be reported correctly to the user. */ /* XXX We should probably be updating the current frame somewhere in ceval.c. */ - InterpreterFrame *prev = tstate->cframe->current_frame; + _PyInterpreterFrame *prev = tstate->cframe->current_frame; frame->previous = prev; tstate->cframe->current_frame = frame; /* Close the generator that we are currently iterating with @@ -479,7 +479,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, PyObject *val; /* Pop subiterator from stack */ assert(gen->gi_frame_valid); - ret = _PyFrame_StackPop((InterpreterFrame *)gen->gi_iframe); + ret = _PyFrame_StackPop((_PyInterpreterFrame *)gen->gi_iframe); assert(ret == yf); Py_DECREF(ret); /* Termination repetition of SEND loop */ @@ -750,7 +750,7 @@ gen_getrunning(PyGenObject *gen, void *Py_UNUSED(ignored)) if (gen->gi_frame_valid == 0) { Py_RETURN_FALSE; } - return PyBool_FromLong(_PyFrame_IsExecuting((InterpreterFrame *)gen->gi_iframe)); + return PyBool_FromLong(_PyFrame_IsExecuting((_PyInterpreterFrame *)gen->gi_iframe)); } static PyObject * @@ -759,7 +759,7 @@ gen_getsuspended(PyGenObject *gen, void *Py_UNUSED(ignored)) if (gen->gi_frame_valid == 0) { Py_RETURN_FALSE; } - return PyBool_FromLong(((InterpreterFrame *)gen->gi_iframe)->f_state == FRAME_SUSPENDED); + return PyBool_FromLong(((_PyInterpreterFrame *)gen->gi_iframe)->f_state == FRAME_SUSPENDED); } static PyObject * @@ -771,7 +771,7 @@ _gen_getframe(PyGenObject *gen, const char *const name) if (gen->gi_frame_valid == 0) { Py_RETURN_NONE; } - return _Py_XNewRef((PyObject *)_PyFrame_GetFrameObject((InterpreterFrame *)gen->gi_iframe)); + return _Py_XNewRef((PyObject *)_PyFrame_GetFrameObject((_PyInterpreterFrame *)gen->gi_iframe)); } static PyObject * @@ -802,7 +802,7 @@ static PyObject * gen_sizeof(PyGenObject *gen, PyObject *Py_UNUSED(ignored)) { Py_ssize_t res; - res = offsetof(PyGenObject, gi_iframe) + offsetof(InterpreterFrame, localsplus); + res = offsetof(PyGenObject, gi_iframe) + offsetof(_PyInterpreterFrame, localsplus); PyCodeObject *code = gen->gi_code; res += (code->co_nlocalsplus+code->co_stacksize) * sizeof(PyObject *); return PyLong_FromSsize_t(res); @@ -831,7 +831,7 @@ PyTypeObject PyGen_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "generator", /* tp_name */ offsetof(PyGenObject, gi_iframe) + - offsetof(InterpreterFrame, localsplus), /* tp_basicsize */ + offsetof(_PyInterpreterFrame, localsplus), /* tp_basicsize */ sizeof(PyObject *), /* tp_itemsize */ /* methods */ (destructor)gen_dealloc, /* tp_dealloc */ @@ -905,7 +905,7 @@ make_gen(PyTypeObject *type, PyFunctionObject *func) } static PyObject * -compute_cr_origin(int origin_depth, InterpreterFrame *current_frame); +compute_cr_origin(int origin_depth, _PyInterpreterFrame *current_frame); PyObject * _Py_MakeCoro(PyFunctionObject *func) @@ -964,8 +964,8 @@ gen_new_with_qualname(PyTypeObject *type, PyFrameObject *f, /* Copy the frame */ assert(f->f_frame->frame_obj == NULL); assert(f->f_owns_frame); - InterpreterFrame *frame = (InterpreterFrame *)gen->gi_iframe; - _PyFrame_Copy((InterpreterFrame *)f->_f_frame_data, frame); + _PyInterpreterFrame *frame = (_PyInterpreterFrame *)gen->gi_iframe; + _PyFrame_Copy((_PyInterpreterFrame *)f->_f_frame_data, frame); gen->gi_frame_valid = 1; assert(frame->frame_obj == f); f->f_owns_frame = 0; @@ -1108,7 +1108,7 @@ cr_getsuspended(PyCoroObject *coro, void *Py_UNUSED(ignored)) if (coro->cr_frame_valid == 0) { Py_RETURN_FALSE; } - return PyBool_FromLong(((InterpreterFrame *)coro->cr_iframe)->f_state == FRAME_SUSPENDED); + return PyBool_FromLong(((_PyInterpreterFrame *)coro->cr_iframe)->f_state == FRAME_SUSPENDED); } static PyObject * @@ -1117,7 +1117,7 @@ cr_getrunning(PyCoroObject *coro, void *Py_UNUSED(ignored)) if (coro->cr_frame_valid == 0) { Py_RETURN_FALSE; } - return PyBool_FromLong(_PyFrame_IsExecuting((InterpreterFrame *)coro->cr_iframe)); + return PyBool_FromLong(_PyFrame_IsExecuting((_PyInterpreterFrame *)coro->cr_iframe)); } static PyObject * @@ -1176,7 +1176,7 @@ PyTypeObject PyCoro_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "coroutine", /* tp_name */ offsetof(PyCoroObject, cr_iframe) + - offsetof(InterpreterFrame, localsplus), /* tp_basicsize */ + offsetof(_PyInterpreterFrame, localsplus), /* tp_basicsize */ sizeof(PyObject *), /* tp_itemsize */ /* methods */ (destructor)gen_dealloc, /* tp_dealloc */ @@ -1314,9 +1314,9 @@ PyTypeObject _PyCoroWrapper_Type = { }; static PyObject * -compute_cr_origin(int origin_depth, InterpreterFrame *current_frame) +compute_cr_origin(int origin_depth, _PyInterpreterFrame *current_frame) { - InterpreterFrame *frame = current_frame; + _PyInterpreterFrame *frame = current_frame; /* First count how many frames we have */ int frame_count = 0; for (; frame && frame_count < origin_depth; ++frame_count) { @@ -1567,7 +1567,7 @@ PyTypeObject PyAsyncGen_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "async_generator", /* tp_name */ offsetof(PyAsyncGenObject, ag_iframe) + - offsetof(InterpreterFrame, localsplus), /* tp_basicsize */ + offsetof(_PyInterpreterFrame, localsplus), /* tp_basicsize */ sizeof(PyObject *), /* tp_itemsize */ /* methods */ (destructor)gen_dealloc, /* tp_dealloc */ @@ -2053,7 +2053,7 @@ static PyObject * async_gen_athrow_send(PyAsyncGenAThrow *o, PyObject *arg) { PyGenObject *gen = (PyGenObject*)o->agt_gen; - InterpreterFrame *frame = (InterpreterFrame *)gen->gi_iframe; + _PyInterpreterFrame *frame = (_PyInterpreterFrame *)gen->gi_iframe; PyObject *retval; if (o->agt_state == AWAITABLE_STATE_CLOSED) { diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 286261226fa8e..1dfeac3b9e660 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -12,7 +12,7 @@ #include "pycore_typeobject.h" // struct type_cache #include "pycore_unionobject.h" // _Py_union_type_or #include "frameobject.h" // PyFrameObject -#include "pycore_frame.h" // InterpreterFrame +#include "pycore_frame.h" // _PyInterpreterFrame #include "opcode.h" // MAKE_CELL #include "structmember.h" // PyMemberDef @@ -8933,7 +8933,7 @@ super_descr_get(PyObject *self, PyObject *obj, PyObject *type) } static int -super_init_without_args(InterpreterFrame *cframe, PyCodeObject *co, +super_init_without_args(_PyInterpreterFrame *cframe, PyCodeObject *co, PyTypeObject **type_p, PyObject **obj_p) { if (co->co_argcount == 0) { @@ -9017,7 +9017,7 @@ super_init(PyObject *self, PyObject *args, PyObject *kwds) /* Call super(), without args -- fill in from __class__ and first local variable on the stack. */ PyThreadState *tstate = _PyThreadState_GET(); - InterpreterFrame *cframe = tstate->cframe->current_frame; + _PyInterpreterFrame *cframe = tstate->cframe->current_frame; if (cframe == NULL) { PyErr_SetString(PyExc_RuntimeError, "super(): no current frame"); diff --git a/Python/ceval.c b/Python/ceval.c index c09914fadcaab..4c0a71b036e8b 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -59,7 +59,7 @@ static PyObject * do_call_core( #ifdef LLTRACE static int lltrace; static int prtrace(PyThreadState *, PyObject *, const char *); -static void lltrace_instruction(InterpreterFrame *frame, int opcode, int oparg) +static void lltrace_instruction(_PyInterpreterFrame *frame, int opcode, int oparg) { if (HAS_ARG(opcode)) { printf("%d: %d, %d\n", @@ -72,20 +72,20 @@ static void lltrace_instruction(InterpreterFrame *frame, int opcode, int oparg) } #endif static int call_trace(Py_tracefunc, PyObject *, - PyThreadState *, InterpreterFrame *, + PyThreadState *, _PyInterpreterFrame *, int, PyObject *); static int call_trace_protected(Py_tracefunc, PyObject *, - PyThreadState *, InterpreterFrame *, + PyThreadState *, _PyInterpreterFrame *, int, PyObject *); static void call_exc_trace(Py_tracefunc, PyObject *, - PyThreadState *, InterpreterFrame *); + PyThreadState *, _PyInterpreterFrame *); static int maybe_call_line_trace(Py_tracefunc, PyObject *, - PyThreadState *, InterpreterFrame *, int); -static void maybe_dtrace_line(InterpreterFrame *, PyTraceInfo *, int); -static void dtrace_function_entry(InterpreterFrame *); -static void dtrace_function_return(InterpreterFrame *); + PyThreadState *, _PyInterpreterFrame *, int); +static void maybe_dtrace_line(_PyInterpreterFrame *, PyTraceInfo *, int); +static void dtrace_function_entry(_PyInterpreterFrame *); +static void dtrace_function_return(_PyInterpreterFrame *); -static PyObject * import_name(PyThreadState *, InterpreterFrame *, +static PyObject * import_name(PyThreadState *, _PyInterpreterFrame *, PyObject *, PyObject *, PyObject *); static PyObject * import_from(PyThreadState *, PyObject *, PyObject *); static int import_all_from(PyThreadState *, PyObject *, PyObject *); @@ -97,12 +97,12 @@ static int check_except_star_type_valid(PyThreadState *tstate, PyObject* right); static void format_kwargs_error(PyThreadState *, PyObject *func, PyObject *kwargs); static void format_awaitable_error(PyThreadState *, PyTypeObject *, int, int); static int get_exception_handler(PyCodeObject *, int, int*, int*, int*); -static InterpreterFrame * +static _PyInterpreterFrame * _PyEvalFramePushAndInit(PyThreadState *tstate, PyFunctionObject *func, PyObject *locals, PyObject* const* args, size_t argcount, PyObject *kwnames); static void -_PyEvalFrameClearAndPop(PyThreadState *tstate, InterpreterFrame *frame); +_PyEvalFrameClearAndPop(PyThreadState *tstate, _PyInterpreterFrame *frame); #define NAME_ERROR_MSG \ "name '%.200s' is not defined" @@ -1509,7 +1509,7 @@ eval_frame_handle_pending(PyThreadState *tstate) static int -trace_function_entry(PyThreadState *tstate, InterpreterFrame *frame) +trace_function_entry(PyThreadState *tstate, _PyInterpreterFrame *frame) { if (tstate->c_tracefunc != NULL) { /* tstate->c_tracefunc, if defined, is a @@ -1548,7 +1548,7 @@ trace_function_entry(PyThreadState *tstate, InterpreterFrame *frame) } static int -trace_function_exit(PyThreadState *tstate, InterpreterFrame *frame, PyObject *retval) +trace_function_exit(PyThreadState *tstate, _PyInterpreterFrame *frame, PyObject *retval) { if (tstate->c_tracefunc) { if (call_trace_protected(tstate->c_tracefunc, tstate->c_traceobj, @@ -1575,10 +1575,10 @@ skip_backwards_over_extended_args(PyCodeObject *code, int offset) return offset; } -static InterpreterFrame * -pop_frame(PyThreadState *tstate, InterpreterFrame *frame) +static _PyInterpreterFrame * +pop_frame(PyThreadState *tstate, _PyInterpreterFrame *frame) { - InterpreterFrame *prev_frame = frame->previous; + _PyInterpreterFrame *prev_frame = frame->previous; _PyEvalFrameClearAndPop(tstate, frame); return prev_frame; } @@ -1599,7 +1599,7 @@ is_method(PyObject **stack_pointer, int args) { (call_shape.kwnames == NULL ? 0 : ((int)PyTuple_GET_SIZE(call_shape.kwnames))) PyObject* _Py_HOT_FUNCTION -_PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int throwflag) +_PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int throwflag) { _Py_EnsureTstateNotNULL(tstate); CALL_STAT_INC(pyeval_calls); @@ -2203,7 +2203,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyCodeObject *code = (PyCodeObject *)getitem->func_code; size_t size = code->co_nlocalsplus + code->co_stacksize + FRAME_SPECIALS_SIZE; assert(code->co_argcount == 2); - InterpreterFrame *new_frame = _PyThreadState_BumpFramePointer(tstate, size); + _PyInterpreterFrame *new_frame = _PyThreadState_BumpFramePointer(tstate, size); if (new_frame == NULL) { goto error; } @@ -4644,7 +4644,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int code_flags = ((PyCodeObject*)PyFunction_GET_CODE(function))->co_flags; PyObject *locals = code_flags & CO_OPTIMIZED ? NULL : PyFunction_GET_GLOBALS(function); STACK_SHRINK(total_args); - InterpreterFrame *new_frame = _PyEvalFramePushAndInit( + _PyInterpreterFrame *new_frame = _PyEvalFramePushAndInit( tstate, (PyFunctionObject *)function, locals, stack_pointer, positional_args, call_shape.kwnames ); @@ -4753,7 +4753,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr PyCodeObject *code = (PyCodeObject *)func->func_code; DEOPT_IF(code->co_argcount != argcount, CALL); STAT_INC(CALL, hit); - InterpreterFrame *new_frame = _PyFrame_Push(tstate, func); + _PyInterpreterFrame *new_frame = _PyFrame_Push(tstate, func); if (new_frame == NULL) { goto error; } @@ -4788,7 +4788,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr int minargs = cache1->min_args; DEOPT_IF(argcount < minargs, CALL); STAT_INC(CALL, hit); - InterpreterFrame *new_frame = _PyFrame_Push(tstate, func); + _PyInterpreterFrame *new_frame = _PyFrame_Push(tstate, func); if (new_frame == NULL) { goto error; } @@ -5306,7 +5306,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr } assert(EMPTY()); _PyFrame_SetStackPointer(frame, stack_pointer); - InterpreterFrame *gen_frame = (InterpreterFrame *)gen->gi_iframe; + _PyInterpreterFrame *gen_frame = (_PyInterpreterFrame *)gen->gi_iframe; _PyFrame_Copy(frame, gen_frame); assert(frame->frame_obj == NULL); gen->gi_frame_valid = 1; @@ -5314,7 +5314,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, InterpreterFrame *frame, int thr gen_frame->f_state = FRAME_CREATED; _Py_LeaveRecursiveCall(tstate); if (!frame->is_entry) { - InterpreterFrame *prev = frame->previous; + _PyInterpreterFrame *prev = frame->previous; _PyThreadState_PopFrame(tstate, frame); frame = cframe.current_frame = prev; _PyFrame_StackPush(frame, (PyObject *)gen); @@ -6292,7 +6292,7 @@ initialize_locals(PyThreadState *tstate, PyFunctionObject *func, } /* Consumes references to func and all the args */ -static InterpreterFrame * +static _PyInterpreterFrame * _PyEvalFramePushAndInit(PyThreadState *tstate, PyFunctionObject *func, PyObject *locals, PyObject* const* args, size_t argcount, PyObject *kwnames) @@ -6300,7 +6300,7 @@ _PyEvalFramePushAndInit(PyThreadState *tstate, PyFunctionObject *func, PyCodeObject * code = (PyCodeObject *)func->func_code; size_t size = code->co_nlocalsplus + code->co_stacksize + FRAME_SPECIALS_SIZE; CALL_STAT_INC(frames_pushed); - InterpreterFrame *frame = _PyThreadState_BumpFramePointer(tstate, size); + _PyInterpreterFrame *frame = _PyThreadState_BumpFramePointer(tstate, size); if (frame == NULL) { goto fail; } @@ -6330,7 +6330,7 @@ _PyEvalFramePushAndInit(PyThreadState *tstate, PyFunctionObject *func, } static void -_PyEvalFrameClearAndPop(PyThreadState *tstate, InterpreterFrame * frame) +_PyEvalFrameClearAndPop(PyThreadState *tstate, _PyInterpreterFrame * frame) { tstate->recursion_remaining--; assert(frame->frame_obj == NULL || frame->frame_obj->f_owns_frame == 0); @@ -6357,7 +6357,7 @@ _PyEval_Vector(PyThreadState *tstate, PyFunctionObject *func, Py_INCREF(args[i+argcount]); } } - InterpreterFrame *frame = _PyEvalFramePushAndInit( + _PyInterpreterFrame *frame = _PyEvalFramePushAndInit( tstate, func, locals, args, argcount, kwnames); if (frame == NULL) { return NULL; @@ -6726,7 +6726,7 @@ prtrace(PyThreadState *tstate, PyObject *v, const char *str) static void call_exc_trace(Py_tracefunc func, PyObject *self, PyThreadState *tstate, - InterpreterFrame *f) + _PyInterpreterFrame *f) { PyObject *type, *value, *traceback, *orig_traceback, *arg; int err; @@ -6756,7 +6756,7 @@ call_exc_trace(Py_tracefunc func, PyObject *self, static int call_trace_protected(Py_tracefunc func, PyObject *obj, - PyThreadState *tstate, InterpreterFrame *frame, + PyThreadState *tstate, _PyInterpreterFrame *frame, int what, PyObject *arg) { PyObject *type, *value, *traceback; @@ -6777,7 +6777,7 @@ call_trace_protected(Py_tracefunc func, PyObject *obj, } static void -initialize_trace_info(PyTraceInfo *trace_info, InterpreterFrame *frame) +initialize_trace_info(PyTraceInfo *trace_info, _PyInterpreterFrame *frame) { PyCodeObject *code = frame->f_code; if (trace_info->code != code) { @@ -6788,7 +6788,7 @@ initialize_trace_info(PyTraceInfo *trace_info, InterpreterFrame *frame) static int call_trace(Py_tracefunc func, PyObject *obj, - PyThreadState *tstate, InterpreterFrame *frame, + PyThreadState *tstate, _PyInterpreterFrame *frame, int what, PyObject *arg) { int result; @@ -6829,7 +6829,7 @@ _PyEval_CallTracing(PyObject *func, PyObject *args) /* See Objects/lnotab_notes.txt for a description of how tracing works. */ static int maybe_call_line_trace(Py_tracefunc func, PyObject *obj, - PyThreadState *tstate, InterpreterFrame *frame, int instr_prev) + PyThreadState *tstate, _PyInterpreterFrame *frame, int instr_prev) { int result = 0; @@ -7007,7 +7007,7 @@ _PyEval_GetAsyncGenFinalizer(void) return tstate->async_gen_finalizer; } -InterpreterFrame * +_PyInterpreterFrame * _PyEval_GetFrame(void) { PyThreadState *tstate = _PyThreadState_GET(); @@ -7031,7 +7031,7 @@ PyEval_GetFrame(void) PyObject * _PyEval_GetBuiltins(PyThreadState *tstate) { - InterpreterFrame *frame = tstate->cframe->current_frame; + _PyInterpreterFrame *frame = tstate->cframe->current_frame; if (frame != NULL) { return frame->f_builtins; } @@ -7070,7 +7070,7 @@ PyObject * PyEval_GetLocals(void) { PyThreadState *tstate = _PyThreadState_GET(); - InterpreterFrame *current_frame = tstate->cframe->current_frame; + _PyInterpreterFrame *current_frame = tstate->cframe->current_frame; if (current_frame == NULL) { _PyErr_SetString(tstate, PyExc_SystemError, "frame does not exist"); return NULL; @@ -7089,7 +7089,7 @@ PyObject * PyEval_GetGlobals(void) { PyThreadState *tstate = _PyThreadState_GET(); - InterpreterFrame *current_frame = tstate->cframe->current_frame; + _PyInterpreterFrame *current_frame = tstate->cframe->current_frame; if (current_frame == NULL) { return NULL; } @@ -7100,7 +7100,7 @@ int PyEval_MergeCompilerFlags(PyCompilerFlags *cf) { PyThreadState *tstate = _PyThreadState_GET(); - InterpreterFrame *current_frame = tstate->cframe->current_frame; + _PyInterpreterFrame *current_frame = tstate->cframe->current_frame; int result = cf->cf_flags != 0; if (current_frame != NULL) { @@ -7304,7 +7304,7 @@ _PyEval_SliceIndexNotNone(PyObject *v, Py_ssize_t *pi) } static PyObject * -import_name(PyThreadState *tstate, InterpreterFrame *frame, +import_name(PyThreadState *tstate, _PyInterpreterFrame *frame, PyObject *name, PyObject *fromlist, PyObject *level) { PyObject *import_func, *res; @@ -7775,7 +7775,7 @@ _PyEval_RequestCodeExtraIndex(freefunc free) } static void -dtrace_function_entry(InterpreterFrame *frame) +dtrace_function_entry(_PyInterpreterFrame *frame) { const char *filename; const char *funcname; @@ -7790,7 +7790,7 @@ dtrace_function_entry(InterpreterFrame *frame) } static void -dtrace_function_return(InterpreterFrame *frame) +dtrace_function_return(_PyInterpreterFrame *frame) { const char *filename; const char *funcname; @@ -7806,7 +7806,7 @@ dtrace_function_return(InterpreterFrame *frame) /* DTrace equivalent of maybe_call_line_trace. */ static void -maybe_dtrace_line(InterpreterFrame *frame, +maybe_dtrace_line(_PyInterpreterFrame *frame, PyTraceInfo *trace_info, int instr_prev) { diff --git a/Python/frame.c b/Python/frame.c index 76697cfa08313..20b4f81425bc8 100644 --- a/Python/frame.c +++ b/Python/frame.c @@ -7,7 +7,7 @@ #include "opcode.h" int -_PyFrame_Traverse(InterpreterFrame *frame, visitproc visit, void *arg) +_PyFrame_Traverse(_PyInterpreterFrame *frame, visitproc visit, void *arg) { Py_VISIT(frame->frame_obj); Py_VISIT(frame->f_locals); @@ -24,7 +24,7 @@ _PyFrame_Traverse(InterpreterFrame *frame, visitproc visit, void *arg) } PyFrameObject * -_PyFrame_MakeAndSetFrameObject(InterpreterFrame *frame) +_PyFrame_MakeAndSetFrameObject(_PyInterpreterFrame *frame) { assert(frame->frame_obj == NULL); PyObject *error_type, *error_value, *error_traceback; @@ -46,7 +46,7 @@ _PyFrame_MakeAndSetFrameObject(InterpreterFrame *frame) } void -_PyFrame_Copy(InterpreterFrame *src, InterpreterFrame *dest) +_PyFrame_Copy(_PyInterpreterFrame *src, _PyInterpreterFrame *dest) { assert(src->stacktop >= src->f_code->co_nlocalsplus); Py_ssize_t size = ((char*)&src->localsplus[src->stacktop]) - (char *)src; @@ -55,17 +55,17 @@ _PyFrame_Copy(InterpreterFrame *src, InterpreterFrame *dest) static void -take_ownership(PyFrameObject *f, InterpreterFrame *frame) +take_ownership(PyFrameObject *f, _PyInterpreterFrame *frame) { assert(f->f_owns_frame == 0); Py_ssize_t size = ((char*)&frame->localsplus[frame->stacktop]) - (char *)frame; - memcpy((InterpreterFrame *)f->_f_frame_data, frame, size); - frame = (InterpreterFrame *)f->_f_frame_data; + memcpy((_PyInterpreterFrame *)f->_f_frame_data, frame, size); + frame = (_PyInterpreterFrame *)f->_f_frame_data; f->f_owns_frame = 1; f->f_frame = frame; assert(f->f_back == NULL); if (frame->previous != NULL) { - /* Link PyFrameObjects.f_back and remove link through InterpreterFrame.previous */ + /* Link PyFrameObjects.f_back and remove link through _PyInterpreterFrame.previous */ PyFrameObject *back = _PyFrame_GetFrameObject(frame->previous); if (back == NULL) { /* Memory error here. */ @@ -84,7 +84,7 @@ take_ownership(PyFrameObject *f, InterpreterFrame *frame) } void -_PyFrame_Clear(InterpreterFrame *frame) +_PyFrame_Clear(_PyInterpreterFrame *frame) { /* It is the responsibility of the owning generator/coroutine * to have cleared the enclosing generator, if any. */ @@ -110,13 +110,13 @@ _PyFrame_Clear(InterpreterFrame *frame) } /* Consumes reference to func */ -InterpreterFrame * +_PyInterpreterFrame * _PyFrame_Push(PyThreadState *tstate, PyFunctionObject *func) { PyCodeObject *code = (PyCodeObject *)func->func_code; size_t size = code->co_nlocalsplus + code->co_stacksize + FRAME_SPECIALS_SIZE; CALL_STAT_INC(frames_pushed); - InterpreterFrame *new_frame = _PyThreadState_BumpFramePointer(tstate, size); + _PyInterpreterFrame *new_frame = _PyThreadState_BumpFramePointer(tstate, size); if (new_frame == NULL) { Py_DECREF(func); return NULL; diff --git a/Python/pystate.c b/Python/pystate.c index 85000016cb4df..edf2f62431f3d 100644 --- a/Python/pystate.c +++ b/Python/pystate.c @@ -1410,7 +1410,7 @@ _PyThread_CurrentFrames(void) for (i = runtime->interpreters.head; i != NULL; i = i->next) { PyThreadState *t; for (t = i->threads.head; t != NULL; t = t->next) { - InterpreterFrame *frame = t->cframe->current_frame; + _PyInterpreterFrame *frame = t->cframe->current_frame; if (frame == NULL) { continue; } @@ -2197,7 +2197,7 @@ push_chunk(PyThreadState *tstate, int size) return res; } -InterpreterFrame * +_PyInterpreterFrame * _PyThreadState_BumpFramePointerSlow(PyThreadState *tstate, size_t size) { assert(size < INT_MAX/sizeof(PyObject *)); @@ -2209,11 +2209,11 @@ _PyThreadState_BumpFramePointerSlow(PyThreadState *tstate, size_t size) else { tstate->datastack_top = top; } - return (InterpreterFrame *)base; + return (_PyInterpreterFrame *)base; } void -_PyThreadState_PopFrame(PyThreadState *tstate, InterpreterFrame * frame) +_PyThreadState_PopFrame(PyThreadState *tstate, _PyInterpreterFrame * frame) { assert(tstate->datastack_chunk); PyObject **base = (PyObject **)frame; diff --git a/Python/sysmodule.c b/Python/sysmodule.c index 342e48ed4222b..a97d0341ddcfd 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -18,7 +18,7 @@ Data members: #include "pycore_call.h" // _PyObject_CallNoArgs() #include "pycore_ceval.h" // _Py_RecursionLimitLowerWaterMark() #include "pycore_code.h" // _Py_QuickenedCount -#include "pycore_frame.h" // InterpreterFrame +#include "pycore_frame.h" // _PyInterpreterFrame #include "pycore_initconfig.h" // _PyStatus_EXCEPTION() #include "pycore_namespace.h" // _PyNamespace_New() #include "pycore_object.h" // _PyObject_IS_GC() @@ -1807,7 +1807,7 @@ sys__getframe_impl(PyObject *module, int depth) /*[clinic end generated code: output=d438776c04d59804 input=c1be8a6464b11ee5]*/ { PyThreadState *tstate = _PyThreadState_GET(); - InterpreterFrame *frame = tstate->cframe->current_frame; + _PyInterpreterFrame *frame = tstate->cframe->current_frame; if (_PySys_Audit(tstate, "sys._getframe", NULL) < 0) { return NULL; diff --git a/Python/traceback.c b/Python/traceback.c index f2ae38e56c330..6a721cf909757 100644 --- a/Python/traceback.c +++ b/Python/traceback.c @@ -1167,7 +1167,7 @@ _Py_DumpASCII(int fd, PyObject *text) This function is signal safe. */ static void -dump_frame(int fd, InterpreterFrame *frame) +dump_frame(int fd, _PyInterpreterFrame *frame) { PyCodeObject *code = frame->f_code; PUTS(fd, " File "); @@ -1205,7 +1205,7 @@ dump_frame(int fd, InterpreterFrame *frame) static void dump_traceback(int fd, PyThreadState *tstate, int write_header) { - InterpreterFrame *frame; + _PyInterpreterFrame *frame; unsigned int depth; if (write_header) { From webhook-mailer at python.org Fri Feb 25 11:41:30 2022 From: webhook-mailer at python.org (JelleZijlstra) Date: Fri, 25 Feb 2022 16:41:30 -0000 Subject: [Python-checkins] bpo-26897: Clarify Popen stdin, stdout, stderr file object docs (GH-30231) Message-ID: https://github.com/python/cpython/commit/dd69f734218ac5d3a551227069ac53ee09b0cd3e commit: dd69f734218ac5d3a551227069ac53ee09b0cd3e branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: JelleZijlstra date: 2022-02-25T08:41:19-08:00 summary: bpo-26897: Clarify Popen stdin, stdout, stderr file object docs (GH-30231) files: M Doc/library/subprocess.rst diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 21a96a4bd5ff5..ab9f1d88a0fc2 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -264,13 +264,14 @@ default values. The arguments that are most commonly needed are: *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive - integer), an existing file object, and ``None``. :data:`PIPE` indicates - that a new pipe to the child should be created. :data:`DEVNULL` indicates - that the special file :data:`os.devnull` will be used. With the default - settings of ``None``, no redirection will occur; the child's file handles - will be inherited from the parent. Additionally, *stderr* can be - :data:`STDOUT`, which indicates that the stderr data from the child - process should be captured into the same file handle as for *stdout*. + integer), an existing file object with a valid file descriptor, and ``None``. + :data:`PIPE` indicates that a new pipe to the child should be created. + :data:`DEVNULL` indicates that the special file :data:`os.devnull` will + be used. With the default settings of ``None``, no redirection will occur; + the child's file handles will be inherited from the parent. + Additionally, *stderr* can be :data:`STDOUT`, which indicates that the + stderr data from the child process should be captured into the same file + handle as for *stdout*. .. index:: single: universal newlines; subprocess module @@ -482,13 +483,14 @@ functions. *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive - integer), an existing :term:`file object`, and ``None``. :data:`PIPE` - indicates that a new pipe to the child should be created. :data:`DEVNULL` - indicates that the special file :data:`os.devnull` will be used. With the - default settings of ``None``, no redirection will occur; the child's file - handles will be inherited from the parent. Additionally, *stderr* can be - :data:`STDOUT`, which indicates that the stderr data from the applications - should be captured into the same file handle as for stdout. + integer), an existing :term:`file object` with a valid file descriptor, + and ``None``. :data:`PIPE` indicates that a new pipe to the child should + be created. :data:`DEVNULL` indicates that the special file + :data:`os.devnull` will be used. With the default settings of ``None``, + no redirection will occur; the child's file handles will be inherited from + the parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates + that the stderr data from the applications should be captured into the same + file handle as for stdout. If *preexec_fn* is set to a callable object, this object will be called in the child process just before the child is executed. From webhook-mailer at python.org Fri Feb 25 12:03:47 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 25 Feb 2022 17:03:47 -0000 Subject: [Python-checkins] bpo-26897: Clarify Popen stdin, stdout, stderr file object docs (GH-30231) Message-ID: https://github.com/python/cpython/commit/1f956d11dbd952fed9b214a0c6e5fd9f91a1f4ca commit: 1f956d11dbd952fed9b214a0c6e5fd9f91a1f4ca branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-25T09:03:37-08:00 summary: bpo-26897: Clarify Popen stdin, stdout, stderr file object docs (GH-30231) (cherry picked from commit dd69f734218ac5d3a551227069ac53ee09b0cd3e) Co-authored-by: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> files: M Doc/library/subprocess.rst diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 21a96a4bd5ff5..ab9f1d88a0fc2 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -264,13 +264,14 @@ default values. The arguments that are most commonly needed are: *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive - integer), an existing file object, and ``None``. :data:`PIPE` indicates - that a new pipe to the child should be created. :data:`DEVNULL` indicates - that the special file :data:`os.devnull` will be used. With the default - settings of ``None``, no redirection will occur; the child's file handles - will be inherited from the parent. Additionally, *stderr* can be - :data:`STDOUT`, which indicates that the stderr data from the child - process should be captured into the same file handle as for *stdout*. + integer), an existing file object with a valid file descriptor, and ``None``. + :data:`PIPE` indicates that a new pipe to the child should be created. + :data:`DEVNULL` indicates that the special file :data:`os.devnull` will + be used. With the default settings of ``None``, no redirection will occur; + the child's file handles will be inherited from the parent. + Additionally, *stderr* can be :data:`STDOUT`, which indicates that the + stderr data from the child process should be captured into the same file + handle as for *stdout*. .. index:: single: universal newlines; subprocess module @@ -482,13 +483,14 @@ functions. *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive - integer), an existing :term:`file object`, and ``None``. :data:`PIPE` - indicates that a new pipe to the child should be created. :data:`DEVNULL` - indicates that the special file :data:`os.devnull` will be used. With the - default settings of ``None``, no redirection will occur; the child's file - handles will be inherited from the parent. Additionally, *stderr* can be - :data:`STDOUT`, which indicates that the stderr data from the applications - should be captured into the same file handle as for stdout. + integer), an existing :term:`file object` with a valid file descriptor, + and ``None``. :data:`PIPE` indicates that a new pipe to the child should + be created. :data:`DEVNULL` indicates that the special file + :data:`os.devnull` will be used. With the default settings of ``None``, + no redirection will occur; the child's file handles will be inherited from + the parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates + that the stderr data from the applications should be captured into the same + file handle as for stdout. If *preexec_fn* is set to a callable object, this object will be called in the child process just before the child is executed. From webhook-mailer at python.org Fri Feb 25 12:07:17 2022 From: webhook-mailer at python.org (miss-islington) Date: Fri, 25 Feb 2022 17:07:17 -0000 Subject: [Python-checkins] bpo-26897: Clarify Popen stdin, stdout, stderr file object docs (GH-30231) Message-ID: https://github.com/python/cpython/commit/de2c91a89aa0e89d5d01d71697d0f27a9f60c11d commit: de2c91a89aa0e89d5d01d71697d0f27a9f60c11d branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-25T09:07:05-08:00 summary: bpo-26897: Clarify Popen stdin, stdout, stderr file object docs (GH-30231) (cherry picked from commit dd69f734218ac5d3a551227069ac53ee09b0cd3e) Co-authored-by: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> files: M Doc/library/subprocess.rst diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 55dcc172bbc28..f9e9f63f2151e 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -264,13 +264,14 @@ default values. The arguments that are most commonly needed are: *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive - integer), an existing file object, and ``None``. :data:`PIPE` indicates - that a new pipe to the child should be created. :data:`DEVNULL` indicates - that the special file :data:`os.devnull` will be used. With the default - settings of ``None``, no redirection will occur; the child's file handles - will be inherited from the parent. Additionally, *stderr* can be - :data:`STDOUT`, which indicates that the stderr data from the child - process should be captured into the same file handle as for *stdout*. + integer), an existing file object with a valid file descriptor, and ``None``. + :data:`PIPE` indicates that a new pipe to the child should be created. + :data:`DEVNULL` indicates that the special file :data:`os.devnull` will + be used. With the default settings of ``None``, no redirection will occur; + the child's file handles will be inherited from the parent. + Additionally, *stderr* can be :data:`STDOUT`, which indicates that the + stderr data from the child process should be captured into the same file + handle as for *stdout*. .. index:: single: universal newlines; subprocess module @@ -463,13 +464,14 @@ functions. *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive - integer), an existing :term:`file object`, and ``None``. :data:`PIPE` - indicates that a new pipe to the child should be created. :data:`DEVNULL` - indicates that the special file :data:`os.devnull` will be used. With the - default settings of ``None``, no redirection will occur; the child's file - handles will be inherited from the parent. Additionally, *stderr* can be - :data:`STDOUT`, which indicates that the stderr data from the applications - should be captured into the same file handle as for stdout. + integer), an existing :term:`file object` with a valid file descriptor, + and ``None``. :data:`PIPE` indicates that a new pipe to the child should + be created. :data:`DEVNULL` indicates that the special file + :data:`os.devnull` will be used. With the default settings of ``None``, + no redirection will occur; the child's file handles will be inherited from + the parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates + that the stderr data from the applications should be captured into the same + file handle as for stdout. If *preexec_fn* is set to a callable object, this object will be called in the child process just before the child is executed. From webhook-mailer at python.org Fri Feb 25 12:24:41 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 17:24:41 -0000 Subject: [Python-checkins] bpo-46857: Fix test_embed.test_no_memleak() on Windows (GH-31589) Message-ID: https://github.com/python/cpython/commit/ea9612a17bc60d44e0058f525d3c02a91c439cef commit: ea9612a17bc60d44e0058f525d3c02a91c439cef branch: main author: Victor Stinner committer: vstinner date: 2022-02-25T18:24:25+01:00 summary: bpo-46857: Fix test_embed.test_no_memleak() on Windows (GH-31589) Tolerate a leak of 1 reference and 1 memory block until it's fixed. files: M Lib/test/test_embed.py diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index e376331063d86..450bbec7005a8 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -1657,10 +1657,16 @@ def test_no_memleak(self): self.fail(f"unexpected output: {out!a}") refs = int(match.group(1)) blocks = int(match.group(2)) - # bpo-46417: Tolerate negative reference count which can occur because - # of bugs in C extensions. It is only wrong if it's greater than 0. - self.assertLessEqual(refs, 0, out) - self.assertEqual(blocks, 0, out) + if not MS_WINDOWS: + # bpo-46417: Tolerate negative reference count which can occur because + # of bugs in C extensions. It is only wrong if it's greater than 0. + self.assertLessEqual(refs, 0, out) + self.assertEqual(blocks, 0, out) + else: + # bpo-46857: on Windows, Python still leaks 1 reference and 1 + # memory block at exit. + self.assertLessEqual(refs, 1, out) + self.assertIn(blocks, (0, 1), out) class StdPrinterTests(EmbeddingTestsMixin, unittest.TestCase): From webhook-mailer at python.org Fri Feb 25 12:49:59 2022 From: webhook-mailer at python.org (ned-deily) Date: Fri, 25 Feb 2022 17:49:59 -0000 Subject: [Python-checkins] bpo-46756: Fix authorization check in urllib.request (GH-31353) (GH-31573) Message-ID: https://github.com/python/cpython/commit/31fef7edf951fb759004c58ae64df18bc32ea376 commit: 31fef7edf951fb759004c58ae64df18bc32ea376 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: ned-deily date: 2022-02-25T12:49:43-05:00 summary: bpo-46756: Fix authorization check in urllib.request (GH-31353) (GH-31573) Fix a bug in urllib.request.HTTPPasswordMgr.find_user_password() and urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated() which allowed to bypass authorization. For example, access to URI "example.org/foobar" was allowed if the user was authorized for URI "example.org/foo". (cherry picked from commit e2e72567a1c94c548868f6ee5329363e6036057a) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst M Lib/test/test_urllib2.py M Lib/urllib/request.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index fe9a32bfdabf0..3bb4a12e86d50 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -160,7 +160,6 @@ def test_password_manager(self): self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam"), ('joe', 'password')) - self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam/spam"), ('joe', 'password')) @@ -169,12 +168,29 @@ def test_password_manager(self): add("c", "http://example.com/foo", "foo", "ni") add("c", "http://example.com/bar", "bar", "nini") + add("c", "http://example.com/foo/bar", "foobar", "nibar") self.assertEqual(find_user_pass("c", "http://example.com/foo"), ('foo', 'ni')) - self.assertEqual(find_user_pass("c", "http://example.com/bar"), ('bar', 'nini')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/bar"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/baz"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foobar"), + (None, None)) + + add("c", "http://example.com/baz/", "baz", "ninini") + + self.assertEqual(find_user_pass("c", "http://example.com/baz"), + (None, None)) + self.assertEqual(find_user_pass("c", "http://example.com/baz/"), + ('baz', 'ninini')) + self.assertEqual(find_user_pass("c", "http://example.com/baz/bar"), + ('baz', 'ninini')) # For the same path, newer password should be considered. @@ -1642,8 +1658,9 @@ def test_basic_prior_auth_auto_send(self): auth_prior_handler.add_password( None, request_url, user, password, is_authenticated=True) - is_auth = pwd_manager.is_authenticated(request_url) - self.assertTrue(is_auth) + self.assertTrue(pwd_manager.is_authenticated(request_url)) + self.assertTrue(pwd_manager.is_authenticated(request_url + '/nested')) + self.assertFalse(pwd_manager.is_authenticated(request_url + 'plain')) opener = OpenerDirector() opener.add_handler(auth_prior_handler) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py index 16b7e1c50985d..993fde87b9225 100644 --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -890,10 +890,10 @@ def is_suburi(self, base, test): return True if base[0] != test[0]: return False - common = posixpath.commonprefix((base[1], test[1])) - if len(common) == len(base[1]): - return True - return False + prefix = base[1] + if prefix[-1:] != '/': + prefix += '/' + return test[1].startswith(prefix) class HTTPPasswordMgrWithDefaultRealm(HTTPPasswordMgr): diff --git a/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst new file mode 100644 index 0000000000000..1660640c5d3fb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst @@ -0,0 +1,5 @@ +Fix a bug in :meth:`urllib.request.HTTPPasswordMgr.find_user_password` and +:meth:`urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated` which +allowed to bypass authorization. For example, access to URI +``example.org/foobar`` was allowed if the user was authorized for URI +``example.org/foo``. From webhook-mailer at python.org Fri Feb 25 13:05:40 2022 From: webhook-mailer at python.org (gvanrossum) Date: Fri, 25 Feb 2022 18:05:40 -0000 Subject: [Python-checkins] bpo-46712: Share global string identifiers in deepfreeze (GH-31261) Message-ID: https://github.com/python/cpython/commit/eb002dbe0da9622245a355db5f0cd5aa2fc70b40 commit: eb002dbe0da9622245a355db5f0cd5aa2fc70b40 branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: gvanrossum date: 2022-02-25T10:05:24-08:00 summary: bpo-46712: Share global string identifiers in deepfreeze (GH-31261) Where appropriate, deepfreeze.c now uses `&_Py_ID(blah)` references instead of locally defining constants. This saves some space. files: A Misc/NEWS.d/next/Core and Builtins/2022-02-24-07-50-43.bpo-46712.pw7vQV.rst M Tools/scripts/deepfreeze.py M Tools/scripts/generate_global_objects.py diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-24-07-50-43.bpo-46712.pw7vQV.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-24-07-50-43.bpo-46712.pw7vQV.rst new file mode 100644 index 0000000000000..9dbf7e074dc61 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-24-07-50-43.bpo-46712.pw7vQV.rst @@ -0,0 +1 @@ +Share global string identifiers in deep-frozen modules. diff --git a/Tools/scripts/deepfreeze.py b/Tools/scripts/deepfreeze.py index b62be3713feb8..8ea232fc46690 100644 --- a/Tools/scripts/deepfreeze.py +++ b/Tools/scripts/deepfreeze.py @@ -15,9 +15,10 @@ from typing import Dict, FrozenSet, TextIO, Tuple import umarshal +from generate_global_objects import get_identifiers_and_strings verbose = False - +identifiers = get_identifiers_and_strings()[0] def isprintable(b: bytes) -> bool: return all(0x20 <= c < 0x7f for c in b) @@ -167,6 +168,8 @@ def generate_bytes(self, name: str, b: bytes) -> str: return f"& {name}.ob_base.ob_base" def generate_unicode(self, name: str, s: str) -> str: + if s in identifiers: + return f"&_Py_ID({s})" kind, ascii = analyze_character_width(s) if kind == PyUnicode_1BYTE_KIND: datatype = "uint8_t" diff --git a/Tools/scripts/generate_global_objects.py b/Tools/scripts/generate_global_objects.py index bad7865f1ff83..506aa86575c4e 100644 --- a/Tools/scripts/generate_global_objects.py +++ b/Tools/scripts/generate_global_objects.py @@ -256,13 +256,10 @@ def generate_runtime_init(identifiers, strings): printer.write(after) -####################################### -# the script - -def main() -> None: +def get_identifiers_and_strings() -> tuple[set[str], dict[str, str]]: identifiers = set(IDENTIFIERS) strings = dict(STRING_LITERALS) - for name, string, filename, lno, _ in iter_global_strings(): + for name, string, *_ in iter_global_strings(): if string is None: if name not in IGNORED: identifiers.add(name) @@ -271,6 +268,13 @@ def main() -> None: strings[name] = string elif string != strings[name]: raise ValueError(f'string mismatch for {name!r} ({string!r} != {strings[name]!r}') + return identifiers, strings + +####################################### +# the script + +def main() -> None: + identifiers, strings = get_identifiers_and_strings() generate_global_strings(identifiers, strings) generate_runtime_init(identifiers, strings) From webhook-mailer at python.org Fri Feb 25 18:17:11 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 23:17:11 -0000 Subject: [Python-checkins] bpo-46816: Remove declarations for non-__STDC__ compilers (GH-31466) Message-ID: https://github.com/python/cpython/commit/4060111f9dc44682f9d7bdafb4e7dacb96706ad3 commit: 4060111f9dc44682f9d7bdafb4e7dacb96706ad3 branch: main author: Oleg Iarygin committer: vstinner date: 2022-02-26T00:16:59+01:00 summary: bpo-46816: Remove declarations for non-__STDC__ compilers (GH-31466) After Python switched to C11, there is a guarantee that every compiler conforms to ISO C so checks of __STDC__ have no sense. files: M Include/internal/pycore_pymath.h M Python/errors.c diff --git a/Include/internal/pycore_pymath.h b/Include/internal/pycore_pymath.h index 1f54b3d81325d..5c6aee2a23890 100644 --- a/Include/internal/pycore_pymath.h +++ b/Include/internal/pycore_pymath.h @@ -8,15 +8,6 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -// Extra declarations -#if !defined(_MSC_VER) && !defined(__STDC__) -extern double fmod (double, double); -extern double frexp (double, int *); -extern double ldexp (double, int); -extern double modf (double, double *); -extern double pow(double, double); -#endif // !defined(_MSC_VER) && !defined(__STDC__) - /* _Py_ADJUST_ERANGE1(x) * _Py_ADJUST_ERANGE2(x, y) diff --git a/Python/errors.c b/Python/errors.c index 10a06a5ae3267..e170c9dff2dbb 100644 --- a/Python/errors.c +++ b/Python/errors.c @@ -10,12 +10,6 @@ #include "pycore_sysmodule.h" // _PySys_Audit() #include "pycore_traceback.h" // _PyTraceBack_FromFrame() -#ifndef __STDC__ -#ifndef MS_WINDOWS -extern char *strerror(int); -#endif -#endif - #include #ifdef MS_WINDOWS # include From webhook-mailer at python.org Fri Feb 25 18:53:38 2022 From: webhook-mailer at python.org (vstinner) Date: Fri, 25 Feb 2022 23:53:38 -0000 Subject: [Python-checkins] bpo-46852: Remove the float.__set_format__() method (GH-31585) Message-ID: https://github.com/python/cpython/commit/5ab745fc51e159ead28b523414e52f0bcc1ef353 commit: 5ab745fc51e159ead28b523414e52f0bcc1ef353 branch: main author: Victor Stinner committer: vstinner date: 2022-02-26T00:53:27+01:00 summary: bpo-46852: Remove the float.__set_format__() method (GH-31585) Remove the undocumented private float.__set_format__() method, previously known as float.__set_format__() in Python 3.7. Its docstring said: "You probably don't want to use this function. It exists mainly to be used in Python's test suite." files: A Misc/NEWS.d/next/Core and Builtins/2022-02-25-01-42-45.bpo-46852.nkRDvV.rst M Doc/library/unittest.mock.rst M Doc/whatsnew/3.11.rst M Lib/test/test_float.py M Lib/unittest/mock.py M Objects/clinic/floatobject.c.h M Objects/floatobject.c diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 69f1035703ba5..dbea928443975 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -2165,7 +2165,7 @@ Magic methods that are supported but not setup by default in ``MagicMock`` are: * ``__reversed__`` and ``__missing__`` * ``__reduce__``, ``__reduce_ex__``, ``__getinitargs__``, ``__getnewargs__``, ``__getstate__`` and ``__setstate__`` -* ``__getformat__`` and ``__setformat__`` +* ``__getformat__`` diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 3f73bdfa3020f..8ebe1cb8cc4ca 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -574,6 +574,12 @@ Removed because it was not used and added by mistake in previous versions. (Contributed by Nikita Sobolev in :issue:`46483`.) +* Remove the undocumented private ``float.__set_format__()`` method, previously + known as ``float.__setformat__()`` in Python 3.7. Its docstring said: "You + probably don't want to use this function. It exists mainly to be used in + Python's test suite." + (Contributed by Victor Stinner in :issue:`46852`.) + Porting to Python 3.11 ====================== diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py index f033736917c65..3d2a21f1522ea 100644 --- a/Lib/test/test_float.py +++ b/Lib/test/test_float.py @@ -19,8 +19,6 @@ have_getformat = hasattr(float, "__getformat__") requires_getformat = unittest.skipUnless(have_getformat, "requires __getformat__") -requires_setformat = unittest.skipUnless(hasattr(float, "__setformat__"), - "requires __setformat__") #locate file with float format test values test_dir = os.path.dirname(__file__) or os.curdir @@ -612,44 +610,6 @@ class F(float, H): self.assertEqual(hash(value), object.__hash__(value)) - at requires_setformat -class FormatFunctionsTestCase(unittest.TestCase): - - def setUp(self): - self.save_formats = {'double':float.__getformat__('double'), - 'float':float.__getformat__('float')} - - def tearDown(self): - float.__setformat__('double', self.save_formats['double']) - float.__setformat__('float', self.save_formats['float']) - - def test_getformat(self): - self.assertIn(float.__getformat__('double'), - ['unknown', 'IEEE, big-endian', 'IEEE, little-endian']) - self.assertIn(float.__getformat__('float'), - ['unknown', 'IEEE, big-endian', 'IEEE, little-endian']) - self.assertRaises(ValueError, float.__getformat__, 'chicken') - self.assertRaises(TypeError, float.__getformat__, 1) - - def test_setformat(self): - for t in 'double', 'float': - float.__setformat__(t, 'unknown') - if self.save_formats[t] == 'IEEE, big-endian': - self.assertRaises(ValueError, float.__setformat__, - t, 'IEEE, little-endian') - elif self.save_formats[t] == 'IEEE, little-endian': - self.assertRaises(ValueError, float.__setformat__, - t, 'IEEE, big-endian') - else: - self.assertRaises(ValueError, float.__setformat__, - t, 'IEEE, big-endian') - self.assertRaises(ValueError, float.__setformat__, - t, 'IEEE, little-endian') - self.assertRaises(ValueError, float.__setformat__, - t, 'chicken') - self.assertRaises(ValueError, float.__setformat__, - 'chicken', 'unknown') - BE_DOUBLE_INF = b'\x7f\xf0\x00\x00\x00\x00\x00\x00' LE_DOUBLE_INF = bytes(reversed(BE_DOUBLE_INF)) BE_DOUBLE_NAN = b'\x7f\xf8\x00\x00\x00\x00\x00\x00' @@ -660,36 +620,6 @@ def test_setformat(self): BE_FLOAT_NAN = b'\x7f\xc0\x00\x00' LE_FLOAT_NAN = bytes(reversed(BE_FLOAT_NAN)) -# on non-IEEE platforms, attempting to unpack a bit pattern -# representing an infinity or a NaN should raise an exception. - - at requires_setformat -class UnknownFormatTestCase(unittest.TestCase): - def setUp(self): - self.save_formats = {'double':float.__getformat__('double'), - 'float':float.__getformat__('float')} - float.__setformat__('double', 'unknown') - float.__setformat__('float', 'unknown') - - def tearDown(self): - float.__setformat__('double', self.save_formats['double']) - float.__setformat__('float', self.save_formats['float']) - - def test_double_specials_dont_unpack(self): - for fmt, data in [('>d', BE_DOUBLE_INF), - ('>d', BE_DOUBLE_NAN), - ('f', BE_FLOAT_INF), - ('>f', BE_FLOAT_NAN), - (' https://github.com/python/cpython/commit/e466faa9df9a1bd377d9725de5484471bc4af8d0 commit: e466faa9df9a1bd377d9725de5484471bc4af8d0 branch: main author: Charlie Zhao <68189100+CharlieZhao95 at users.noreply.github.com> committer: tim-one date: 2022-02-25T22:17:13-06:00 summary: bpo-45735: Promise the long-time truth that `args=list` works (GH-30982) For threads, and for multiprocessing, it's always been the case that ``args=list`` works fine when passed to ``Process()`` or ``Thread()``, and such code is common in the wild. But, according to the docs, only a tuple can be used. This brings the docs into synch with reality. Doc changes by Charlie Zhao. Co-authored-by: Tim Peters files: M Doc/library/multiprocessing.rst M Doc/library/threading.rst M Lib/test/_test_multiprocessing.py M Lib/test/test_threading.py M Lib/threading.py M Misc/ACKS diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index cbbe1840fc351..ee40688781690 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -485,7 +485,9 @@ The :mod:`multiprocessing` package mostly replicates the API of the to ``True`` or ``False``. If ``None`` (the default), this flag will be inherited from the creating process. - By default, no arguments are passed to *target*. + By default, no arguments are passed to *target*. The *args* argument, + which defaults to ``()``, can be used to specify a list or tuple of the arguments + to pass to *target*. If a subclass overrides the constructor, it must make sure it invokes the base class constructor (:meth:`Process.__init__`) before doing anything else @@ -503,6 +505,19 @@ The :mod:`multiprocessing` package mostly replicates the API of the the target argument, if any, with sequential and keyword arguments taken from the *args* and *kwargs* arguments, respectively. + Using a list or tuple as the *args* argument passed to :class:`Process` + achieves the same effect. + + Example:: + + >>> from multiprocessing import Process + >>> p = Process(target=print, args=[1]) + >>> p.run() + 1 + >>> p = Process(target=print, args=(1,)) + >>> p.run() + 1 + .. method:: start() Start the process's activity. diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index 8c7664328a49d..2bcb72b6d4e50 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -314,7 +314,7 @@ since it is impossible to detect the termination of alien threads. or "Thread-*N* (target)" where "target" is ``target.__name__`` if the *target* argument is specified. - *args* is the argument tuple for the target invocation. Defaults to ``()``. + *args* is a list or tuple of arguments for the target invocation. Defaults to ``()``. *kwargs* is a dictionary of keyword arguments for the target invocation. Defaults to ``{}``. @@ -353,6 +353,19 @@ since it is impossible to detect the termination of alien threads. the *target* argument, if any, with positional and keyword arguments taken from the *args* and *kwargs* arguments, respectively. + Using list or tuple as the *args* argument which passed to the :class:`Thread` + could achieve the same effect. + + Example:: + + >>> from threading import Thread + >>> t = Thread(target=print, args=[1]) + >>> t.run() + 1 + >>> t = Thread(target=print, args=(1,)) + >>> t.run() + 1 + .. method:: join(timeout=None) Wait until the thread terminates. This blocks the calling thread until diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index b2d656ab42897..6b1b1677910d1 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -247,6 +247,30 @@ def test_current(self): self.assertEqual(current.ident, os.getpid()) self.assertEqual(current.exitcode, None) + def test_args_argument(self): + # bpo-45735: Using list or tuple as *args* in constructor could + # achieve the same effect. + args_cases = (1, "str", [1], (1,)) + args_types = (list, tuple) + + test_cases = itertools.product(args_cases, args_types) + + for args, args_type in test_cases: + with self.subTest(args=args, args_type=args_type): + q = self.Queue(1) + # pass a tuple or list as args + p = self.Process(target=self._test_args, args=args_type((q, args))) + p.daemon = True + p.start() + child_args = q.get() + self.assertEqual(child_args, args) + p.join() + close_queue(q) + + @classmethod + def _test_args(cls, q, arg): + q.put(arg) + def test_daemon_argument(self): if self.TYPE == "threads": self.skipTest('test not appropriate for {}'.format(self.TYPE)) diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index 4830571474b5b..16c6934c6d432 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -123,6 +123,32 @@ def func(): pass thread = threading.Thread(target=func) self.assertEqual(thread.name, "Thread-5 (func)") + def test_args_argument(self): + # bpo-45735: Using list or tuple as *args* in constructor could + # achieve the same effect. + num_list = [1] + num_tuple = (1,) + + str_list = ["str"] + str_tuple = ("str",) + + list_in_tuple = ([1],) + tuple_in_list = [(1,)] + + test_cases = ( + (num_list, lambda arg: self.assertEqual(arg, 1)), + (num_tuple, lambda arg: self.assertEqual(arg, 1)), + (str_list, lambda arg: self.assertEqual(arg, "str")), + (str_tuple, lambda arg: self.assertEqual(arg, "str")), + (list_in_tuple, lambda arg: self.assertEqual(arg, [1])), + (tuple_in_list, lambda arg: self.assertEqual(arg, (1,))) + ) + + for args, target in test_cases: + with self.subTest(target=target, args=args): + t = threading.Thread(target=target, args=args) + t.start() + @cpython_only def test_disallow_instantiation(self): # Ensure that the type disallows instantiation (bpo-43916) diff --git a/Lib/threading.py b/Lib/threading.py index 6068d06ab6c90..642f93e1eec31 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -852,7 +852,7 @@ class is implemented. *name* is the thread name. By default, a unique name is constructed of the form "Thread-N" where N is a small decimal number. - *args* is the argument tuple for the target invocation. Defaults to (). + *args* is a list or tuple of arguments for the target invocation. Defaults to (). *kwargs* is a dictionary of keyword arguments for the target invocation. Defaults to {}. diff --git a/Misc/ACKS b/Misc/ACKS index bab04b4613646..da2c82610d5ad 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -2004,6 +2004,7 @@ Yuxiao Zeng Uwe Zessin Cheng Zhang George Zhang +Charlie Zhao Kai Zhu Tarek Ziad? Jelle Zijlstra From webhook-mailer at python.org Sat Feb 26 07:05:51 2022 From: webhook-mailer at python.org (serhiy-storchaka) Date: Sat, 26 Feb 2022 12:05:51 -0000 Subject: [Python-checkins] bpo-38415: Remove redundant AsyncContextDecorator.__call__ override from _AsyncGeneratorContextManager (GH-30233) Message-ID: https://github.com/python/cpython/commit/b57dbe5d1be925b99f16fe5893e339f92fc05888 commit: b57dbe5d1be925b99f16fe5893e339f92fc05888 branch: main author: Thomas Grainger committer: serhiy-storchaka date: 2022-02-26T14:05:20+02:00 summary: bpo-38415: Remove redundant AsyncContextDecorator.__call__ override from _AsyncGeneratorContextManager (GH-30233) files: M Lib/contextlib.py diff --git a/Lib/contextlib.py b/Lib/contextlib.py index ee72258505714..4cff9c6a46117 100644 --- a/Lib/contextlib.py +++ b/Lib/contextlib.py @@ -193,14 +193,6 @@ class _AsyncGeneratorContextManager( ): """Helper for @asynccontextmanager decorator.""" - def __call__(self, func): - @wraps(func) - async def inner(*args, **kwds): - async with self.__class__(self.func, self.args, self.kwds): - return await func(*args, **kwds) - - return inner - async def __aenter__(self): # do not keep args and kwds alive unnecessarily # they are only needed for recreation, which is not possible anymore From webhook-mailer at python.org Sat Feb 26 08:06:51 2022 From: webhook-mailer at python.org (asvetlov) Date: Sat, 26 Feb 2022 13:06:51 -0000 Subject: [Python-checkins] bpo-44011: Document ssl_shutdown_timeout added by revisited asyncio SSL implementation (GH-31597) Message-ID: https://github.com/python/cpython/commit/41ddcd3f40f8171a396e57b841a74fcc83884eab commit: 41ddcd3f40f8171a396e57b841a74fcc83884eab branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: asvetlov date: 2022-02-26T15:06:43+02:00 summary: bpo-44011: Document ssl_shutdown_timeout added by revisited asyncio SSL implementation (GH-31597) files: M Doc/library/asyncio-eventloop.rst diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 0c65d75dbf8e0..1b762a75aed0a 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -369,6 +369,7 @@ Opening network connections family=0, proto=0, flags=0, sock=None, \ local_addr=None, server_hostname=None, \ ssl_handshake_timeout=None, \ + ssl_shutdown_timeout=None, \ happy_eyeballs_delay=None, interleave=None) Open a streaming transport connection to a given @@ -456,6 +457,10 @@ Opening network connections to wait for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). + * *ssl_shutdown_timeout* is the time in seconds to wait for the SSL shutdown + to complete before aborting the connection. ``30.0`` seconds if ``None`` + (default). + .. versionchanged:: 3.5 Added support for SSL/TLS in :class:`ProactorEventLoop`. @@ -484,6 +489,10 @@ Opening network connections For more information: https://tools.ietf.org/html/rfc6555 + .. versionchanged:: 3.11 + + Added the *ssl_shutdown_timeout* parameter. + .. seealso:: The :func:`open_connection` function is a high-level alternative @@ -569,7 +578,8 @@ Opening network connections .. coroutinemethod:: loop.create_unix_connection(protocol_factory, \ path=None, *, ssl=None, sock=None, \ - server_hostname=None, ssl_handshake_timeout=None) + server_hostname=None, ssl_handshake_timeout=None, \ + ssl_shutdown_timeout=None) Create a Unix connection. @@ -592,6 +602,10 @@ Opening network connections Added the *ssl_handshake_timeout* parameter. The *path* parameter can now be a :term:`path-like object`. + .. versionchanged:: 3.11 + + Added the *ssl_shutdown_timeout* parameter. + Creating network servers ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -602,7 +616,9 @@ Creating network servers flags=socket.AI_PASSIVE, \ sock=None, backlog=100, ssl=None, \ reuse_address=None, reuse_port=None, \ - ssl_handshake_timeout=None, start_serving=True) + ssl_handshake_timeout=None, \ + ssl_shutdown_timeout=None, \ + start_serving=True) Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) listening on *port* of the *host* address. @@ -662,6 +678,10 @@ Creating network servers for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). + * *ssl_shutdown_timeout* is the time in seconds to wait for the SSL shutdown + to complete before aborting the connection. ``30.0`` seconds if ``None`` + (default). + * *start_serving* set to ``True`` (the default) causes the created server to start accepting connections immediately. When set to ``False``, the user should await on :meth:`Server.start_serving` or @@ -682,6 +702,10 @@ Creating network servers The socket option :py:data:`~socket.TCP_NODELAY` is set by default for all TCP connections. + .. versionchanged:: 3.11 + + Added the *ssl_shutdown_timeout* parameter. + .. seealso:: The :func:`start_server` function is a higher-level alternative API @@ -691,7 +715,9 @@ Creating network servers .. coroutinemethod:: loop.create_unix_server(protocol_factory, path=None, \ *, sock=None, backlog=100, ssl=None, \ - ssl_handshake_timeout=None, start_serving=True) + ssl_handshake_timeout=None, \ + ssl_shutdown_timeout=None, \ + start_serving=True) Similar to :meth:`loop.create_server` but works with the :py:data:`~socket.AF_UNIX` socket family. @@ -711,8 +737,14 @@ Creating network servers Added the *ssl_handshake_timeout* and *start_serving* parameters. The *path* parameter can now be a :class:`~pathlib.Path` object. + .. versionchanged:: 3.11 + + Added the *ssl_shutdown_timeout* parameter. + + .. coroutinemethod:: loop.connect_accepted_socket(protocol_factory, \ - sock, *, ssl=None, ssl_handshake_timeout=None) + sock, *, ssl=None, ssl_handshake_timeout=None, \ + ssl_shutdown_timeout=None) Wrap an already accepted connection into a transport/protocol pair. @@ -734,6 +766,10 @@ Creating network servers wait for the SSL handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). + * *ssl_shutdown_timeout* is the time in seconds to wait for the SSL shutdown + to complete before aborting the connection. ``30.0`` seconds if ``None`` + (default). + Returns a ``(transport, protocol)`` pair. .. versionadded:: 3.5.3 @@ -742,6 +778,10 @@ Creating network servers Added the *ssl_handshake_timeout* parameter. + .. versionchanged:: 3.11 + + Added the *ssl_shutdown_timeout* parameter. + Transferring files ^^^^^^^^^^^^^^^^^^ @@ -778,7 +818,8 @@ TLS Upgrade .. coroutinemethod:: loop.start_tls(transport, protocol, \ sslcontext, *, server_side=False, \ - server_hostname=None, ssl_handshake_timeout=None) + server_hostname=None, ssl_handshake_timeout=None, \ + ssl_shutdown_timeout=None) Upgrade an existing transport-based connection to TLS. @@ -804,8 +845,17 @@ TLS Upgrade wait for the TLS handshake to complete before aborting the connection. ``60.0`` seconds if ``None`` (default). + * *ssl_shutdown_timeout* is the time in seconds to wait for the SSL shutdown + to complete before aborting the connection. ``30.0`` seconds if ``None`` + (default). + .. versionadded:: 3.7 + .. versionchanged:: 3.11 + + Added the *ssl_shutdown_timeout* parameter. + + Watching file descriptors ^^^^^^^^^^^^^^^^^^^^^^^^^ From webhook-mailer at python.org Sat Feb 26 11:18:55 2022 From: webhook-mailer at python.org (gvanrossum) Date: Sat, 26 Feb 2022 16:18:55 -0000 Subject: [Python-checkins] Taskgroup tweaks (GH-31559) Message-ID: https://github.com/python/cpython/commit/edbee56d698ebb4489aa68311f44d104a23f5eb7 commit: edbee56d698ebb4489aa68311f44d104a23f5eb7 branch: main author: Tin Tvrtkovi? committer: gvanrossum date: 2022-02-26T08:18:48-08:00 summary: Taskgroup tweaks (GH-31559) Now uses .cancel()/.uncancel(), for even fewer broken edge cases. files: M Lib/asyncio/taskgroups.py M Lib/test/test_asyncio/test_taskgroups.py diff --git a/Lib/asyncio/taskgroups.py b/Lib/asyncio/taskgroups.py index 756fc551e013d..c3ce94a4dd0a9 100644 --- a/Lib/asyncio/taskgroups.py +++ b/Lib/asyncio/taskgroups.py @@ -66,31 +66,28 @@ async def __aexit__(self, et, exc, tb): self._base_error is None): self._base_error = exc - if et is exceptions.CancelledError: - if self._parent_cancel_requested: - # Only if we did request task to cancel ourselves - # we mark it as no longer cancelled. - self._parent_task.uncancel() - else: - propagate_cancellation_error = et - - if et is not None and not self._aborting: - # Our parent task is being cancelled: - # - # async with TaskGroup() as g: - # g.create_task(...) - # await ... # <- CancelledError - # + if et is not None: if et is exceptions.CancelledError: - propagate_cancellation_error = et - - # or there's an exception in "async with": - # - # async with TaskGroup() as g: - # g.create_task(...) - # 1 / 0 - # - self._abort() + if self._parent_cancel_requested and not self._parent_task.uncancel(): + # Do nothing, i.e. swallow the error. + pass + else: + propagate_cancellation_error = exc + + if not self._aborting: + # Our parent task is being cancelled: + # + # async with TaskGroup() as g: + # g.create_task(...) + # await ... # <- CancelledError + # + # or there's an exception in "async with": + # + # async with TaskGroup() as g: + # g.create_task(...) + # 1 / 0 + # + self._abort() # We use while-loop here because "self._on_completed_fut" # can be cancelled multiple times if our parent task @@ -118,7 +115,6 @@ async def __aexit__(self, et, exc, tb): self._on_completed_fut = None assert self._unfinished_tasks == 0 - self._on_completed_fut = None # no longer needed if self._base_error is not None: raise self._base_error @@ -199,8 +195,7 @@ def _on_task_done(self, task): }) return - self._abort() - if not self._parent_task.cancelling(): + if not self._aborting and not self._parent_cancel_requested: # If parent task *is not* being cancelled, it means that we want # to manually cancel it to abort whatever is being run right now # in the TaskGroup. But we want to mark parent task as @@ -219,5 +214,6 @@ def _on_task_done(self, task): # pass # await something_else # this line has to be called # # after TaskGroup is finished. + self._abort() self._parent_cancel_requested = True self._parent_task.cancel() diff --git a/Lib/test/test_asyncio/test_taskgroups.py b/Lib/test/test_asyncio/test_taskgroups.py index 40774a8993e3f..df51528e10793 100644 --- a/Lib/test/test_asyncio/test_taskgroups.py +++ b/Lib/test/test_asyncio/test_taskgroups.py @@ -120,7 +120,11 @@ async def runner(): self.assertTrue(t2_cancel) self.assertTrue(t2.cancelled()) - async def test_taskgroup_05(self): + async def test_cancel_children_on_child_error(self): + """ + When a child task raises an error, the rest of the children + are cancelled and the errors are gathered into an EG. + """ NUM = 0 t2_cancel = False @@ -165,7 +169,7 @@ async def runner(): self.assertTrue(t2_cancel) self.assertTrue(runner_cancel) - async def test_taskgroup_06(self): + async def test_cancellation(self): NUM = 0 @@ -186,10 +190,12 @@ async def runner(): await asyncio.sleep(0.1) self.assertFalse(r.done()) - r.cancel() - with self.assertRaises(asyncio.CancelledError): + r.cancel("test") + with self.assertRaises(asyncio.CancelledError) as cm: await r + self.assertEqual(cm.exception.args, ('test',)) + self.assertEqual(NUM, 5) async def test_taskgroup_07(self): @@ -226,7 +232,7 @@ async def runner(): self.assertEqual(NUM, 15) - async def test_taskgroup_08(self): + async def test_cancellation_in_body(self): async def foo(): await asyncio.sleep(0.1) @@ -246,10 +252,12 @@ async def runner(): await asyncio.sleep(0.1) self.assertFalse(r.done()) - r.cancel() - with self.assertRaises(asyncio.CancelledError): + r.cancel("test") + with self.assertRaises(asyncio.CancelledError) as cm: await r + self.assertEqual(cm.exception.args, ('test',)) + async def test_taskgroup_09(self): t1 = t2 = None @@ -699,3 +707,7 @@ async def coro(): async with taskgroups.TaskGroup() as g: t = g.create_task(coro(), name="yolo") self.assertEqual(t.get_name(), "yolo") + + +if __name__ == "__main__": + unittest.main() From webhook-mailer at python.org Sat Feb 26 11:35:13 2022 From: webhook-mailer at python.org (gvanrossum) Date: Sat, 26 Feb 2022 16:35:13 -0000 Subject: [Python-checkins] Propagate errors (however unlikely) from _Py_Deepfreeze_Init() (GH-31596) Message-ID: https://github.com/python/cpython/commit/0d9b565e62a5fc8c3e9b8c64cce764fe084ccb2b commit: 0d9b565e62a5fc8c3e9b8c64cce764fe084ccb2b branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: gvanrossum date: 2022-02-26T08:35:03-08:00 summary: Propagate errors (however unlikely) from _Py_Deepfreeze_Init() (GH-31596) files: M Include/internal/pycore_code.h M Include/internal/pycore_pylifecycle.h M Objects/codeobject.c M Programs/_bootstrap_python.c M Programs/_freeze_module.c M Python/pylifecycle.c M Tools/scripts/deepfreeze.py diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index d83df5e300468..0c4850f98a318 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -317,7 +317,7 @@ extern void _Py_Specialize_UnpackSequence(PyObject *seq, _Py_CODEUNIT *instr, /* Deallocator function for static codeobjects used in deepfreeze.py */ extern void _PyStaticCode_Dealloc(PyCodeObject *co); /* Function to intern strings of codeobjects */ -extern void _PyStaticCode_InternStrings(PyCodeObject *co); +extern int _PyStaticCode_InternStrings(PyCodeObject *co); #ifdef Py_STATS diff --git a/Include/internal/pycore_pylifecycle.h b/Include/internal/pycore_pylifecycle.h index 00d13b85d2c25..295505f1f3735 100644 --- a/Include/internal/pycore_pylifecycle.h +++ b/Include/internal/pycore_pylifecycle.h @@ -65,7 +65,7 @@ extern PyStatus _Py_HashRandomization_Init(const PyConfig *); extern PyStatus _PyImportZip_Init(PyThreadState *tstate); extern PyStatus _PyGC_Init(PyInterpreterState *interp); extern PyStatus _PyAtExit_Init(PyInterpreterState *interp); -extern void _Py_Deepfreeze_Init(void); +extern int _Py_Deepfreeze_Init(void); /* Various internal finalizers */ diff --git a/Objects/codeobject.c b/Objects/codeobject.c index f947595803aed..5a87e6c4ff877 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -1931,14 +1931,20 @@ _PyStaticCode_Dealloc(PyCodeObject *co) } } -void +int _PyStaticCode_InternStrings(PyCodeObject *co) { int res = intern_strings(co->co_names); - assert(res == 0); + if (res < 0) { + return -1; + } res = intern_string_constants(co->co_consts, NULL); - assert(res == 0); + if (res < 0) { + return -1; + } res = intern_strings(co->co_localsplusnames); - assert(res == 0); - (void)res; + if (res < 0) { + return -1; + } + return 0; } diff --git a/Programs/_bootstrap_python.c b/Programs/_bootstrap_python.c index 75d455ca17983..f6b49c8c806a1 100644 --- a/Programs/_bootstrap_python.c +++ b/Programs/_bootstrap_python.c @@ -15,8 +15,9 @@ /* End includes */ /* Empty initializer for deepfrozen modules */ -void _Py_Deepfreeze_Init(void) +int _Py_Deepfreeze_Init(void) { + return 0; } /* Empty finalizer for deepfrozen modules */ void diff --git a/Programs/_freeze_module.c b/Programs/_freeze_module.c index d5a236a0c635c..3d27b79c237c3 100644 --- a/Programs/_freeze_module.c +++ b/Programs/_freeze_module.c @@ -23,8 +23,9 @@ #endif /* Empty initializer for deepfrozen modules */ -void _Py_Deepfreeze_Init(void) +int _Py_Deepfreeze_Init(void) { + return 0; } /* Empty finalizer for deepfrozen modules */ void diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index a671bcaf42c66..61534742005a3 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -828,7 +828,9 @@ pycore_interp_init(PyThreadState *tstate) } // Intern strings in deep-frozen modules first so that others // can use it instead of creating a heap allocated string. - _Py_Deepfreeze_Init(); + if (_Py_Deepfreeze_Init() < 0) { + return _PyStatus_ERR("failed to initialize deep-frozen modules"); + } status = pycore_init_types(interp); if (_PyStatus_EXCEPTION(status)) { diff --git a/Tools/scripts/deepfreeze.py b/Tools/scripts/deepfreeze.py index 8ea232fc46690..cdd34ddb8b1e8 100644 --- a/Tools/scripts/deepfreeze.py +++ b/Tools/scripts/deepfreeze.py @@ -283,7 +283,7 @@ def generate_code(self, name: str, code: types.CodeType) -> str: self.write(f".co_cellvars = {co_cellvars},") self.write(f".co_freevars = {co_freevars},") self.deallocs.append(f"_PyStaticCode_Dealloc(&{name});") - self.interns.append(f"_PyStaticCode_InternStrings(&{name});") + self.interns.append(f"_PyStaticCode_InternStrings(&{name})") return f"& {name}.ob_base" def generate_tuple(self, name: str, t: Tuple[object, ...]) -> str: @@ -450,9 +450,11 @@ def generate(args: list[str], output: TextIO) -> None: with printer.block(f"void\n_Py_Deepfreeze_Fini(void)"): for p in printer.deallocs: printer.write(p) - with printer.block(f"void\n_Py_Deepfreeze_Init(void)"): + with printer.block(f"int\n_Py_Deepfreeze_Init(void)"): for p in printer.interns: - printer.write(p) + with printer.block(f"if ({p} < 0)"): + printer.write("return -1;") + printer.write("return 0;") if verbose: print(f"Cache hits: {printer.hits}, misses: {printer.misses}") From webhook-mailer at python.org Sat Feb 26 13:30:10 2022 From: webhook-mailer at python.org (vsajip) Date: Sat, 26 Feb 2022 18:30:10 -0000 Subject: [Python-checkins] [doc] Add a note in howto/logging.rst about "do not log to root logger in libraries" (GH-31271) Message-ID: https://github.com/python/cpython/commit/fc44b8136ffc501264731ccc7456729b5cf4e74b commit: fc44b8136ffc501264731ccc7456729b5cf4e74b branch: main author: Yuxin Wu committer: vsajip date: 2022-02-26T18:29:51Z summary: [doc] Add a note in howto/logging.rst about "do not log to root logger in libraries" (GH-31271) files: M Doc/howto/logging.rst diff --git a/Doc/howto/logging.rst b/Doc/howto/logging.rst index fcc6bec768800..4d76c27332ccd 100644 --- a/Doc/howto/logging.rst +++ b/Doc/howto/logging.rst @@ -831,6 +831,13 @@ should have the desired effect. If an organisation produces a number of libraries, then the logger name specified can be 'orgname.foo' rather than just 'foo'. +.. note:: It is strongly advised that you *do not log to the root logger* + in your library. Instead, use a logger with a unique and easily + identifiable name, such as the ``__name__`` for your library's top-level package + or module. Logging to the root logger will make it difficult or impossible for + the application developer to configure the logging verbosity or handlers of + your library as they wish. + .. note:: It is strongly advised that you *do not add any handlers other than* :class:`~logging.NullHandler` *to your library's loggers*. This is because the configuration of handlers is the prerogative of the application From webhook-mailer at python.org Sat Feb 26 18:14:36 2022 From: webhook-mailer at python.org (vstinner) Date: Sat, 26 Feb 2022 23:14:36 -0000 Subject: [Python-checkins] bpo-46606: os.getgroups() doesn't overallocate (GH-31569) Message-ID: https://github.com/python/cpython/commit/e02c47528b31f513d5f5d6eb91b8c9714134cea2 commit: e02c47528b31f513d5f5d6eb91b8c9714134cea2 branch: main author: Victor Stinner committer: vstinner date: 2022-02-27T00:14:28+01:00 summary: bpo-46606: os.getgroups() doesn't overallocate (GH-31569) files: M Modules/posixmodule.c diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index d3cfc828184e5..3431c85e2dfde 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -7623,29 +7623,19 @@ static PyObject * os_getgroups_impl(PyObject *module) /*[clinic end generated code: output=42b0c17758561b56 input=d3f109412e6a155c]*/ { - /* On MacOSX getgroups(2) can return more than MAX_GROUPS results - * This is a helper variable to store the intermediate result when - * that happens. - * - * See bpo-7900. - */ - gid_t *grouplist = NULL; - int n; - - /* Issue #17557: As of OS X 10.8, getgroups(2) no longer raises EINVAL if - * there are more groups than can fit in grouplist. Therefore, on OS X - * always first call getgroups with length 0 to get the actual number - * of groups. - */ - n = getgroups(0, NULL); + // Call getgroups with length 0 to get the actual number of groups + int n = getgroups(0, NULL); if (n < 0) { return posix_error(); - } else { - n++; // Avoid malloc(0) - grouplist = PyMem_New(gid_t, n+1); - if (grouplist == NULL) { - return PyErr_NoMemory(); - } + } + + if (n == 0) { + return PyList_New(0); + } + + gid_t *grouplist = PyMem_New(gid_t, n); + if (grouplist == NULL) { + return PyErr_NoMemory(); } n = getgroups(n, grouplist); @@ -7655,22 +7645,25 @@ os_getgroups_impl(PyObject *module) } PyObject *result = PyList_New(n); - if (result != NULL) { - int i; - for (i = 0; i < n; ++i) { - PyObject *o = _PyLong_FromGid(grouplist[i]); - if (o == NULL) { - Py_DECREF(result); - result = NULL; - break; - } - PyList_SET_ITEM(result, i, o); - } + if (result == NULL) { + goto error; } + for (int i = 0; i < n; ++i) { + PyObject *group = _PyLong_FromGid(grouplist[i]); + if (group == NULL) { + goto error; + } + PyList_SET_ITEM(result, i, group); + } PyMem_Free(grouplist); return result; + +error: + PyMem_Free(grouplist); + Py_XDECREF(result); + return NULL; } #endif /* HAVE_GETGROUPS */ From webhook-mailer at python.org Sat Feb 26 18:25:58 2022 From: webhook-mailer at python.org (vstinner) Date: Sat, 26 Feb 2022 23:25:58 -0000 Subject: [Python-checkins] bpo-46748: Fix ctypes test_frozentable() (GH-31600) Message-ID: https://github.com/python/cpython/commit/e182c660b63bc23420fb9f0593d77a3fa3b7f1c7 commit: e182c660b63bc23420fb9f0593d77a3fa3b7f1c7 branch: main author: Kumar Aditya <59607654+kumaraditya303 at users.noreply.github.com> committer: vstinner date: 2022-02-27T00:25:46+01:00 summary: bpo-46748: Fix ctypes test_frozentable() (GH-31600) files: M Lib/ctypes/test/test_values.py diff --git a/Lib/ctypes/test/test_values.py b/Lib/ctypes/test/test_values.py index b2db426d10f43..435fdd22ea2be 100644 --- a/Lib/ctypes/test/test_values.py +++ b/Lib/ctypes/test/test_values.py @@ -54,7 +54,7 @@ class struct_frozen(Structure): _fields_ = [("name", c_char_p), ("code", POINTER(c_ubyte)), ("size", c_int), - ("is_package", c_bool), + ("is_package", c_int), ("get_code", POINTER(c_ubyte)), # Function ptr ] FrozenTable = POINTER(struct_frozen) From webhook-mailer at python.org Sat Feb 26 18:28:44 2022 From: webhook-mailer at python.org (vstinner) Date: Sat, 26 Feb 2022 23:28:44 -0000 Subject: [Python-checkins] bpo-46857: Fix refleak in OSError INIT_ALIAS() (GH-31594) Message-ID: https://github.com/python/cpython/commit/ad56919c5ed54523f866e6605a2573ab7b7d5235 commit: ad56919c5ed54523f866e6605a2573ab7b7d5235 branch: main author: Victor Stinner committer: vstinner date: 2022-02-27T00:28:24+01:00 summary: bpo-46857: Fix refleak in OSError INIT_ALIAS() (GH-31594) _Py_GetRefTotal() no longer decrements _PySet_Dummy refcount. files: M Lib/test/test_embed.py M Objects/exceptions.c M Objects/object.c diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 450bbec7005a8..80b9674c1c258 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -1657,15 +1657,11 @@ def test_no_memleak(self): self.fail(f"unexpected output: {out!a}") refs = int(match.group(1)) blocks = int(match.group(2)) + self.assertEqual(refs, 0, out) if not MS_WINDOWS: - # bpo-46417: Tolerate negative reference count which can occur because - # of bugs in C extensions. It is only wrong if it's greater than 0. - self.assertLessEqual(refs, 0, out) self.assertEqual(blocks, 0, out) else: - # bpo-46857: on Windows, Python still leaks 1 reference and 1 - # memory block at exit. - self.assertLessEqual(refs, 1, out) + # bpo-46857: on Windows, Python still leaks 1 memory block at exit self.assertIn(blocks, (0, 1), out) diff --git a/Objects/exceptions.c b/Objects/exceptions.c index 977dce5020382..9dbbd40f1de1c 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -15,10 +15,10 @@ /* Compatibility aliases */ -PyObject *PyExc_EnvironmentError = NULL; -PyObject *PyExc_IOError = NULL; +PyObject *PyExc_EnvironmentError = NULL; // borrowed ref +PyObject *PyExc_IOError = NULL; // borrowed ref #ifdef MS_WINDOWS -PyObject *PyExc_WindowsError = NULL; +PyObject *PyExc_WindowsError = NULL; // borrowed ref #endif @@ -3647,10 +3647,8 @@ _PyBuiltins_AddExceptions(PyObject *bltinmod) #define INIT_ALIAS(NAME, TYPE) \ do { \ - Py_INCREF(PyExc_ ## TYPE); \ - Py_XDECREF(PyExc_ ## NAME); \ PyExc_ ## NAME = PyExc_ ## TYPE; \ - if (PyDict_SetItemString(mod_dict, # NAME, PyExc_ ## NAME)) { \ + if (PyDict_SetItemString(mod_dict, # NAME, PyExc_ ## TYPE)) { \ return -1; \ } \ } while (0) diff --git a/Objects/object.c b/Objects/object.c index 77a457223764a..38919ff47a94a 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -61,12 +61,7 @@ Py_ssize_t _Py_RefTotal; Py_ssize_t _Py_GetRefTotal(void) { - PyObject *o; - Py_ssize_t total = _Py_RefTotal; - o = _PySet_Dummy; - if (o != NULL) - total -= Py_REFCNT(o); - return total; + return _Py_RefTotal; } void From webhook-mailer at python.org Sat Feb 26 19:12:37 2022 From: webhook-mailer at python.org (vstinner) Date: Sun, 27 Feb 2022 00:12:37 -0000 Subject: [Python-checkins] bpo-46852: Restore test_getformat() test (GH-31601) Message-ID: https://github.com/python/cpython/commit/5a1c637ec6264790d3cfeef46815c62c32b510f3 commit: 5a1c637ec6264790d3cfeef46815c62c32b510f3 branch: main author: Victor Stinner committer: vstinner date: 2022-02-27T01:12:33+01:00 summary: bpo-46852: Restore test_getformat() test (GH-31601) files: M Lib/test/test_float.py diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py index 3d2a21f1522ea..61950289ae1d2 100644 --- a/Lib/test/test_float.py +++ b/Lib/test/test_float.py @@ -16,9 +16,6 @@ INF = float("inf") NAN = float("nan") -have_getformat = hasattr(float, "__getformat__") -requires_getformat = unittest.skipUnless(have_getformat, - "requires __getformat__") #locate file with float format test values test_dir = os.path.dirname(__file__) or os.curdir @@ -610,6 +607,17 @@ class F(float, H): self.assertEqual(hash(value), object.__hash__(value)) + at unittest.skipUnless(hasattr(float, "__getformat__"), "requires __getformat__") +class FormatFunctionsTestCase(unittest.TestCase): + def test_getformat(self): + self.assertIn(float.__getformat__('double'), + ['unknown', 'IEEE, big-endian', 'IEEE, little-endian']) + self.assertIn(float.__getformat__('float'), + ['unknown', 'IEEE, big-endian', 'IEEE, little-endian']) + self.assertRaises(ValueError, float.__getformat__, 'chicken') + self.assertRaises(TypeError, float.__getformat__, 1) + + BE_DOUBLE_INF = b'\x7f\xf0\x00\x00\x00\x00\x00\x00' LE_DOUBLE_INF = bytes(reversed(BE_DOUBLE_INF)) BE_DOUBLE_NAN = b'\x7f\xf8\x00\x00\x00\x00\x00\x00' From webhook-mailer at python.org Sun Feb 27 09:26:21 2022 From: webhook-mailer at python.org (scoder) Date: Sun, 27 Feb 2022 14:26:21 -0000 Subject: [Python-checkins] bpo-46786: Make ElementTree write the HTML tags embed, source, track, wbr as empty tags (GH-31406) Message-ID: https://github.com/python/cpython/commit/345572a1a0263076081020524016eae867677cac commit: 345572a1a0263076081020524016eae867677cac branch: main author: Jannis Vajen committer: scoder date: 2022-02-27T15:25:54+01:00 summary: bpo-46786: Make ElementTree write the HTML tags embed, source, track, wbr as empty tags (GH-31406) See https://html.spec.whatwg.org/multipage/syntax.html#void-elements for reference. files: A Misc/NEWS.d/next/Library/2022-02-18-12-10-26.bpo-46786.P0xRvS.rst M Lib/test/test_xml_etree.py M Lib/xml/etree/ElementTree.py diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index c5292b5e9ef68..35d901f9d0824 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -1350,8 +1350,9 @@ def test_processinginstruction(self): def test_html_empty_elems_serialization(self): # issue 15970 # from http://www.w3.org/TR/html401/index/elements.html - for element in ['AREA', 'BASE', 'BASEFONT', 'BR', 'COL', 'FRAME', 'HR', - 'IMG', 'INPUT', 'ISINDEX', 'LINK', 'META', 'PARAM']: + for element in ['AREA', 'BASE', 'BASEFONT', 'BR', 'COL', 'EMBED', 'FRAME', + 'HR', 'IMG', 'INPUT', 'ISINDEX', 'LINK', 'META', 'PARAM', + 'SOURCE', 'TRACK', 'WBR']: for elem in [element, element.lower()]: expected = '<%s>' % elem serialized = serialize(ET.XML('<%s />' % elem), method='html') diff --git a/Lib/xml/etree/ElementTree.py b/Lib/xml/etree/ElementTree.py index e9409fd29a115..6059e2f592d2d 100644 --- a/Lib/xml/etree/ElementTree.py +++ b/Lib/xml/etree/ElementTree.py @@ -918,13 +918,9 @@ def _serialize_xml(write, elem, qnames, namespaces, if elem.tail: write(_escape_cdata(elem.tail)) -HTML_EMPTY = ("area", "base", "basefont", "br", "col", "frame", "hr", - "img", "input", "isindex", "link", "meta", "param") - -try: - HTML_EMPTY = set(HTML_EMPTY) -except NameError: - pass +HTML_EMPTY = {"area", "base", "basefont", "br", "col", "embed", "frame", "hr", + "img", "input", "isindex", "link", "meta", "param", "source", + "track", "wbr"} def _serialize_html(write, elem, qnames, namespaces, **kwargs): tag = elem.tag diff --git a/Misc/NEWS.d/next/Library/2022-02-18-12-10-26.bpo-46786.P0xRvS.rst b/Misc/NEWS.d/next/Library/2022-02-18-12-10-26.bpo-46786.P0xRvS.rst new file mode 100644 index 0000000000000..e0384a8558dee --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-18-12-10-26.bpo-46786.P0xRvS.rst @@ -0,0 +1,2 @@ +The HTML serialisation in xml.etree.ElementTree now writes ``embed``, +``source``, ``track`` and ``wbr`` as empty tags, as defined in HTML 5. From webhook-mailer at python.org Mon Feb 28 02:07:26 2022 From: webhook-mailer at python.org (tiran) Date: Mon, 28 Feb 2022 07:07:26 -0000 Subject: [Python-checkins] bpo-46541: Remove unnecessary Py_VISIT (GH-31608) Message-ID: https://github.com/python/cpython/commit/088dd76dba68c2538776d9920607f81e54544cbd commit: 088dd76dba68c2538776d9920607f81e54544cbd branch: main author: Dong-hee Na committer: tiran date: 2022-02-28T08:06:58+01:00 summary: bpo-46541: Remove unnecessary Py_VISIT (GH-31608) files: M Modules/_csv.c M Modules/arraymodule.c diff --git a/Modules/_csv.c b/Modules/_csv.c index ebc3328d0cf0d..991b623d6d6d3 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -60,7 +60,6 @@ _csv_traverse(PyObject *module, visitproc visit, void *arg) Py_VISIT(module_state->dialect_type); Py_VISIT(module_state->reader_type); Py_VISIT(module_state->writer_type); - Py_VISIT(module_state->str_write); return 0; } diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c index 2d6da1aaac85a..73104ce8f1787 100644 --- a/Modules/arraymodule.c +++ b/Modules/arraymodule.c @@ -3034,11 +3034,6 @@ array_traverse(PyObject *module, visitproc visit, void *arg) array_state *state = get_array_state(module); Py_VISIT(state->ArrayType); Py_VISIT(state->ArrayIterType); - Py_VISIT(state->str_read); - Py_VISIT(state->str_write); - Py_VISIT(state->str__array_reconstructor); - Py_VISIT(state->str___dict__); - Py_VISIT(state->str_iter); return 0; } From webhook-mailer at python.org Mon Feb 28 04:07:45 2022 From: webhook-mailer at python.org (corona10) Date: Mon, 28 Feb 2022 09:07:45 -0000 Subject: [Python-checkins] bpo-46541: Remove unneeded visits from sqlite3 (GH-31609) Message-ID: https://github.com/python/cpython/commit/c32aef48533769161e1247927a5b418322e0860c commit: c32aef48533769161e1247927a5b418322e0860c branch: main author: Erlend Egeberg Aasland committer: corona10 date: 2022-02-28T18:07:40+09:00 summary: bpo-46541: Remove unneeded visits from sqlite3 (GH-31609) files: M Modules/_sqlite/module.c diff --git a/Modules/_sqlite/module.c b/Modules/_sqlite/module.c index 35cdcbe0a5c38..70fde4910f6a4 100644 --- a/Modules/_sqlite/module.c +++ b/Modules/_sqlite/module.c @@ -590,13 +590,6 @@ module_traverse(PyObject *module, visitproc visit, void *arg) Py_VISIT(state->lru_cache); Py_VISIT(state->psyco_adapters); - // Interned strings - Py_VISIT(state->str___adapt__); - Py_VISIT(state->str___conform__); - Py_VISIT(state->str_executescript); - Py_VISIT(state->str_finalize); - Py_VISIT(state->str_upper); - return 0; } From webhook-mailer at python.org Mon Feb 28 06:54:27 2022 From: webhook-mailer at python.org (markshannon) Date: Mon, 28 Feb 2022 11:54:27 -0000 Subject: [Python-checkins] bpo-46841: Use inline caching for `UNPACK_SEQUENCE` (GH-31591) Message-ID: https://github.com/python/cpython/commit/424ecab494d538650ba34937cdd710094ccb2275 commit: 424ecab494d538650ba34937cdd710094ccb2275 branch: main author: Brandt Bucher committer: markshannon date: 2022-02-28T11:54:14Z summary: bpo-46841: Use inline caching for `UNPACK_SEQUENCE` (GH-31591) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-25-15-18-40.bpo-46841.tmLpgC.rst M Include/internal/pycore_code.h M Include/opcode.h M Lib/importlib/_bootstrap_external.py M Lib/opcode.py M Python/ceval.c M Python/specialize.c diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index 0c4850f98a318..0e401d61f7673 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -68,9 +68,16 @@ typedef struct { _Py_CODEUNIT counter; } _PyBinaryOpCache; +typedef struct { + _Py_CODEUNIT counter; +} _PyUnpackSequenceCache; + #define INLINE_CACHE_ENTRIES_BINARY_OP \ (sizeof(_PyBinaryOpCache) / sizeof(_Py_CODEUNIT)) +#define INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE \ + (sizeof(_PyUnpackSequenceCache) / sizeof(_Py_CODEUNIT)) + /* Maximum size of code to quicken, in code units. */ #define MAX_SIZE_TO_QUICKEN 5000 @@ -312,7 +319,7 @@ extern void _Py_Specialize_BinaryOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT * int oparg); extern void _Py_Specialize_CompareOp(PyObject *lhs, PyObject *rhs, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); extern void _Py_Specialize_UnpackSequence(PyObject *seq, _Py_CODEUNIT *instr, - SpecializedCacheEntry *cache); + int oparg); /* Deallocator function for static codeobjects used in deepfreeze.py */ extern void _PyStaticCode_Dealloc(PyCodeObject *co); diff --git a/Include/opcode.h b/Include/opcode.h index 0862f96e7de9e..ae21d92a865e5 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -241,6 +241,7 @@ static uint32_t _PyOpcode_Jump[8] = { #define NB_INPLACE_XOR 25 static const uint8_t _PyOpcode_InlineCacheEntries[256] = { + [UNPACK_SEQUENCE] = 1, [BINARY_OP] = 1, }; diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py index 53fe1b8fc217a..be23eeed9df74 100644 --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -388,6 +388,7 @@ def _write_atomic(path, data, mode=0o666): # Python 3.11a5 3479 (Add PUSH_NULL opcode) # Python 3.11a5 3480 (New CALL opcodes, second iteration) # Python 3.11a5 3481 (Use inline CACHE instructions) +# Python 3.11a5 3482 (Use inline caching for UNPACK_SEQUENCE) # Python 3.12 will start with magic number 3500 @@ -402,7 +403,7 @@ def _write_atomic(path, data, mode=0o666): # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # in PC/launcher.c must also be updated. -MAGIC_NUMBER = (3481).to_bytes(2, 'little') + b'\r\n' +MAGIC_NUMBER = (3482).to_bytes(2, 'little') + b'\r\n' _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _PYCACHE = '__pycache__' diff --git a/Lib/opcode.py b/Lib/opcode.py index 84ad002c8dde8..8fa71bf4d180a 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -109,7 +109,7 @@ def jabs_op(name, op, entries=0): name_op('STORE_NAME', 90) # Index in name list name_op('DELETE_NAME', 91) # "" -def_op('UNPACK_SEQUENCE', 92) # Number of tuple items +def_op('UNPACK_SEQUENCE', 92, 1) # Number of tuple items jrel_op('FOR_ITER', 93) def_op('UNPACK_EX', 94) name_op('STORE_ATTR', 95) # Index in name list diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-25-15-18-40.bpo-46841.tmLpgC.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-15-18-40.bpo-46841.tmLpgC.rst new file mode 100644 index 0000000000000..fec18aa51369d --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-15-18-40.bpo-46841.tmLpgC.rst @@ -0,0 +1 @@ +Use inline caching for :opcode:`UNPACK_SEQUENCE`. diff --git a/Python/ceval.c b/Python/ceval.c index 4c0a71b036e8b..fe75782972992 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2758,22 +2758,22 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int } STACK_GROW(oparg); Py_DECREF(seq); + JUMPBY(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE); DISPATCH(); } TARGET(UNPACK_SEQUENCE_ADAPTIVE) { assert(cframe.use_tracing == 0); - SpecializedCacheEntry *cache = GET_CACHE(); - if (cache->adaptive.counter == 0) { + _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)next_instr; + if (cache->counter == 0) { PyObject *seq = TOP(); next_instr--; - _Py_Specialize_UnpackSequence(seq, next_instr, cache); + _Py_Specialize_UnpackSequence(seq, next_instr, oparg); DISPATCH(); } else { STAT_INC(UNPACK_SEQUENCE, deferred); - cache->adaptive.counter--; - oparg = cache->adaptive.original_oparg; + cache->counter--; JUMP_TO_INSTRUCTION(UNPACK_SEQUENCE); } } @@ -2786,36 +2786,37 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int SET_TOP(Py_NewRef(PyTuple_GET_ITEM(seq, 1))); PUSH(Py_NewRef(PyTuple_GET_ITEM(seq, 0))); Py_DECREF(seq); + JUMPBY(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE); NOTRACE_DISPATCH(); } TARGET(UNPACK_SEQUENCE_TUPLE) { PyObject *seq = TOP(); - int len = GET_CACHE()->adaptive.original_oparg; DEOPT_IF(!PyTuple_CheckExact(seq), UNPACK_SEQUENCE); - DEOPT_IF(PyTuple_GET_SIZE(seq) != len, UNPACK_SEQUENCE); + DEOPT_IF(PyTuple_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); STACK_SHRINK(1); PyObject **items = _PyTuple_ITEMS(seq); - while (len--) { - PUSH(Py_NewRef(items[len])); + while (oparg--) { + PUSH(Py_NewRef(items[oparg])); } Py_DECREF(seq); + JUMPBY(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE); NOTRACE_DISPATCH(); } TARGET(UNPACK_SEQUENCE_LIST) { PyObject *seq = TOP(); - int len = GET_CACHE()->adaptive.original_oparg; DEOPT_IF(!PyList_CheckExact(seq), UNPACK_SEQUENCE); - DEOPT_IF(PyList_GET_SIZE(seq) != len, UNPACK_SEQUENCE); + DEOPT_IF(PyList_GET_SIZE(seq) != oparg, UNPACK_SEQUENCE); STAT_INC(UNPACK_SEQUENCE, hit); STACK_SHRINK(1); PyObject **items = _PyList_ITEMS(seq); - while (len--) { - PUSH(Py_NewRef(items[len])); + while (oparg--) { + PUSH(Py_NewRef(items[oparg])); } Py_DECREF(seq); + JUMPBY(INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE); NOTRACE_DISPATCH(); } @@ -5600,7 +5601,7 @@ MISS_WITH_CACHE(CALL) MISS_WITH_INLINE_CACHE(BINARY_OP) MISS_WITH_CACHE(COMPARE_OP) MISS_WITH_CACHE(BINARY_SUBSCR) -MISS_WITH_CACHE(UNPACK_SEQUENCE) +MISS_WITH_INLINE_CACHE(UNPACK_SEQUENCE) MISS_WITH_OPARG_COUNTER(STORE_SUBSCR) LOAD_ATTR_INSTANCE_VALUE_miss: diff --git a/Python/specialize.c b/Python/specialize.c index 1624f1955d42a..b88c5d517bd8f 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -67,7 +67,6 @@ static uint8_t cache_requirements[256] = { [PRECALL] = 2, /* _PyAdaptiveEntry and _PyObjectCache/_PyCallCache */ [STORE_ATTR] = 1, // _PyAdaptiveEntry [COMPARE_OP] = 1, /* _PyAdaptiveEntry */ - [UNPACK_SEQUENCE] = 1, // _PyAdaptiveEntry }; Py_ssize_t _Py_QuickenedCount = 0; @@ -2133,39 +2132,39 @@ unpack_sequence_fail_kind(PyObject *seq) #endif void -_Py_Specialize_UnpackSequence(PyObject *seq, _Py_CODEUNIT *instr, - SpecializedCacheEntry *cache) +_Py_Specialize_UnpackSequence(PyObject *seq, _Py_CODEUNIT *instr, int oparg) { - _PyAdaptiveEntry *adaptive = &cache->adaptive; + assert(_PyOpcode_InlineCacheEntries[UNPACK_SEQUENCE] == + INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE); + _PyUnpackSequenceCache *cache = (_PyUnpackSequenceCache *)(instr + 1); if (PyTuple_CheckExact(seq)) { - if (PyTuple_GET_SIZE(seq) != adaptive->original_oparg) { + if (PyTuple_GET_SIZE(seq) != oparg) { SPECIALIZATION_FAIL(UNPACK_SEQUENCE, SPEC_FAIL_EXPECTED_ERROR); goto failure; } if (PyTuple_GET_SIZE(seq) == 2) { - *instr = _Py_MAKECODEUNIT(UNPACK_SEQUENCE_TWO_TUPLE, - _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(UNPACK_SEQUENCE_TWO_TUPLE, oparg); goto success; } - *instr = _Py_MAKECODEUNIT(UNPACK_SEQUENCE_TUPLE, _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(UNPACK_SEQUENCE_TUPLE, oparg); goto success; } if (PyList_CheckExact(seq)) { - if (PyList_GET_SIZE(seq) != adaptive->original_oparg) { + if (PyList_GET_SIZE(seq) != oparg) { SPECIALIZATION_FAIL(UNPACK_SEQUENCE, SPEC_FAIL_EXPECTED_ERROR); goto failure; } - *instr = _Py_MAKECODEUNIT(UNPACK_SEQUENCE_LIST, _Py_OPARG(*instr)); + *instr = _Py_MAKECODEUNIT(UNPACK_SEQUENCE_LIST, oparg); goto success; } SPECIALIZATION_FAIL(UNPACK_SEQUENCE, unpack_sequence_fail_kind(seq)); failure: STAT_INC(UNPACK_SEQUENCE, failure); - cache_backoff(adaptive); + cache->counter = ADAPTIVE_CACHE_BACKOFF; return; success: STAT_INC(UNPACK_SEQUENCE, success); - adaptive->counter = initial_counter_value(); + cache->counter = initial_counter_value(); } #ifdef Py_STATS From webhook-mailer at python.org Mon Feb 28 07:06:48 2022 From: webhook-mailer at python.org (zooba) Date: Mon, 28 Feb 2022 12:06:48 -0000 Subject: [Python-checkins] bpo-46567: Add Tcl/Tk build for Windows ARM64 (GH-31574) Message-ID: https://github.com/python/cpython/commit/da7d99a4de72aac8d436cecedf16ab2676f9b785 commit: da7d99a4de72aac8d436cecedf16ab2676f9b785 branch: main author: Steve Dower committer: zooba date: 2022-02-28T12:06:43Z summary: bpo-46567: Add Tcl/Tk build for Windows ARM64 (GH-31574) files: A Misc/NEWS.d/next/Windows/2022-02-25-01-22-31.bpo-46567.37WEue.rst M .azure-pipelines/tcltk-build.yml M .azure-pipelines/windows-release/msi-steps.yml M .azure-pipelines/windows-release/stage-build.yml M .azure-pipelines/windows-release/stage-layout-full.yml M .azure-pipelines/windows-release/stage-layout-msix.yml M PCbuild/get_externals.bat M PCbuild/pcbuild.proj M PCbuild/prepare_tcltk.bat M PCbuild/tcl.vcxproj M PCbuild/tcltk.props M PCbuild/tix.vcxproj M PCbuild/tk.vcxproj M Tools/msi/bundle/Default.ARM64.xsl M Tools/msi/bundle/bundle.targets M Tools/msi/bundle/bundle.wxs diff --git a/.azure-pipelines/tcltk-build.yml b/.azure-pipelines/tcltk-build.yml index 27968e886cc14..f9e50d3711a46 100644 --- a/.azure-pipelines/tcltk-build.yml +++ b/.azure-pipelines/tcltk-build.yml @@ -60,6 +60,12 @@ jobs: & "$(msbuild)" PCbuild\tix.vcxproj "@msbuild.rsp" /p:Platform=x64 /p:tcltkDir="$(OutDir)\amd64" displayName: 'Build for amd64' + - powershell: | + & "$(msbuild)" PCbuild\tcl.vcxproj "@msbuild.rsp" /p:Platform=ARM64 /p:tcltkDir="$(OutDir)\arm64" + & "$(msbuild)" PCbuild\tk.vcxproj "@msbuild.rsp" /p:Platform=ARM64 /p:tcltkDir="$(OutDir)\arm64" + & "$(msbuild)" PCbuild\tix.vcxproj "@msbuild.rsp" /p:Platform=ARM64 /p:tcltkDir="$(OutDir)\arm64" + displayName: 'Build for arm64' + - publish: '$(OutDir)' artifact: 'tcltk' displayName: 'Publishing tcltk' diff --git a/.azure-pipelines/windows-release/msi-steps.yml b/.azure-pipelines/windows-release/msi-steps.yml index ef98d56c78363..3c08a0660abe9 100644 --- a/.azure-pipelines/windows-release/msi-steps.yml +++ b/.azure-pipelines/windows-release/msi-steps.yml @@ -71,6 +71,13 @@ steps: artifactName: tcltk_lib_amd64 targetPath: $(Build.BinariesDirectory)\tcltk_lib_amd64 + - task: DownloadPipelineArtifact at 1 + displayName: 'Download artifact: tcltk_lib_arm64' + condition: and(succeeded(), eq(variables['PublishARM64'], 'true')) + inputs: + artifactName: tcltk_lib_arm64 + targetPath: $(Build.BinariesDirectory)\tcltk_lib_arm64 + - powershell: | copy $(Build.BinariesDirectory)\amd64\Activate.ps1 Lib\venv\scripts\common\Activate.ps1 -Force displayName: 'Copy signed files into sources' @@ -107,7 +114,6 @@ steps: PYTHONHOME: $(Build.SourcesDirectory) TclTkLibraryDir: $(Build.BinariesDirectory)\tcltk_lib_win32 BuildForRelease: true - SuppressMinGWLib: true - script: | %MSBUILD% Tools\msi\bundle\releaselocal.wixproj /t:Rebuild /p:RebuildAll=true @@ -120,7 +126,6 @@ steps: PYTHONHOME: $(Build.SourcesDirectory) TclTkLibraryDir: $(Build.BinariesDirectory)\tcltk_lib_amd64 BuildForRelease: true - SuppressMinGWLib: true - script: | %MSBUILD% Tools\msi\bundle\releaselocal.wixproj /t:Rebuild /p:RebuildAll=true @@ -132,8 +137,8 @@ steps: PYTHON: $(Build.BinariesDirectory)\win32\python.exe PythonForBuild: $(Build.BinariesDirectory)\win32\python.exe PYTHONHOME: $(Build.SourcesDirectory) + TclTkLibraryDir: $(Build.BinariesDirectory)\tcltk_lib_arm64 BuildForRelease: true - SuppressMinGWLib: true - task: CopyFiles at 2 displayName: 'Assemble artifact: msi (win32)' diff --git a/.azure-pipelines/windows-release/stage-build.yml b/.azure-pipelines/windows-release/stage-build.yml index f70414ba21145..e45034f650fb9 100644 --- a/.azure-pipelines/windows-release/stage-build.yml +++ b/.azure-pipelines/windows-release/stage-build.yml @@ -166,6 +166,13 @@ jobs: platform: x64 msbuildArguments: /t:CopyTclTkLib /p:OutDir="$(Build.ArtifactStagingDirectory)\tcl_amd64" + - task: MSBuild at 1 + displayName: 'Copy Tcl/Tk lib for publish' + inputs: + solution: PCbuild\tcltk.props + platform: ARM64 + msbuildArguments: /t:CopyTclTkLib /p:OutDir="$(Build.ArtifactStagingDirectory)\tcl_arm64" + - task: PublishPipelineArtifact at 0 displayName: 'Publish artifact: tcltk_lib_win32' inputs: @@ -177,3 +184,9 @@ jobs: inputs: targetPath: '$(Build.ArtifactStagingDirectory)\tcl_amd64' artifactName: tcltk_lib_amd64 + + - task: PublishPipelineArtifact at 0 + displayName: 'Publish artifact: tcltk_lib_arm64' + inputs: + targetPath: '$(Build.ArtifactStagingDirectory)\tcl_arm64' + artifactName: tcltk_lib_arm64 diff --git a/.azure-pipelines/windows-release/stage-layout-full.yml b/.azure-pipelines/windows-release/stage-layout-full.yml index 0ba2fc017d987..3546df60e4e03 100644 --- a/.azure-pipelines/windows-release/stage-layout-full.yml +++ b/.azure-pipelines/windows-release/stage-layout-full.yml @@ -26,6 +26,7 @@ jobs: HostArch: amd64 Python: $(Build.BinariesDirectory)\bin_amd64\python.exe PYTHONHOME: $(Build.SourcesDirectory) + TclLibrary: $(Build.BinariesDirectory)\tcltk_lib\tcl8 steps: - template: ./checkout.yml diff --git a/.azure-pipelines/windows-release/stage-layout-msix.yml b/.azure-pipelines/windows-release/stage-layout-msix.yml index 6efd327bdb32e..913bfcd919704 100644 --- a/.azure-pipelines/windows-release/stage-layout-msix.yml +++ b/.azure-pipelines/windows-release/stage-layout-msix.yml @@ -25,6 +25,7 @@ jobs: HostArch: amd64 Python: $(Build.BinariesDirectory)\bin_amd64\python.exe PYTHONHOME: $(Build.SourcesDirectory) + TclLibrary: $(Build.BinariesDirectory)\tcltk_lib\tcl8 steps: - template: ./checkout.yml diff --git a/Misc/NEWS.d/next/Windows/2022-02-25-01-22-31.bpo-46567.37WEue.rst b/Misc/NEWS.d/next/Windows/2022-02-25-01-22-31.bpo-46567.37WEue.rst new file mode 100644 index 0000000000000..10a2b764b7fce --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2022-02-25-01-22-31.bpo-46567.37WEue.rst @@ -0,0 +1,2 @@ +Adds Tcl and Tk support for Windows ARM64. This also adds IDLE to the +installation. diff --git a/PCbuild/get_externals.bat b/PCbuild/get_externals.bat index 5b0fe640eb84e..d4e052ef32c82 100644 --- a/PCbuild/get_externals.bat +++ b/PCbuild/get_externals.bat @@ -55,8 +55,8 @@ set libraries=%libraries% bzip2-1.0.6 if NOT "%IncludeLibffiSrc%"=="false" set libraries=%libraries% libffi-3.4.2 if NOT "%IncludeSSLSrc%"=="false" set libraries=%libraries% openssl-1.1.1m set libraries=%libraries% sqlite-3.37.2.0 -if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tcl-core-8.6.12.0 -if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tk-8.6.12.0 +if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tcl-core-8.6.12.1 +if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tk-8.6.12.1 if NOT "%IncludeTkinterSrc%"=="false" set libraries=%libraries% tix-8.4.3.6 set libraries=%libraries% xz-5.2.2 set libraries=%libraries% zlib-1.2.11 @@ -78,7 +78,7 @@ echo.Fetching external binaries... set binaries= if NOT "%IncludeLibffi%"=="false" set binaries=%binaries% libffi-3.4.2 if NOT "%IncludeSSL%"=="false" set binaries=%binaries% openssl-bin-1.1.1m -if NOT "%IncludeTkinter%"=="false" set binaries=%binaries% tcltk-8.6.12.0 +if NOT "%IncludeTkinter%"=="false" set binaries=%binaries% tcltk-8.6.12.1 if NOT "%IncludeSSLSrc%"=="false" set binaries=%binaries% nasm-2.11.06 for %%b in (%binaries%) do ( diff --git a/PCbuild/pcbuild.proj b/PCbuild/pcbuild.proj index f32422a0acf44..2ba0627b83369 100644 --- a/PCbuild/pcbuild.proj +++ b/PCbuild/pcbuild.proj @@ -73,7 +73,7 @@ - + diff --git a/PCbuild/prepare_tcltk.bat b/PCbuild/prepare_tcltk.bat index 77075af94f2c6..4a43ed1582ce3 100644 --- a/PCbuild/prepare_tcltk.bat +++ b/PCbuild/prepare_tcltk.bat @@ -53,3 +53,7 @@ call "%PCBUILD%\get_externals.bat" --tkinter-src %ORG_SETTING% %MSBUILD% "%PCBUILD%\tcl.vcxproj" /p:Configuration=Release /p:Platform=x64 %MSBUILD% "%PCBUILD%\tk.vcxproj" /p:Configuration=Release /p:Platform=x64 %MSBUILD% "%PCBUILD%\tix.vcxproj" /p:Configuration=Release /p:Platform=x64 + +%MSBUILD% "%PCBUILD%\tcl.vcxproj" /p:Configuration=Release /p:Platform=ARM64 +%MSBUILD% "%PCBUILD%\tk.vcxproj" /p:Configuration=Release /p:Platform=ARM64 +%MSBUILD% "%PCBUILD%\tix.vcxproj" /p:Configuration=Release /p:Platform=ARM64 diff --git a/PCbuild/tcl.vcxproj b/PCbuild/tcl.vcxproj index 4536cbc925bd9..ab68db9210fbe 100644 --- a/PCbuild/tcl.vcxproj +++ b/PCbuild/tcl.vcxproj @@ -33,6 +33,10 @@ Release x64 + + Release + ARM64 + {B5FD6F1D-129E-4BFF-9340-03606FAC7283} @@ -59,8 +63,8 @@ setlocal set VCINSTALLDIR=$(VCInstallDir) cd /D "$(tclDir)win" -nmake -f makefile.vc MACHINE=$(TclMachine) OPTS=$(TclOpts) $(TclDirs) $(DebugFlags) $(WarningsFlags) core shell dlls -nmake -f makefile.vc MACHINE=$(TclMachine) OPTS=$(TclOpts) $(TclDirs) $(DebugFlags) $(WarningsFlags) install-binaries install-libraries +nmake -f makefile.vc MACHINE=$(TclMachine) OPTS=$(TclOpts) $(TclDirs) $(DebugFlags) $(WarningsFlags) TCLSH_NATIVE="$(tclWin32Exe)" core shell dlls +nmake -f makefile.vc MACHINE=$(TclMachine) OPTS=$(TclOpts) $(TclDirs) $(DebugFlags) $(WarningsFlags) TCLSH_NATIVE="$(tclWin32Exe)" install-binaries install-libraries copy /Y ..\license.terms "$(OutDir)\tcllicense.terms" diff --git a/PCbuild/tcltk.props b/PCbuild/tcltk.props index 72cffc3ea1da6..7fd43e8279e8e 100644 --- a/PCbuild/tcltk.props +++ b/PCbuild/tcltk.props @@ -5,7 +5,7 @@ 8 6 12 - 0 + 1 $(TclMajorVersion) $(TclMinorVersion) $(TclPatchLevel) @@ -18,6 +18,9 @@ $(ExternalsDir)tk-$(TkMajorVersion).$(TkMinorVersion).$(TkPatchLevel).$(TkRevision)\ $(ExternalsDir)tix-$(TixMajorVersion).$(TixMinorVersion).$(TixPatchLevel).$(TixRevision)\ $(ExternalsDir)tcltk-$(TclMajorVersion).$(TclMinorVersion).$(TclPatchLevel).$(TclRevision)\$(ArchName)\ + $(tcltkDir)\bin\tclsh$(TclMajorVersion)$(TclMinorVersion)t.exe + $(tcltkDir)\..\win32\bin\tclsh$(TclMajorVersion)$(TclMinorVersion)t.exe + tcl$(TclMajorVersion)$(TclMinorVersion)t$(TclDebugExt).dll tcl$(TclMajorVersion)$(TclMinorVersion)t$(TclDebugExt).lib diff --git a/PCbuild/tix.vcxproj b/PCbuild/tix.vcxproj index 51682094cb844..48abcd27c8775 100644 --- a/PCbuild/tix.vcxproj +++ b/PCbuild/tix.vcxproj @@ -33,6 +33,10 @@ Release x64 + + Release + ARM64 + {C5A3E7FB-9695-4B2E-960B-1D9F43F1E555} @@ -53,7 +57,7 @@ - BUILDDIRTOP="$(BuildDirTop)" TCL_DIR="$(tclDir.TrimEnd(`\`))" TK_DIR="$(tkDir.TrimEnd(`\`))" INSTALL_DIR="$(OutDir.TrimEnd(`\`))" + BUILDDIRTOP="$(BuildDirTop)" TCL_DIR="$(tclDir.TrimEnd(`\`))" TK_DIR="$(tkDir.TrimEnd(`\`))" INSTALL_DIR="$(OutDir.TrimEnd(`\`))" TCLSH_EXE="$(tclWin32Exe)" DEBUG=1 NODEBUG=0 TCL_DBGX=g TK_DBGX=g DEBUG=0 NODEBUG=1 -c -W3 -nologo -MD -wd4028 -wd4090 -wd4244 -wd4267 -wd4312 diff --git a/PCbuild/tk.vcxproj b/PCbuild/tk.vcxproj index 70b5459a081c2..b111969ca5de6 100644 --- a/PCbuild/tk.vcxproj +++ b/PCbuild/tk.vcxproj @@ -32,6 +32,10 @@ Release x64 + + + Release + ARM64 @@ -60,8 +64,8 @@ setlocal set VCINSTALLDIR=$(VCInstallDir) cd /D "$(tkDir)win" -nmake /nologo -f makefile.vc RC=rc MACHINE=$(TclMachine) OPTS=$(TkOpts) $(TkDirs) $(DebugFlags) $(WarningsFlags) all -nmake /nologo -f makefile.vc RC=rc MACHINE=$(TclMachine) OPTS=$(TkOpts) $(TkDirs) $(DebugFlags) $(WarningsFlags) install-binaries install-libraries +nmake /nologo -f makefile.vc RC=rc MACHINE=$(TclMachine) OPTS=$(TkOpts) $(TkDirs) $(DebugFlags) $(WarningsFlags) TCLSH_NATIVE="$(tclWin32Exe)" all +nmake /nologo -f makefile.vc RC=rc MACHINE=$(TclMachine) OPTS=$(TkOpts) $(TkDirs) $(DebugFlags) $(WarningsFlags) TCLSH_NATIVE="$(tclWin32Exe)" install-binaries install-libraries copy /Y ..\license.terms "$(OutDir)\tklicense.terms" diff --git a/Tools/msi/bundle/Default.ARM64.xsl b/Tools/msi/bundle/Default.ARM64.xsl index 0dd1c9e6a3d52..f63da4e7274cb 100644 --- a/Tools/msi/bundle/Default.ARM64.xsl +++ b/Tools/msi/bundle/Default.ARM64.xsl @@ -13,14 +13,11 @@ [TargetDir] -Includes pip and documentation +Includes IDLE, pip and documentation Creates shortcuts but no file associations (The 'py' launcher is currently unavailable on ARM64.) - - (tcl/tk and IDLE are currently unavailable on ARM64.) - \ No newline at end of file diff --git a/Tools/msi/bundle/bundle.targets b/Tools/msi/bundle/bundle.targets index 4f4306f7b61f0..89a5960a50efe 100644 --- a/Tools/msi/bundle/bundle.targets +++ b/Tools/msi/bundle/bundle.targets @@ -69,7 +69,7 @@ - + diff --git a/Tools/msi/bundle/bundle.wxs b/Tools/msi/bundle/bundle.wxs index 5c9fd6dca394c..340c72acf408b 100644 --- a/Tools/msi/bundle/bundle.wxs +++ b/Tools/msi/bundle/bundle.wxs @@ -77,12 +77,7 @@ - - - - - @@ -122,8 +117,8 @@ - + From webhook-mailer at python.org Mon Feb 28 07:56:36 2022 From: webhook-mailer at python.org (markshannon) Date: Mon, 28 Feb 2022 12:56:36 -0000 Subject: [Python-checkins] bpo-46841: Move the cache for `LOAD_GLOBAL` inline. (GH-31575) Message-ID: https://github.com/python/cpython/commit/4558af5a8f8e56a9b0dc11f6e834c47e0fd05f9e commit: 4558af5a8f8e56a9b0dc11f6e834c47e0fd05f9e branch: main author: Mark Shannon committer: markshannon date: 2022-02-28T12:56:29Z summary: bpo-46841: Move the cache for `LOAD_GLOBAL` inline. (GH-31575) files: A Misc/NEWS.d/next/Core and Builtins/2022-02-25-13-18-18.bpo-46841.86QiQu.rst M Include/internal/pycore_code.h M Include/opcode.h M Lib/importlib/_bootstrap_external.py M Lib/opcode.py M Lib/test/test_dis.py M Python/ceval.c M Python/compile.c M Python/specialize.c M Tools/scripts/generate_opcode_h.py diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h index 0e401d61f7673..dfa15b8cd61e4 100644 --- a/Include/internal/pycore_code.h +++ b/Include/internal/pycore_code.h @@ -26,11 +26,6 @@ typedef struct { uint32_t dk_version; } _PyAttrCache; -typedef struct { - uint32_t module_keys_version; - uint32_t builtin_keys_version; -} _PyLoadGlobalCache; - typedef struct { /* Borrowed ref in LOAD_METHOD */ PyObject *obj; @@ -57,23 +52,35 @@ typedef union { _PyEntryZero zero; _PyAdaptiveEntry adaptive; _PyAttrCache attr; - _PyLoadGlobalCache load_global; _PyObjectCache obj; _PyCallCache call; } SpecializedCacheEntry; #define INSTRUCTIONS_PER_ENTRY (sizeof(SpecializedCacheEntry)/sizeof(_Py_CODEUNIT)) +/* Inline caches */ + +#define CACHE_ENTRIES(cache) (sizeof(cache)/sizeof(_Py_CODEUNIT)) + +typedef struct { + _Py_CODEUNIT counter; + _Py_CODEUNIT index; + _Py_CODEUNIT module_keys_version; + _Py_CODEUNIT _m1; + _Py_CODEUNIT builtin_keys_version; +} _PyLoadGlobalCache; + +#define INLINE_CACHE_ENTRIES_LOAD_GLOBAL CACHE_ENTRIES(_PyLoadGlobalCache) + typedef struct { _Py_CODEUNIT counter; } _PyBinaryOpCache; +#define INLINE_CACHE_ENTRIES_BINARY_OP CACHE_ENTRIES(_PyBinaryOpCache) typedef struct { _Py_CODEUNIT counter; } _PyUnpackSequenceCache; -#define INLINE_CACHE_ENTRIES_BINARY_OP \ - (sizeof(_PyBinaryOpCache) / sizeof(_Py_CODEUNIT)) #define INLINE_CACHE_ENTRIES_UNPACK_SEQUENCE \ (sizeof(_PyUnpackSequenceCache) / sizeof(_Py_CODEUNIT)) @@ -307,7 +314,7 @@ cache_backoff(_PyAdaptiveEntry *entry) { extern int _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); extern int _Py_Specialize_StoreAttr(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); -extern int _Py_Specialize_LoadGlobal(PyObject *globals, PyObject *builtins, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); +extern int _Py_Specialize_LoadGlobal(PyObject *globals, PyObject *builtins, _Py_CODEUNIT *instr, PyObject *name); extern int _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache); extern int _Py_Specialize_BinarySubscr(PyObject *sub, PyObject *container, _Py_CODEUNIT *instr, SpecializedCacheEntry *cache); extern int _Py_Specialize_StoreSubscr(PyObject *container, PyObject *sub, _Py_CODEUNIT *instr); @@ -388,6 +395,38 @@ extern PyObject* _Py_GetSpecializationStats(void); #define OBJECT_STAT_INC(name) ((void)0) #endif +// Cache values are only valid in memory, so use native endianness. +#ifdef WORDS_BIGENDIAN + +static inline void +write32(uint16_t *p, uint32_t val) +{ + p[0] = val >> 16; + p[1] = (uint16_t)val; +} + +static inline uint32_t +read32(uint16_t *p) +{ + return (p[0] << 16) | p[1]; +} + +#else + +static inline void +write32(uint16_t *p, uint32_t val) +{ + p[0] = (uint16_t)val; + p[1] = val >> 16; +} + +static inline uint32_t +read32(uint16_t *p) +{ + return p[0] | (p[1] << 16); +} + +#endif #ifdef __cplusplus } diff --git a/Include/opcode.h b/Include/opcode.h index ae21d92a865e5..99480def30da5 100644 --- a/Include/opcode.h +++ b/Include/opcode.h @@ -185,8 +185,11 @@ extern "C" { #define STORE_FAST__STORE_FAST 175 #define LOAD_FAST__LOAD_ATTR_INSTANCE_VALUE 176 #define DO_TRACING 255 -#ifdef NEED_OPCODE_JUMP_TABLES -static uint32_t _PyOpcode_RelativeJump[8] = { + +extern const uint8_t _PyOpcode_InlineCacheEntries[256]; + +#ifdef NEED_OPCODE_TABLES +static const uint32_t _PyOpcode_RelativeJump[8] = { 0U, 0U, 536870912U, @@ -196,7 +199,7 @@ static uint32_t _PyOpcode_RelativeJump[8] = { 0U, 0U, }; -static uint32_t _PyOpcode_Jump[8] = { +static const uint32_t _PyOpcode_Jump[8] = { 0U, 0U, 536870912U, @@ -206,6 +209,12 @@ static uint32_t _PyOpcode_Jump[8] = { 0U, 0U, }; + +const uint8_t _PyOpcode_InlineCacheEntries[256] = { + [UNPACK_SEQUENCE] = 1, + [LOAD_GLOBAL] = 5, + [BINARY_OP] = 1, +}; #endif /* OPCODE_TABLES */ #define HAS_CONST(op) (false\ @@ -240,11 +249,6 @@ static uint32_t _PyOpcode_Jump[8] = { #define NB_INPLACE_TRUE_DIVIDE 24 #define NB_INPLACE_XOR 25 -static const uint8_t _PyOpcode_InlineCacheEntries[256] = { - [UNPACK_SEQUENCE] = 1, - [BINARY_OP] = 1, -}; - #define HAS_ARG(op) ((op) >= HAVE_ARGUMENT) /* Reserve some bytecodes for internal use in the compiler. diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py index be23eeed9df74..f051dfe9492f5 100644 --- a/Lib/importlib/_bootstrap_external.py +++ b/Lib/importlib/_bootstrap_external.py @@ -387,8 +387,8 @@ def _write_atomic(path, data, mode=0o666): # Python 3.11a5 3478 (New CALL opcodes) # Python 3.11a5 3479 (Add PUSH_NULL opcode) # Python 3.11a5 3480 (New CALL opcodes, second iteration) -# Python 3.11a5 3481 (Use inline CACHE instructions) -# Python 3.11a5 3482 (Use inline caching for UNPACK_SEQUENCE) +# Python 3.11a5 3481 (Use inline cache for BINARY_OP) +# Python 3.11a5 3482 (Use inline caching for UNPACK_SEQUENCE and LOAD_GLOBAL) # Python 3.12 will start with magic number 3500 diff --git a/Lib/opcode.py b/Lib/opcode.py index 8fa71bf4d180a..fb2d24fd1f007 100644 --- a/Lib/opcode.py +++ b/Lib/opcode.py @@ -135,7 +135,7 @@ def jabs_op(name, op, entries=0): jabs_op('JUMP_ABSOLUTE', 113) # "" jabs_op('POP_JUMP_IF_FALSE', 114) # "" jabs_op('POP_JUMP_IF_TRUE', 115) # "" -name_op('LOAD_GLOBAL', 116) # Index in name list +name_op('LOAD_GLOBAL', 116, 5) # Index in name list def_op('IS_OP', 117) def_op('CONTAINS_OP', 118) def_op('RERAISE', 119) @@ -198,6 +198,7 @@ def jabs_op(name, op, entries=0): def_op('KW_NAMES', 172) hasconst.append(172) + del def_op, name_op, jrel_op, jabs_op _nb_ops = [ diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index e478ca966612b..a13e0f6edb874 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -149,10 +149,10 @@ def bug708901(): %3d PRECALL 2 CALL 2 GET_ITER - >> FOR_ITER 2 (to 22) + >> FOR_ITER 2 (to 32) STORE_FAST 0 (res) -%3d JUMP_ABSOLUTE 8 (to 16) +%3d JUMP_ABSOLUTE 13 (to 26) %3d >> LOAD_CONST 0 (None) RETURN_VALUE @@ -375,7 +375,7 @@ def bug42562(): >> PUSH_EXC_INFO %3d LOAD_GLOBAL 0 (Exception) - JUMP_IF_NOT_EXC_MATCH 26 (to 52) + JUMP_IF_NOT_EXC_MATCH 31 (to 62) STORE_FAST 0 (e) %3d LOAD_FAST 0 (e) @@ -1156,18 +1156,18 @@ def _prepare_test_cases(): Instruction(opname='STORE_FAST', opcode=125, arg=2, argval='f', argrepr='f', offset=18, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=20, starts_line=7, is_jump_target=False, positions=None), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='print', argrepr='print', offset=22, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=0, argval='a', argrepr='a', offset=24, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=1, argval='b', argrepr='b', offset=26, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='', argrepr="''", offset=28, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=30, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='BUILD_LIST', opcode=103, arg=0, argval=0, argrepr='', offset=32, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='BUILD_MAP', opcode=105, arg=0, argval=0, argrepr='', offset=34, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Hello world!', argrepr="'Hello world!'", offset=36, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=7, argval=7, argrepr='', offset=38, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=7, argval=7, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=42, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='f', argrepr='f', offset=44, starts_line=8, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=46, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=0, argval='a', argrepr='a', offset=34, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=1, argval='b', argrepr='b', offset=36, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='', argrepr="''", offset=38, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=40, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='BUILD_LIST', opcode=103, arg=0, argval=0, argrepr='', offset=42, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='BUILD_MAP', opcode=105, arg=0, argval=0, argrepr='', offset=44, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Hello world!', argrepr="'Hello world!'", offset=46, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=7, argval=7, argrepr='', offset=48, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=7, argval=7, argrepr='', offset=50, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=52, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='f', argrepr='f', offset=54, starts_line=8, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=56, starts_line=None, is_jump_target=False, positions=None), ] expected_opinfo_f = [ @@ -1186,15 +1186,15 @@ def _prepare_test_cases(): Instruction(opname='STORE_FAST', opcode=125, arg=2, argval='inner', argrepr='inner', offset=24, starts_line=None, is_jump_target=False, positions=None), Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=26, starts_line=5, is_jump_target=False, positions=None), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='print', argrepr='print', offset=28, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=3, argval='a', argrepr='a', offset=30, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=4, argval='b', argrepr='b', offset=32, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=0, argval='c', argrepr='c', offset=34, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=1, argval='d', argrepr='d', offset=36, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=4, argval=4, argrepr='', offset=38, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=4, argval=4, argrepr='', offset=40, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=42, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='inner', argrepr='inner', offset=44, starts_line=6, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=46, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=3, argval='a', argrepr='a', offset=40, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=4, argval='b', argrepr='b', offset=42, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=0, argval='c', argrepr='c', offset=44, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=1, argval='d', argrepr='d', offset=46, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=4, argval=4, argrepr='', offset=48, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=4, argval=4, argrepr='', offset=50, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=52, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=2, argval='inner', argrepr='inner', offset=54, starts_line=6, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=56, starts_line=None, is_jump_target=False, positions=None), ] expected_opinfo_inner = [ @@ -1202,160 +1202,160 @@ def _prepare_test_cases(): Instruction(opname='RESUME', opcode=151, arg=0, argval=0, argrepr='', offset=2, starts_line=3, is_jump_target=False, positions=None), Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=4, starts_line=4, is_jump_target=False, positions=None), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='print', argrepr='print', offset=6, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=2, argval='a', argrepr='a', offset=8, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=3, argval='b', argrepr='b', offset=10, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=4, argval='c', argrepr='c', offset=12, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_DEREF', opcode=137, arg=5, argval='d', argrepr='d', offset=14, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='e', argrepr='e', offset=16, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=1, argval='f', argrepr='f', offset=18, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=6, argval=6, argrepr='', offset=20, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=6, argval=6, argrepr='', offset=22, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=24, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=26, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=28, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=2, argval='a', argrepr='a', offset=18, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=3, argval='b', argrepr='b', offset=20, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=4, argval='c', argrepr='c', offset=22, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_DEREF', opcode=137, arg=5, argval='d', argrepr='d', offset=24, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='e', argrepr='e', offset=26, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=1, argval='f', argrepr='f', offset=28, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=6, argval=6, argrepr='', offset=30, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=6, argval=6, argrepr='', offset=32, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=34, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=36, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=38, starts_line=None, is_jump_target=False, positions=None), ] expected_opinfo_jumpy = [ Instruction(opname='RESUME', opcode=151, arg=0, argval=0, argrepr='', offset=0, starts_line=1, is_jump_target=False, positions=None), Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=2, starts_line=3, is_jump_target=False, positions=None), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='range', argrepr='range', offset=4, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval=10, argrepr='10', offset=6, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=8, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=10, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='GET_ITER', opcode=68, arg=None, argval=None, argrepr='', offset=12, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='FOR_ITER', opcode=93, arg=19, argval=54, argrepr='to 54', offset=14, starts_line=None, is_jump_target=True, positions=None), - Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=16, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=18, starts_line=4, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=20, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=22, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=24, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=26, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=28, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=30, starts_line=5, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=32, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=34, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=20, argval=40, argrepr='to 40', offset=36, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=7, argval=14, argrepr='to 14', offset=38, starts_line=6, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=40, starts_line=7, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=42, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=44, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=26, argval=52, argrepr='to 52', offset=46, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=48, starts_line=8, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=7, argval=66, argrepr='to 66', offset=50, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=7, argval=14, argrepr='to 14', offset=52, starts_line=7, is_jump_target=True, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=54, starts_line=10, is_jump_target=True, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=56, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='I can haz else clause?', argrepr="'I can haz else clause?'", offset=58, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=60, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=62, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=64, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=66, starts_line=11, is_jump_target=True, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=58, argval=116, argrepr='to 116', offset=68, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=70, starts_line=12, is_jump_target=True, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=72, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=74, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=76, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=78, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=80, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=82, starts_line=13, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=84, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='BINARY_OP', opcode=122, arg=23, argval=23, argrepr='-=', offset=86, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=90, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=92, starts_line=14, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=94, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=96, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=51, argval=102, argrepr='to 102', offset=98, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=33, argval=66, argrepr='to 66', offset=100, starts_line=15, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=102, starts_line=16, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=104, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=106, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=56, argval=112, argrepr='to 112', offset=108, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=8, argval=128, argrepr='to 128', offset=110, starts_line=17, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=112, starts_line=11, is_jump_target=True, positions=None), - Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=35, argval=70, argrepr='to 70', offset=114, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=116, starts_line=19, is_jump_target=True, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=118, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=120, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=122, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=124, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=126, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=128, starts_line=20, is_jump_target=True, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=130, starts_line=21, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=132, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='BINARY_OP', opcode=122, arg=11, argval=11, argrepr='/', offset=134, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=140, starts_line=25, is_jump_target=False, positions=None), - Instruction(opname='BEFORE_WITH', opcode=53, arg=None, argval=None, argrepr='', offset=142, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=144, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=146, starts_line=26, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=148, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Never reach this', argrepr="'Never reach this'", offset=150, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=152, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=154, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=156, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=158, starts_line=25, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=160, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=162, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=2, argval=2, argrepr='', offset=164, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=2, argval=2, argrepr='', offset=166, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=168, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=194, argrepr='to 194', offset=170, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=174, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=93, argval=186, argrepr='to 186', offset=176, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=2, argval=2, argrepr='', offset=178, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=180, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=184, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=True, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=190, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=192, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=16, argval=228, argrepr='to 228', offset=194, starts_line=None, is_jump_target=True, positions=None), - Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=196, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=198, starts_line=22, is_jump_target=False, positions=None), - Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=110, argval=220, argrepr='to 220', offset=200, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=202, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=204, starts_line=23, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=206, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=208, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=210, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=212, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=214, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=216, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='JUMP_FORWARD', opcode=110, arg=12, argval=244, argrepr='to 244', offset=218, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=220, starts_line=22, is_jump_target=True, positions=None), - Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=222, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=224, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=226, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=228, starts_line=28, is_jump_target=True, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=230, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=232, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=234, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=236, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=238, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=240, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=242, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=244, starts_line=23, is_jump_target=True, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=246, starts_line=28, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=248, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=250, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=252, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=254, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=256, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=258, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=260, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=262, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=264, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=266, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=268, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=270, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=272, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=274, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=276, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=278, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=280, starts_line=None, is_jump_target=False, positions=None), - Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=282, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval=10, argrepr='10', offset=16, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=18, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=20, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='GET_ITER', opcode=68, arg=None, argval=None, argrepr='', offset=22, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='FOR_ITER', opcode=93, arg=24, argval=74, argrepr='to 74', offset=24, starts_line=None, is_jump_target=True, positions=None), + Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=26, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=28, starts_line=4, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=30, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=42, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=44, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=46, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=48, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=50, starts_line=5, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=52, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=54, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=30, argval=60, argrepr='to 60', offset=56, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=12, argval=24, argrepr='to 24', offset=58, starts_line=6, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=60, starts_line=7, is_jump_target=True, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=62, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=64, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=36, argval=72, argrepr='to 72', offset=66, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=68, starts_line=8, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=12, argval=96, argrepr='to 96', offset=70, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=12, argval=24, argrepr='to 24', offset=72, starts_line=7, is_jump_target=True, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=74, starts_line=10, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=76, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='I can haz else clause?', argrepr="'I can haz else clause?'", offset=88, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=90, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=92, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=94, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=96, starts_line=11, is_jump_target=True, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=78, argval=156, argrepr='to 156', offset=98, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=100, starts_line=12, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=102, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=114, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=116, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=118, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=120, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=122, starts_line=13, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=124, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='BINARY_OP', opcode=122, arg=23, argval=23, argrepr='-=', offset=126, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=130, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=132, starts_line=14, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=134, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=136, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=71, argval=142, argrepr='to 142', offset=138, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=48, argval=96, argrepr='to 96', offset=140, starts_line=15, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=142, starts_line=16, is_jump_target=True, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=144, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=146, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=76, argval=152, argrepr='to 152', offset=148, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=13, argval=178, argrepr='to 178', offset=150, starts_line=17, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=152, starts_line=11, is_jump_target=True, positions=None), + Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=50, argval=100, argrepr='to 100', offset=154, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=156, starts_line=19, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=158, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=170, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=172, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=174, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=176, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=178, starts_line=20, is_jump_target=True, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=180, starts_line=21, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=182, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='BINARY_OP', opcode=122, arg=11, argval=11, argrepr='/', offset=184, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=190, starts_line=25, is_jump_target=False, positions=None), + Instruction(opname='BEFORE_WITH', opcode=53, arg=None, argval=None, argrepr='', offset=192, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=194, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=196, starts_line=26, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=198, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Never reach this', argrepr="'Never reach this'", offset=210, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=212, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=214, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=216, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=218, starts_line=25, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=220, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=222, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=2, argval=2, argrepr='', offset=224, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=2, argval=2, argrepr='', offset=226, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=228, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=254, argrepr='to 254', offset=230, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=232, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=234, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=123, argval=246, argrepr='to 246', offset=236, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=2, argval=2, argrepr='', offset=238, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=240, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=242, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=244, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=246, starts_line=None, is_jump_target=True, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=248, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=250, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=252, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=26, argval=308, argrepr='to 308', offset=254, starts_line=None, is_jump_target=True, positions=None), + Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=256, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=258, starts_line=22, is_jump_target=False, positions=None), + Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=150, argval=300, argrepr='to 300', offset=270, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=272, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=274, starts_line=23, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=276, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=288, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=290, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=292, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=294, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=296, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='JUMP_FORWARD', opcode=110, arg=17, argval=334, argrepr='to 334', offset=298, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=300, starts_line=22, is_jump_target=True, positions=None), + Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=302, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=304, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=306, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=308, starts_line=28, is_jump_target=True, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=310, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=322, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=324, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=326, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=328, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=330, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=332, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=334, starts_line=23, is_jump_target=True, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=336, starts_line=28, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=338, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=350, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=352, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=354, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=356, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=358, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=360, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=362, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PUSH_NULL', opcode=2, arg=None, argval=None, argrepr='', offset=364, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=366, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=378, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='PRECALL', opcode=166, arg=1, argval=1, argrepr='', offset=380, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='CALL', opcode=171, arg=1, argval=1, argrepr='', offset=382, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=384, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=386, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='COPY', opcode=120, arg=3, argval=3, argrepr='', offset=388, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=390, starts_line=None, is_jump_target=False, positions=None), + Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=392, starts_line=None, is_jump_target=False, positions=None), ] # One last piece of inspect fodder to check the default line number handling diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-02-25-13-18-18.bpo-46841.86QiQu.rst b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-13-18-18.bpo-46841.86QiQu.rst new file mode 100644 index 0000000000000..de8261fe4784e --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-02-25-13-18-18.bpo-46841.86QiQu.rst @@ -0,0 +1 @@ +Use inline cache for :opcode:`LOAD_GLOBAL`. diff --git a/Python/ceval.c b/Python/ceval.c index fe75782972992..6f1165b768041 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2993,25 +2993,26 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int } } } + /* Skip over inline cache */ + JUMPBY(INLINE_CACHE_ENTRIES_LOAD_GLOBAL); PUSH(v); DISPATCH(); } TARGET(LOAD_GLOBAL_ADAPTIVE) { assert(cframe.use_tracing == 0); - SpecializedCacheEntry *cache = GET_CACHE(); - if (cache->adaptive.counter == 0) { - PyObject *name = GETITEM(names, cache->adaptive.original_oparg); + uint16_t counter = *next_instr; + if (counter == 0) { + PyObject *name = GETITEM(names, oparg); next_instr--; - if (_Py_Specialize_LoadGlobal(GLOBALS(), BUILTINS(), next_instr, name, cache) < 0) { + if (_Py_Specialize_LoadGlobal(GLOBALS(), BUILTINS(), next_instr, name) < 0) { goto error; } DISPATCH(); } else { STAT_INC(LOAD_GLOBAL, deferred); - cache->adaptive.counter--; - oparg = cache->adaptive.original_oparg; + *next_instr = counter-1; JUMP_TO_INSTRUCTION(LOAD_GLOBAL); } } @@ -3020,13 +3021,13 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int assert(cframe.use_tracing == 0); DEOPT_IF(!PyDict_CheckExact(GLOBALS()), LOAD_GLOBAL); PyDictObject *dict = (PyDictObject *)GLOBALS(); - SpecializedCacheEntry *caches = GET_CACHE(); - _PyAdaptiveEntry *cache0 = &caches[0].adaptive; - _PyLoadGlobalCache *cache1 = &caches[-1].load_global; - DEOPT_IF(dict->ma_keys->dk_version != cache1->module_keys_version, LOAD_GLOBAL); - PyDictKeyEntry *ep = DK_ENTRIES(dict->ma_keys) + cache0->index; + _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; + uint32_t version = read32(&cache->module_keys_version); + DEOPT_IF(dict->ma_keys->dk_version != version, LOAD_GLOBAL); + PyDictKeyEntry *ep = DK_ENTRIES(dict->ma_keys) + cache->index; PyObject *res = ep->me_value; DEOPT_IF(res == NULL, LOAD_GLOBAL); + JUMPBY(INLINE_CACHE_ENTRIES_LOAD_GLOBAL); STAT_INC(LOAD_GLOBAL, hit); Py_INCREF(res); PUSH(res); @@ -3039,14 +3040,15 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int DEOPT_IF(!PyDict_CheckExact(BUILTINS()), LOAD_GLOBAL); PyDictObject *mdict = (PyDictObject *)GLOBALS(); PyDictObject *bdict = (PyDictObject *)BUILTINS(); - SpecializedCacheEntry *caches = GET_CACHE(); - _PyAdaptiveEntry *cache0 = &caches[0].adaptive; - _PyLoadGlobalCache *cache1 = &caches[-1].load_global; - DEOPT_IF(mdict->ma_keys->dk_version != cache1->module_keys_version, LOAD_GLOBAL); - DEOPT_IF(bdict->ma_keys->dk_version != cache1->builtin_keys_version, LOAD_GLOBAL); - PyDictKeyEntry *ep = DK_ENTRIES(bdict->ma_keys) + cache0->index; + _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)next_instr; + uint32_t mod_version = read32(&cache->module_keys_version); + uint16_t bltn_version = cache->builtin_keys_version; + DEOPT_IF(mdict->ma_keys->dk_version != mod_version, LOAD_GLOBAL); + DEOPT_IF(bdict->ma_keys->dk_version != bltn_version, LOAD_GLOBAL); + PyDictKeyEntry *ep = DK_ENTRIES(bdict->ma_keys) + cache->index; PyObject *res = ep->me_value; DEOPT_IF(res == NULL, LOAD_GLOBAL); + JUMPBY(INLINE_CACHE_ENTRIES_LOAD_GLOBAL); STAT_INC(LOAD_GLOBAL, hit); Py_INCREF(res); PUSH(res); @@ -5594,7 +5596,7 @@ opname ## _miss: \ MISS_WITH_CACHE(LOAD_ATTR) MISS_WITH_CACHE(STORE_ATTR) -MISS_WITH_CACHE(LOAD_GLOBAL) +MISS_WITH_INLINE_CACHE(LOAD_GLOBAL) MISS_WITH_CACHE(LOAD_METHOD) MISS_WITH_CACHE(PRECALL) MISS_WITH_CACHE(CALL) diff --git a/Python/compile.c b/Python/compile.c index 8ae4d44e9c6f3..ea9c3c8d48269 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -31,7 +31,7 @@ #include "pycore_long.h" // _PyLong_GetZero() #include "pycore_symtable.h" // PySTEntryObject -#define NEED_OPCODE_JUMP_TABLES +#define NEED_OPCODE_TABLES #include "opcode.h" // EXTENDED_ARG @@ -108,7 +108,7 @@ typedef struct exceptstack { #define MASK_LOW_LOG_BITS 31 static inline int -is_bit_set_in_table(uint32_t *table, int bitindex) { +is_bit_set_in_table(const uint32_t *table, int bitindex) { /* Is the relevant bit set in the relevant word? */ /* 256 bits fit into 8 32-bits words. * Word is indexed by (bitindex>>ln(size of int in bits)). diff --git a/Python/specialize.c b/Python/specialize.c index b88c5d517bd8f..38907f675e809 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -59,7 +59,6 @@ static uint8_t adaptive_opcodes[256] = { /* The number of cache entries required for a "family" of instructions. */ static uint8_t cache_requirements[256] = { [LOAD_ATTR] = 1, // _PyAdaptiveEntry - [LOAD_GLOBAL] = 2, /* _PyAdaptiveEntry and _PyLoadGlobalCache */ [LOAD_METHOD] = 3, /* _PyAdaptiveEntry, _PyAttrCache and _PyObjectCache */ [BINARY_SUBSCR] = 2, /* _PyAdaptiveEntry, _PyObjectCache */ [STORE_SUBSCR] = 0, @@ -1208,11 +1207,12 @@ _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, int _Py_Specialize_LoadGlobal( PyObject *globals, PyObject *builtins, - _Py_CODEUNIT *instr, PyObject *name, - SpecializedCacheEntry *cache) + _Py_CODEUNIT *instr, PyObject *name) { - _PyAdaptiveEntry *cache0 = &cache->adaptive; - _PyLoadGlobalCache *cache1 = &cache[-1].load_global; + assert(_PyOpcode_InlineCacheEntries[LOAD_GLOBAL] == + INLINE_CACHE_ENTRIES_LOAD_GLOBAL); + /* Use inline cache */ + _PyLoadGlobalCache *cache = (_PyLoadGlobalCache *)(instr + 1); assert(PyUnicode_CheckExact(name)); if (!PyDict_CheckExact(globals)) { goto fail; @@ -1231,8 +1231,8 @@ _Py_Specialize_LoadGlobal( if (keys_version == 0) { goto fail; } - cache1->module_keys_version = keys_version; - cache0->index = (uint16_t)index; + cache->index = (uint16_t)index; + write32(&cache->module_keys_version, keys_version); *instr = _Py_MAKECODEUNIT(LOAD_GLOBAL_MODULE, _Py_OPARG(*instr)); goto success; } @@ -1258,20 +1258,24 @@ _Py_Specialize_LoadGlobal( SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_OUT_OF_VERSIONS); goto fail; } - cache1->module_keys_version = globals_version; - cache1->builtin_keys_version = builtins_version; - cache0->index = (uint16_t)index; + if (builtins_version > UINT16_MAX) { + SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_OUT_OF_RANGE); + goto fail; + } + cache->index = (uint16_t)index; + write32(&cache->module_keys_version, globals_version); + cache->builtin_keys_version = (uint16_t)builtins_version; *instr = _Py_MAKECODEUNIT(LOAD_GLOBAL_BUILTIN, _Py_OPARG(*instr)); goto success; fail: STAT_INC(LOAD_GLOBAL, failure); assert(!PyErr_Occurred()); - cache_backoff(cache0); + cache->counter = ADAPTIVE_CACHE_BACKOFF; return 0; success: STAT_INC(LOAD_GLOBAL, success); assert(!PyErr_Occurred()); - cache0->counter = initial_counter_value(); + cache->counter = initial_counter_value(); return 0; } diff --git a/Tools/scripts/generate_opcode_h.py b/Tools/scripts/generate_opcode_h.py index 1f21347935d83..75a9c3f3bfadb 100644 --- a/Tools/scripts/generate_opcode_h.py +++ b/Tools/scripts/generate_opcode_h.py @@ -36,7 +36,7 @@ def write_int_array_from_ops(name, ops, out): bits = 0 for op in ops: bits |= 1<>= 32 @@ -75,9 +75,16 @@ def main(opcode_py, outfile='Include/opcode.h'): fobj.write(DEFINE.format(name, next_op)) used[next_op] = True fobj.write(DEFINE.format('DO_TRACING', 255)) - fobj.write("#ifdef NEED_OPCODE_JUMP_TABLES\n") + fobj.write("\nextern const uint8_t _PyOpcode_InlineCacheEntries[256];\n") + fobj.write("\n#ifdef NEED_OPCODE_TABLES\n") write_int_array_from_ops("_PyOpcode_RelativeJump", opcode['hasjrel'], fobj) write_int_array_from_ops("_PyOpcode_Jump", opcode['hasjrel'] + opcode['hasjabs'], fobj) + + fobj.write("\nconst uint8_t _PyOpcode_InlineCacheEntries[256] = {\n") + for i, entries in enumerate(opcode["_inline_cache_entries"]): + if entries: + fobj.write(f" [{opname[i]}] = {entries},\n") + fobj.write("};\n") fobj.write("#endif /* OPCODE_TABLES */\n") fobj.write("\n") @@ -90,12 +97,6 @@ def main(opcode_py, outfile='Include/opcode.h'): for i, (op, _) in enumerate(opcode["_nb_ops"]): fobj.write(DEFINE.format(op, i)) - fobj.write("\nstatic const uint8_t _PyOpcode_InlineCacheEntries[256] = {\n") - for i, entries in enumerate(opcode["_inline_cache_entries"]): - if entries: - fobj.write(f" [{opname[i]}] = {entries},\n") - fobj.write("};\n") - fobj.write(footer) From webhook-mailer at python.org Mon Feb 28 10:04:17 2022 From: webhook-mailer at python.org (vstinner) Date: Mon, 28 Feb 2022 15:04:17 -0000 Subject: [Python-checkins] bpo-45431: Rename CFrame to _PyCFrame in the C API (GH-31584) Message-ID: https://github.com/python/cpython/commit/7496f9587306772b56ed074092c020f3ef16bf95 commit: 7496f9587306772b56ed074092c020f3ef16bf95 branch: main author: Victor Stinner committer: vstinner date: 2022-02-28T16:03:57+01:00 summary: bpo-45431: Rename CFrame to _PyCFrame in the C API (GH-31584) Rename also struct _cframe to struct _PyCFrame. Add a comment suggesting using public functions rather than using directly the private _PyCFrame structure. files: M Include/cpython/pystate.h M Include/internal/pycore_frame.h M Python/ceval.c diff --git a/Include/cpython/pystate.h b/Include/cpython/pystate.h index 0d38604636b2b..26d6f7576e524 100644 --- a/Include/cpython/pystate.h +++ b/Include/cpython/pystate.h @@ -33,7 +33,9 @@ typedef struct { PyCodeAddressRange bounds; // Only valid if code != NULL. } PyTraceInfo; -typedef struct _cframe { +// Internal structure: you should not use it directly, but use public functions +// like PyThreadState_EnterTracing() and PyThreadState_LeaveTracing(). +typedef struct _PyCFrame { /* This struct will be threaded through the C stack * allowing fast access to per-thread state that needs * to be accessed quickly by the interpreter, but can @@ -47,8 +49,8 @@ typedef struct _cframe { int use_tracing; /* Pointer to the currently executing frame (it can be NULL) */ struct _PyInterpreterFrame *current_frame; - struct _cframe *previous; -} CFrame; + struct _PyCFrame *previous; +} _PyCFrame; typedef struct _err_stackitem { /* This struct represents a single execution context where we might @@ -102,9 +104,9 @@ struct _ts { the trace/profile. */ int tracing; - /* Pointer to current CFrame in the C stack frame of the currently, + /* Pointer to current _PyCFrame in the C stack frame of the currently, * or most recently, executing _PyEval_EvalFrameDefault. */ - CFrame *cframe; + _PyCFrame *cframe; Py_tracefunc c_profilefunc; Py_tracefunc c_tracefunc; @@ -196,7 +198,7 @@ struct _ts { _PyErr_StackItem exc_state; /* The bottom-most frame on the stack. */ - CFrame root_cframe; + _PyCFrame root_cframe; }; diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h index 20e81b7849b20..207983dcc22d7 100644 --- a/Include/internal/pycore_frame.h +++ b/Include/internal/pycore_frame.h @@ -59,7 +59,7 @@ typedef struct _PyInterpreterFrame { int f_lasti; /* Last instruction if called */ int stacktop; /* Offset of TOS from localsplus */ PyFrameState f_state; /* What state the frame is in */ - bool is_entry; // Whether this is the "root" frame for the current CFrame. + bool is_entry; // Whether this is the "root" frame for the current _PyCFrame. bool is_generator; PyObject *localsplus[1]; } _PyInterpreterFrame; diff --git a/Python/ceval.c b/Python/ceval.c index 6f1165b768041..13866ba355e97 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1616,15 +1616,15 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int int oparg; /* Current opcode argument, if any */ _Py_atomic_int * const eval_breaker = &tstate->interp->ceval.eval_breaker; - CFrame cframe; + _PyCFrame cframe; CallShape call_shape; call_shape.kwnames = NULL; // Borrowed reference. Reset by CALL instructions. - /* WARNING: Because the CFrame lives on the C stack, + /* WARNING: Because the _PyCFrame lives on the C stack, * but can be accessed from a heap allocated object (tstate) * strict stack discipline must be maintained. */ - CFrame *prev_cframe = tstate->cframe; + _PyCFrame *prev_cframe = tstate->cframe; cframe.use_tracing = prev_cframe->use_tracing; cframe.previous = prev_cframe; tstate->cframe = &cframe; From webhook-mailer at python.org Mon Feb 28 12:44:02 2022 From: webhook-mailer at python.org (rhettinger) Date: Mon, 28 Feb 2022 17:44:02 -0000 Subject: [Python-checkins] Statistics internals: Make fewer calls to _coerce() when data types are mixed (GH-31619) Message-ID: https://github.com/python/cpython/commit/d5b7bba43b7d9559e9894d263ece337084dc4b8d commit: d5b7bba43b7d9559e9894d263ece337084dc4b8d branch: main author: Raymond Hettinger committer: rhettinger date: 2022-02-28T11:43:52-06:00 summary: Statistics internals: Make fewer calls to _coerce() when data types are mixed (GH-31619) files: M Lib/statistics.py diff --git a/Lib/statistics.py b/Lib/statistics.py index eef2453bc7394..f3bc9a1f34639 100644 --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -137,6 +137,7 @@ from itertools import groupby, repeat from bisect import bisect_left, bisect_right from math import hypot, sqrt, fabs, exp, erf, tau, log, fsum +from functools import reduce from operator import mul from collections import Counter, namedtuple, defaultdict @@ -183,11 +184,12 @@ def _sum(data): allowed. """ count = 0 + types = set() + types_add = types.add partials = {} partials_get = partials.get - T = int for typ, values in groupby(data, type): - T = _coerce(T, typ) # or raise TypeError + types_add(typ) for n, d in map(_exact_ratio, values): count += 1 partials[d] = partials_get(d, 0) + n @@ -199,6 +201,7 @@ def _sum(data): else: # Sum all the partial sums using builtin sum. total = sum(Fraction(n, d) for d, n in partials.items()) + T = reduce(_coerce, types, int) # or raise TypeError return (T, total, count) @@ -214,11 +217,12 @@ def _ss(data, c=None): T, total, count = _sum((d := x - c) * d for x in data) return (T, total, count) count = 0 + types = set() + types_add = types.add sx_partials = defaultdict(int) sxx_partials = defaultdict(int) - T = int for typ, values in groupby(data, type): - T = _coerce(T, typ) # or raise TypeError + types_add(typ) for n, d in map(_exact_ratio, values): count += 1 sx_partials[d] += n @@ -236,6 +240,7 @@ def _ss(data, c=None): # This formula has poor numeric properties for floats, # but with fractions it is exact. total = (count * sxx - sx * sx) / count + T = reduce(_coerce, types, int) # or raise TypeError return (T, total, count) From webhook-mailer at python.org Mon Feb 28 17:15:57 2022 From: webhook-mailer at python.org (ericsnowcurrently) Date: Mon, 28 Feb 2022 22:15:57 -0000 Subject: [Python-checkins] bpo-46753: Add the empty tuple to the _PyRuntimeState.global_objects. (gh-31345) Message-ID: https://github.com/python/cpython/commit/08deed1af56bec8668c6cb4d5cfd89e393e1fe5e commit: 08deed1af56bec8668c6cb4d5cfd89e393e1fe5e branch: main author: Eric Snow committer: ericsnowcurrently date: 2022-02-28T15:15:48-07:00 summary: bpo-46753: Add the empty tuple to the _PyRuntimeState.global_objects. (gh-31345) https://bugs.python.org/issue46753 files: M Include/internal/pycore_gc.h M Include/internal/pycore_global_objects.h M Include/internal/pycore_runtime_init.h M Include/internal/pycore_tuple.h M Objects/tupleobject.c M Python/pylifecycle.c M Tools/scripts/deepfreeze.py M Tools/scripts/generate_global_objects.py diff --git a/Include/internal/pycore_gc.h b/Include/internal/pycore_gc.h index 56a23e9970752..16c1893639f35 100644 --- a/Include/internal/pycore_gc.h +++ b/Include/internal/pycore_gc.h @@ -20,6 +20,7 @@ typedef struct { } PyGC_Head; #define _Py_AS_GC(o) ((PyGC_Head *)(o)-1) +#define _PyGC_Head_UNUSED PyGC_Head /* True if the object is currently tracked by the GC. */ #define _PyObject_GC_IS_TRACKED(o) (_Py_AS_GC(o)->_gc_next != 0) diff --git a/Include/internal/pycore_global_objects.h b/Include/internal/pycore_global_objects.h index 2135fa3b5d79a..98673d4efcedc 100644 --- a/Include/internal/pycore_global_objects.h +++ b/Include/internal/pycore_global_objects.h @@ -8,6 +8,7 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif +#include "pycore_gc.h" // PyGC_Head #include "pycore_global_strings.h" // struct _Py_global_strings @@ -40,6 +41,9 @@ struct _Py_global_objects { } bytes_characters[256]; struct _Py_global_strings strings; + + _PyGC_Head_UNUSED _tuple_empty_gc_not_used; + PyTupleObject tuple_empty; } singletons; }; diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h index 8f62343443351..2f2bc65cd7111 100644 --- a/Include/internal/pycore_runtime_init.h +++ b/Include/internal/pycore_runtime_init.h @@ -964,6 +964,10 @@ extern "C" { INIT_ID(zipimporter), \ }, \ }, \ + \ + .tuple_empty = { \ + .ob_base = _PyVarObject_IMMORTAL_INIT(&PyTuple_Type, 0) \ + }, \ }, \ } /* End auto-generated code */ diff --git a/Include/internal/pycore_tuple.h b/Include/internal/pycore_tuple.h index 624c21caec1cc..1efe4fa2bdef9 100644 --- a/Include/internal/pycore_tuple.h +++ b/Include/internal/pycore_tuple.h @@ -20,30 +20,45 @@ extern void _PyTuple_Fini(PyInterpreterState *); /* other API */ -#ifndef WITH_FREELISTS -// without freelists -// for tuples only store empty tuple singleton -# define PyTuple_MAXSAVESIZE 1 -# define PyTuple_MAXFREELIST 1 -#endif +// PyTuple_MAXSAVESIZE - largest tuple to save on free list +// PyTuple_MAXFREELIST - maximum number of tuples of each size to save -/* Speed optimization to avoid frequent malloc/free of small tuples */ -#ifndef PyTuple_MAXSAVESIZE - // Largest tuple to save on free list -# define PyTuple_MAXSAVESIZE 20 -#endif -#ifndef PyTuple_MAXFREELIST - // Maximum number of tuples of each size to save -# define PyTuple_MAXFREELIST 2000 +#if defined(PyTuple_MAXSAVESIZE) && PyTuple_MAXSAVESIZE <= 0 + // A build indicated that tuple freelists should not be used. +# define PyTuple_NFREELISTS 0 +# undef PyTuple_MAXSAVESIZE +# undef PyTuple_MAXFREELIST + +#elif !defined(WITH_FREELISTS) +# define PyTuple_NFREELISTS 0 +# undef PyTuple_MAXSAVESIZE +# undef PyTuple_MAXFREELIST + +#else + // We are using a freelist for tuples. +# ifndef PyTuple_MAXSAVESIZE +# define PyTuple_MAXSAVESIZE 20 +# endif +# define PyTuple_NFREELISTS PyTuple_MAXSAVESIZE +# ifndef PyTuple_MAXFREELIST +# define PyTuple_MAXFREELIST 2000 +# endif #endif struct _Py_tuple_state { -#if PyTuple_MAXSAVESIZE > 0 - /* Entries 1 up to PyTuple_MAXSAVESIZE are free lists, - entry 0 is the empty tuple () of which at most one instance - will be allocated. */ - PyTupleObject *free_list[PyTuple_MAXSAVESIZE]; - int numfree[PyTuple_MAXSAVESIZE]; +#if PyTuple_NFREELISTS > 0 + /* There is one freelist for each size from 1 to PyTuple_MAXSAVESIZE. + The empty tuple is handled separately. + + Each tuple stored in the array is the head of the linked list + (and the next available tuple) for that size. The actual tuple + object is used as the linked list node, with its first item + (ob_item[0]) pointing to the next node (i.e. the previous head). + Each linked list is initially NULL. */ + PyTupleObject *free_list[PyTuple_NFREELISTS]; + int numfree[PyTuple_NFREELISTS]; +#else + char _unused; // Empty structs are not allowed. #endif }; diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c index ec9147cdcfedf..306a9112b0600 100644 --- a/Objects/tupleobject.c +++ b/Objects/tupleobject.c @@ -7,7 +7,6 @@ #include "pycore_initconfig.h" // _PyStatus_OK() #include "pycore_object.h" // _PyObject_GC_TRACK() #include "pycore_pyerrors.h" // _Py_FatalRefcountError() -#include "pycore_tuple.h" // struct _Py_tuple_state() /*[clinic input] class tuple "PyTupleObject *" "&PyTuple_Type" @@ -17,31 +16,9 @@ class tuple "PyTupleObject *" "&PyTuple_Type" #include "clinic/tupleobject.c.h" -#if PyTuple_MAXSAVESIZE > 0 -static struct _Py_tuple_state * -get_tuple_state(void) -{ - PyInterpreterState *interp = _PyInterpreterState_GET(); - return &interp->tuple; -} -#endif - +static inline PyTupleObject * maybe_freelist_pop(Py_ssize_t); +static inline int maybe_freelist_push(PyTupleObject *); -/* Print summary info about the state of the optimized allocator */ -void -_PyTuple_DebugMallocStats(FILE *out) -{ -#if PyTuple_MAXSAVESIZE > 0 - struct _Py_tuple_state *state = get_tuple_state(); - for (int i = 1; i < PyTuple_MAXSAVESIZE; i++) { - char buf[128]; - PyOS_snprintf(buf, sizeof(buf), - "free %d-sized PyTupleObject", i); - _PyDebugAllocatorStats(out, buf, state->numfree[i], - _PyObject_VAR_SIZE(&PyTuple_Type, i)); - } -#endif -} /* Allocate an uninitialized tuple object. Before making it public, following steps must be done: @@ -56,38 +33,16 @@ _PyTuple_DebugMallocStats(FILE *out) static PyTupleObject * tuple_alloc(Py_ssize_t size) { - PyTupleObject *op; -#if PyTuple_MAXSAVESIZE > 0 - // If Python is built with the empty tuple singleton, - // tuple_alloc(0) must not be called. - assert(size != 0); -#endif if (size < 0) { PyErr_BadInternalCall(); return NULL; } - -// Check for max save size > 1. Empty tuple singleton is special case. -#if PyTuple_MAXSAVESIZE > 1 - struct _Py_tuple_state *state = get_tuple_state(); #ifdef Py_DEBUG - // tuple_alloc() must not be called after _PyTuple_Fini() - assert(state->numfree[0] != -1); + assert(size != 0); // The empty tuple is statically allocated. #endif - if (size < PyTuple_MAXSAVESIZE && (op = state->free_list[size]) != NULL) { - assert(size != 0); - state->free_list[size] = (PyTupleObject *) op->ob_item[0]; - state->numfree[size]--; - /* Inlined _PyObject_InitVar() without _PyType_HasFeature() test */ -#ifdef Py_TRACE_REFS - Py_SET_SIZE(op, size); - Py_SET_TYPE(op, &PyTuple_Type); -#endif - _Py_NewReference((PyObject *)op); - } - else -#endif - { + + PyTupleObject *op = maybe_freelist_pop(size); + if (op == NULL) { /* Check for overflow */ if ((size_t)size > ((size_t)PY_SSIZE_T_MAX - (sizeof(PyTupleObject) - sizeof(PyObject *))) / sizeof(PyObject *)) { @@ -100,58 +55,24 @@ tuple_alloc(Py_ssize_t size) return op; } -static int -tuple_create_empty_tuple_singleton(struct _Py_tuple_state *state) -{ -#if PyTuple_MAXSAVESIZE > 0 - assert(state->free_list[0] == NULL); +// The empty tuple singleton is not tracked by the GC. +// It does not contain any Python object. +// Note that tuple subclasses have their own empty instances. - PyTupleObject *op = PyObject_GC_NewVar(PyTupleObject, &PyTuple_Type, 0); - if (op == NULL) { - return -1; - } - // The empty tuple singleton is not tracked by the GC. - // It does not contain any Python object. - - state->free_list[0] = op; - state->numfree[0]++; - - assert(state->numfree[0] == 1); -#endif - return 0; -} - - -static PyObject * +static inline PyObject * tuple_get_empty(void) { -#if PyTuple_MAXSAVESIZE > 0 - struct _Py_tuple_state *state = get_tuple_state(); - PyTupleObject *op = state->free_list[0]; - // tuple_get_empty() must not be called before _PyTuple_Init() - // or after _PyTuple_Fini() - assert(op != NULL); -#ifdef Py_DEBUG - assert(state->numfree[0] != -1); -#endif - - Py_INCREF(op); - return (PyObject *) op; -#else - return PyTuple_New(0); -#endif + Py_INCREF(&_Py_SINGLETON(tuple_empty)); + return (PyObject *)&_Py_SINGLETON(tuple_empty); } - PyObject * PyTuple_New(Py_ssize_t size) { PyTupleObject *op; -#if PyTuple_MAXSAVESIZE > 0 if (size == 0) { return tuple_get_empty(); } -#endif op = tuple_alloc(size); if (op == NULL) { return NULL; @@ -265,47 +186,33 @@ PyTuple_Pack(Py_ssize_t n, ...) static void tupledealloc(PyTupleObject *op) { - Py_ssize_t len = Py_SIZE(op); - PyObject_GC_UnTrack(op); - Py_TRASHCAN_BEGIN(op, tupledealloc) - if (len > 0) { - Py_ssize_t i = len; - while (--i >= 0) { - Py_XDECREF(op->ob_item[i]); - } -#if PyTuple_MAXSAVESIZE > 0 - struct _Py_tuple_state *state = get_tuple_state(); + if (Py_SIZE(op) == 0) { + /* The empty tuple is statically allocated. */ + if (op == &_Py_SINGLETON(tuple_empty)) { #ifdef Py_DEBUG - // tupledealloc() must not be called after _PyTuple_Fini() - assert(state->numfree[0] != -1); + _Py_FatalRefcountError("deallocating the empty tuple singleton"); +#else + return; #endif - if (len < PyTuple_MAXSAVESIZE - && state->numfree[len] < PyTuple_MAXFREELIST - && Py_IS_TYPE(op, &PyTuple_Type)) - { - op->ob_item[0] = (PyObject *) state->free_list[len]; - state->numfree[len]++; - state->free_list[len] = op; - goto done; /* return */ } +#ifdef Py_DEBUG + /* tuple subclasses have their own empty instances. */ + assert(!PyTuple_CheckExact(op)); #endif } -#if defined(Py_DEBUG) && PyTuple_MAXSAVESIZE > 0 - else { - assert(len == 0); - struct _Py_tuple_state *state = get_tuple_state(); - // The empty tuple singleton must only be deallocated by - // _PyTuple_Fini(): not before, not after - if (op == state->free_list[0] && state->numfree[0] != 0) { - _Py_FatalRefcountError("deallocating the empty tuple singleton"); - } + + PyObject_GC_UnTrack(op); + Py_TRASHCAN_BEGIN(op, tupledealloc) + + Py_ssize_t i = Py_SIZE(op); + while (--i >= 0) { + Py_XDECREF(op->ob_item[i]); + } + // This will abort on the empty singleton (if there is one). + if (!maybe_freelist_push(op)) { + Py_TYPE(op)->tp_free((PyObject *)op); } -#endif - Py_TYPE(op)->tp_free((PyObject *)op); -#if PyTuple_MAXSAVESIZE > 0 -done: -#endif Py_TRASHCAN_END } @@ -838,6 +745,7 @@ tuple_subtype_new(PyTypeObject *type, PyObject *iterable) if (tmp == NULL) return NULL; assert(PyTuple_Check(tmp)); + /* This may allocate an empty tuple that is not the global one. */ newobj = type->tp_alloc(type, n = PyTuple_GET_SIZE(tmp)); if (newobj == NULL) { Py_DECREF(tmp); @@ -1020,14 +928,22 @@ _PyTuple_Resize(PyObject **pv, Py_ssize_t newsize) PyErr_BadInternalCall(); return -1; } + oldsize = Py_SIZE(v); - if (oldsize == newsize) + if (oldsize == newsize) { return 0; - + } + if (newsize == 0) { + Py_DECREF(v); + *pv = tuple_get_empty(); + return 0; + } if (oldsize == 0) { - /* Empty tuples are often shared, so we should never - resize them in-place even if we do own the only - (current) reference */ +#ifdef Py_DEBUG + assert(v == &_Py_SINGLETON(tuple_empty)); +#endif + /* The empty tuple is statically allocated so we never + resize it in-place. */ Py_DECREF(v); *pv = PyTuple_New(newsize); return *pv == NULL ? -1 : 0; @@ -1063,36 +979,6 @@ _PyTuple_Resize(PyObject **pv, Py_ssize_t newsize) return 0; } -void -_PyTuple_ClearFreeList(PyInterpreterState *interp) -{ -#if PyTuple_MAXSAVESIZE > 0 - struct _Py_tuple_state *state = &interp->tuple; - for (Py_ssize_t i = 1; i < PyTuple_MAXSAVESIZE; i++) { - PyTupleObject *p = state->free_list[i]; - state->free_list[i] = NULL; - state->numfree[i] = 0; - while (p) { - PyTupleObject *q = p; - p = (PyTupleObject *)(p->ob_item[0]); - PyObject_GC_Del(q); - } - } - // the empty tuple singleton is only cleared by _PyTuple_Fini() -#endif -} - - -PyStatus -_PyTuple_InitGlobalObjects(PyInterpreterState *interp) -{ - struct _Py_tuple_state *state = &interp->tuple; - if (tuple_create_empty_tuple_singleton(state) < 0) { - return _PyStatus_NO_MEMORY(); - } - return _PyStatus_OK(); -} - PyStatus _PyTuple_InitTypes(PyInterpreterState *interp) @@ -1112,24 +998,18 @@ _PyTuple_InitTypes(PyInterpreterState *interp) return _PyStatus_OK(); } +static void maybe_freelist_clear(PyInterpreterState *, int); + void _PyTuple_Fini(PyInterpreterState *interp) { -#if PyTuple_MAXSAVESIZE > 0 - struct _Py_tuple_state *state = &interp->tuple; - // The empty tuple singleton must not be tracked by the GC - assert(!_PyObject_GC_IS_TRACKED(state->free_list[0])); - -#ifdef Py_DEBUG - state->numfree[0] = 0; -#endif - Py_CLEAR(state->free_list[0]); -#ifdef Py_DEBUG - state->numfree[0] = -1; -#endif + maybe_freelist_clear(interp, 1); +} - _PyTuple_ClearFreeList(interp); -#endif +void +_PyTuple_ClearFreeList(PyInterpreterState *interp) +{ + maybe_freelist_clear(interp, 0); } /*********************** Tuple Iterator **************************/ @@ -1277,3 +1157,113 @@ tuple_iter(PyObject *seq) _PyObject_GC_TRACK(it); return (PyObject *)it; } + + +/************* + * freelists * + *************/ + +#define STATE (interp->tuple) +#define FREELIST_FINALIZED (STATE.numfree[0] < 0) + +static inline PyTupleObject * +maybe_freelist_pop(Py_ssize_t size) +{ +#if PyTuple_NFREELISTS > 0 + PyInterpreterState *interp = _PyInterpreterState_GET(); +#ifdef Py_DEBUG + /* maybe_freelist_pop() must not be called after maybe_freelist_fini(). */ + assert(!FREELIST_FINALIZED); +#endif + if (size == 0) { + return NULL; + } + assert(size > 0); + if (size < PyTuple_MAXSAVESIZE) { + Py_ssize_t index = size - 1; + PyTupleObject *op = STATE.free_list[index]; + if (op != NULL) { + /* op is the head of a linked list, with the first item + pointing to the next node. Here we pop off the old head. */ + STATE.free_list[index] = (PyTupleObject *) op->ob_item[0]; + STATE.numfree[index]--; + /* Inlined _PyObject_InitVar() without _PyType_HasFeature() test */ +#ifdef Py_TRACE_REFS + /* maybe_freelist_push() ensures these were already set. */ + // XXX Can we drop these? See commit 68055ce6fe01 (GvR, Dec 1998). + Py_SET_SIZE(op, size); + Py_SET_TYPE(op, &PyTuple_Type); +#endif + _Py_NewReference((PyObject *)op); + /* END inlined _PyObject_InitVar() */ + return op; + } + } +#endif + return NULL; +} + +static inline int +maybe_freelist_push(PyTupleObject *op) +{ +#if PyTuple_NFREELISTS > 0 + PyInterpreterState *interp = _PyInterpreterState_GET(); +#ifdef Py_DEBUG + /* maybe_freelist_push() must not be called after maybe_freelist_fini(). */ + assert(!FREELIST_FINALIZED); +#endif + if (Py_SIZE(op) == 0) { + return 0; + } + Py_ssize_t index = Py_SIZE(op) - 1; + if (index < PyTuple_NFREELISTS + && STATE.numfree[index] < PyTuple_MAXFREELIST + && Py_IS_TYPE(op, &PyTuple_Type)) + { + /* op is the head of a linked list, with the first item + pointing to the next node. Here we set op as the new head. */ + op->ob_item[0] = (PyObject *) STATE.free_list[index]; + STATE.free_list[index] = op; + STATE.numfree[index]++; + return 1; + } +#endif + return 0; +} + +static void +maybe_freelist_clear(PyInterpreterState *interp, int fini) +{ +#if PyTuple_NFREELISTS > 0 + for (Py_ssize_t i = 0; i < PyTuple_NFREELISTS; i++) { + PyTupleObject *p = STATE.free_list[i]; + STATE.free_list[i] = NULL; + STATE.numfree[i] = fini ? -1 : 0; + while (p) { + PyTupleObject *q = p; + p = (PyTupleObject *)(p->ob_item[0]); + PyObject_GC_Del(q); + } + } +#endif +} + +/* Print summary info about the state of the optimized allocator */ +void +_PyTuple_DebugMallocStats(FILE *out) +{ +#if PyTuple_NFREELISTS > 0 + PyInterpreterState *interp = _PyInterpreterState_GET(); + for (int i = 0; i < PyTuple_NFREELISTS; i++) { + int len = i + 1; + char buf[128]; + PyOS_snprintf(buf, sizeof(buf), + "free %d-sized PyTupleObject", len); + _PyDebugAllocatorStats(out, buf, STATE.numfree[i], + _PyObject_VAR_SIZE(&PyTuple_Type, len)); + } +#endif +} + +#undef STATE +#undef FREELIST_FINALIZED diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 61534742005a3..9228778d6bd04 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -682,11 +682,6 @@ pycore_init_global_objects(PyInterpreterState *interp) _PyUnicode_InitState(interp); - status = _PyTuple_InitGlobalObjects(interp); - if (_PyStatus_EXCEPTION(status)) { - return status; - } - return _PyStatus_OK(); } diff --git a/Tools/scripts/deepfreeze.py b/Tools/scripts/deepfreeze.py index cdd34ddb8b1e8..954fca81b51e9 100644 --- a/Tools/scripts/deepfreeze.py +++ b/Tools/scripts/deepfreeze.py @@ -287,6 +287,8 @@ def generate_code(self, name: str, code: types.CodeType) -> str: return f"& {name}.ob_base" def generate_tuple(self, name: str, t: Tuple[object, ...]) -> str: + if len(t) == 0: + return f"(PyObject *)& _Py_SINGLETON(tuple_empty)" items = [self.generate(f"{name}_{i}", it) for i, it in enumerate(t)] self.write("static") with self.indent(): diff --git a/Tools/scripts/generate_global_objects.py b/Tools/scripts/generate_global_objects.py index 506aa86575c4e..639d8fa91c68b 100644 --- a/Tools/scripts/generate_global_objects.py +++ b/Tools/scripts/generate_global_objects.py @@ -252,6 +252,9 @@ def generate_runtime_init(identifiers, strings): for name in sorted(identifiers): assert name.isidentifier(), name printer.write(f'INIT_ID({name}),') + printer.write('') + with printer.block('.tuple_empty =', ','): + printer.write('.ob_base = _PyVarObject_IMMORTAL_INIT(&PyTuple_Type, 0)') printer.write(END) printer.write(after) From webhook-mailer at python.org Mon Feb 28 18:16:09 2022 From: webhook-mailer at python.org (gvanrossum) Date: Mon, 28 Feb 2022 23:16:09 -0000 Subject: [Python-checkins] bpo-46771: Remove two controversial lines from Task.cancel() (GH-31623) Message-ID: https://github.com/python/cpython/commit/7d611b4cabaf7925f5f94daddf711d54aeae2cf9 commit: 7d611b4cabaf7925f5f94daddf711d54aeae2cf9 branch: main author: Guido van Rossum committer: gvanrossum date: 2022-02-28T15:15:56-08:00 summary: bpo-46771: Remove two controversial lines from Task.cancel() (GH-31623) Also from the _asyncio C accelerator module, and adjust one test that the change caused to fail. For more discussion see the discussion starting here: https://github.com/python/cpython/pull/31394#issuecomment-1053545331 (Basically, @asvetlov proposed to return False from cancel() when there is already a pending cancellation, and I went along, even though it wasn't necessary for the task group implementation, and @agronholm has come up with a counterexample that fails because of this change. So now I'm changing it back to the old semantics (but still bumping the counter) until we can have a proper discussion about this.) files: M Lib/asyncio/tasks.py M Lib/test/test_asyncio/test_tasks.py M Modules/_asynciomodule.c diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py index 38c23851102a8..059143fb9086f 100644 --- a/Lib/asyncio/tasks.py +++ b/Lib/asyncio/tasks.py @@ -205,8 +205,11 @@ def cancel(self, msg=None): if self.done(): return False self._num_cancels_requested += 1 - if self._num_cancels_requested > 1: - return False + # These two lines are controversial. See discussion starting at + # https://github.com/python/cpython/pull/31394#issuecomment-1053545331 + # Also remember that this is duplicated in _asynciomodule.c. + # if self._num_cancels_requested > 1: + # return False if self._fut_waiter is not None: if self._fut_waiter.cancel(msg=msg): # Leave self._fut_waiter; it may be a Task that diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index b30f8f56dfa72..950879204e703 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -514,7 +514,11 @@ async def task(): self.assertTrue(t.cancel()) self.assertTrue(t.cancelling()) self.assertIn(" cancelling ", repr(t)) - self.assertFalse(t.cancel()) + + # Since we commented out two lines from Task.cancel(), + # this t.cancel() call now returns True. + # self.assertFalse(t.cancel()) + self.assertTrue(t.cancel()) with self.assertRaises(asyncio.CancelledError): loop.run_until_complete(t) diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 88471239529a5..2a6c0b335ccfb 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2206,9 +2206,13 @@ _asyncio_Task_cancel_impl(TaskObj *self, PyObject *msg) } self->task_num_cancels_requested += 1; - if (self->task_num_cancels_requested > 1) { - Py_RETURN_FALSE; - } + + // These three lines are controversial. See discussion starting at + // https://github.com/python/cpython/pull/31394#issuecomment-1053545331 + // and corresponding code in tasks.py. + // if (self->task_num_cancels_requested > 1) { + // Py_RETURN_FALSE; + // } if (self->task_fut_waiter) { PyObject *res; From webhook-mailer at python.org Mon Feb 28 20:20:37 2022 From: webhook-mailer at python.org (brandtbucher) Date: Tue, 01 Mar 2022 01:20:37 -0000 Subject: [Python-checkins] Add missing "to" to two tp_flags notes (GH-31624) Message-ID: https://github.com/python/cpython/commit/422fdb37172c145043d4f1276adad43ff375f0d5 commit: 422fdb37172c145043d4f1276adad43ff375f0d5 branch: main author: Brandt Bucher committer: brandtbucher date: 2022-02-28T17:20:14-08:00 summary: Add missing "to" to two tp_flags notes (GH-31624) files: M Doc/c-api/typeobj.rst diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 3630990368cdf..4a54139c5d498 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1233,7 +1233,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are - mutually exclusive; it is an error enable both flags simultaneously. + mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** @@ -1255,7 +1255,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are - mutually exclusive; it is an error enable both flags simultaneously. + mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** From webhook-mailer at python.org Mon Feb 28 20:27:34 2022 From: webhook-mailer at python.org (methane) Date: Tue, 01 Mar 2022 01:27:34 -0000 Subject: [Python-checkins] compiler: Merge except_table and cnotab (GH-31614) Message-ID: https://github.com/python/cpython/commit/df9f7597559b6256924fcd3a1c3dc24cd5c5edaf commit: df9f7597559b6256924fcd3a1c3dc24cd5c5edaf branch: main author: Inada Naoki committer: methane date: 2022-03-01T10:27:20+09:00 summary: compiler: Merge except_table and cnotab (GH-31614) files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index ea9c3c8d48269..3609ff8f4fb9e 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8272,6 +8272,9 @@ assemble(struct compiler *c, int addNone) if (_PyBytes_Resize(&a.a_except_table, a.a_except_table_off) < 0) { goto error; } + if (!merge_const_one(c, &a.a_except_table)) { + goto error; + } if (!assemble_start_line_range(&a)) { return 0; } @@ -8293,6 +8296,9 @@ assemble(struct compiler *c, int addNone) if (_PyBytes_Resize(&a.a_cnotab, a.a_cnotab_off) < 0) { goto error; } + if (!merge_const_one(c, &a.a_cnotab)) { + goto error; + } if (_PyBytes_Resize(&a.a_bytecode, a.a_offset * sizeof(_Py_CODEUNIT)) < 0) { goto error; } From webhook-mailer at python.org Mon Feb 28 20:45:36 2022 From: webhook-mailer at python.org (miss-islington) Date: Tue, 01 Mar 2022 01:45:36 -0000 Subject: [Python-checkins] Add missing "to" to two tp_flags notes (GH-31624) Message-ID: https://github.com/python/cpython/commit/98e2ee60bd7f66be18fee3f5b9c47c473bbb160c commit: 98e2ee60bd7f66be18fee3f5b9c47c473bbb160c branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: miss-islington <31488909+miss-islington at users.noreply.github.com> date: 2022-02-28T17:45:26-08:00 summary: Add missing "to" to two tp_flags notes (GH-31624) (cherry picked from commit 422fdb37172c145043d4f1276adad43ff375f0d5) Co-authored-by: Brandt Bucher files: M Doc/c-api/typeobj.rst diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index f0dd3923e8085..93492200ea44e 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1232,7 +1232,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are - mutually exclusive; it is an error enable both flags simultaneously. + mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:** @@ -1254,7 +1254,7 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. note:: :const:`Py_TPFLAGS_MAPPING` and :const:`Py_TPFLAGS_SEQUENCE` are - mutually exclusive; it is an error enable both flags simultaneously. + mutually exclusive; it is an error to enable both flags simultaneously. **Inheritance:**